Testing, releases & troubleshooting

Use this section as your checklist before shipping changes or when diagnosing issues in Tmates Cloud or self-hosted environments.

Testing

• Unit tests – live under app/tests/unit. Run with pytest app/tests/unit -q after editing services, billing, or API logic.
• Integration tests – app/tests/integration spin up lightweight fakes for Supabase/OpenAI. Run them before touching Celery or agent registry logic.
• Agent tests – each agent folder can ship its own test suite (app/agents/<key>/tests). Run targeted suites when editing prompts or tools.
• CLI smoke tests – python run.py agent <key> --message "ping" ensures user context resolution, agent loading, and SDK glue still work.

Add regression coverage when:

• You change API contracts (schemas in app/api/schemas.py).
• You touch billing enforcement.
• You alter file storage or attachment handling.

Release workflow

1. Run tests locally – both pytest and linting (ruff if available).
2. Rebuild tmates – if manifests change, regenerate bundles with the internal agent bundle builder.
3. Update docs – Pinboard release notes or README changes keep users informed.
4. Deploy backend – push to your hosting target (Docker, Kubernetes, Fly.io, etc.). Workers must be redeployed alongside the API when agent code changes.
5. Verify – call /health, send a sample chat, and ensure WebSocket pushes arrive.

For Tmates Cloud deployments add Stripe test runs and Supabase migrations (SQL files in app/db/) to the checklist.

Troubleshooting guide

Symptom	Checks
API returns 503 for most routes	Confirm Supabase credentials; get_database_client raises when SUPABASE_URL or keys are missing.
WebSocket closes immediately	Token mismatch between user_id path param and JWT id claim (validate_websocket_token).
File uploads fail	Ensure FILE_STORAGE_BACKEND is set and the bucket/directory exists. For images, confirm the MIME type starts with image/.
Team Chat silent	Inspect team_chat_dispatcher logs; missing OpenAI credentials or exceeded ModelSettings tokens can cause declines.
Agents disabled unexpectedly	Billing plan may be over capacity; check GET /v1/billing/plan or Supabase subscription records.
Worker spam retries	Look for transient DB messages defined in TRANSIENT_DB_MARKERS inside app/worker/tasks.py. Fix connectivity to Supabase/Postgres.

Observability & logs

• Enable ENABLE_LOGFIRE or TEAM_CHAT_ENABLE_LOGFIRE to stream traces from dispatcher decisions.
• Set TEAM_CHAT_OPENAI_API_KEY / OPENAI_API_KEY to allow OpenAI Agents tracing.
• Use app/logger.py helpers (log(...)) instead of raw prints to keep output structured across uvicorn and Celery.

Document what you find in Pinboard or your incident tracker so the whole team learns from each outage.