Troubleshooting

Possible reasons:

1. MULTICA_SERVER_URL points to the wrong address - the default is ws://localhost:8080/ws, self-host should be changed to your own server address
2. Network/Firewall Blocking - daemon and server are not on the same network, or outbound is blocked
3. Token expired or invalid - you have never run multica login, or PAT is revoked
4. Server refused registration - the account you logged in with is not in the target workspace (register returns 403)
5. DNS resolution failed - hostname cannot be resolved on the daemon machine

How to check:

multica daemon logs --lines 100 # See daemon side errors
echo $MULTICA_SERVER_URL # Confirm address matching
curl -i http://<server-host>:8080/health # poke server directly
curl -i http://<server-host>:8080/readyz # Check together with DB + migration readiness
cat ~/.multica/config.json # Check if api_token exists
multica workspace list # Confirm that you are a member of the target workspace

How to fix: Correct the problem according to the above reasons. The two most common ones are change MULTICA_SERVER_URL to restart the daemon (multica daemon restart) and re-login (multica logout && multica login).

Possible reasons (arranged by trigger probability):

1. The agent’s concurrency upper limit is full - the agent’s max_concurrent_tasks (default 6) has been occupied by other running tasks
2. There is another task with the same agent on the same issue that has not yet ended - same agent × same issue to force serialization (to prevent repeated execution)
3. The agent has been archived - After being archived, new tasks can still be added to the queue, but they cannot be claimed and will be stuck with a 5-minute timeout (code-issue G-01)
4. Daemon has not registered the runtime in the current workspace - Restart the daemon or reselect the runtime in the UI
5. Daemon lost contact - no heartbeat in the last 45 seconds. daemon status looks online or it may have just lost contact.

How to check:

multica daemon status --output json # runtime list + last_seen_at
multica agent list # Check the archived status of the agent
multica issue show <issue-id> # View task history

On the server side (self-host), you can grep "no_tasks" / "no_capacity" to see the claim results.

How to fix:

• Full concurrency → Wait for existing tasks to finish running, or multica agent update <id> --max-concurrent-tasks 10 to increase the upper limit
• Serialize the same issue → wait for the previous task to end, or assign it to a different agent
• Agent is archived → multica agent restore <id>
• Runtime is not registered → multica daemon restart, the daemon will be re-registered

Possible reasons:

1. Origin verification failed - Your front-end domain name is not in the server's CORS whitelist. The default whitelist only contains localhost:3000/5173/5174, and self-host must be configured with FRONTEND_ORIGIN when going to the public network.
2. Protocol mismatch - The front-end uses https:// and requires wss://, HTTP uses ws://
3. Reverse proxy is not enabled WebSocket upgrade - Nginx / Envoy / HAProxy does not forward the Upgrade header by default
4. JWT cookie expired or lost - No re-login after 30 days of expiration

How to check:

• Browser DevTools → Network → Filter "WS" to see the connection status and status code
• grep "rejected origin" / "websocket" in the Server log - if it is an origin problem, it will be clearly written out
• curl -i http://<server-host>:8080/ws should return 101 Switching Protocols (requires Upgrade header)

How to fix:

• Origin error → Set FRONTEND_ORIGIN=https://multica.yourdomain.com (or comma-separated CORS_ALLOWED_ORIGINS) in the server's .env and restart the server
• Protocol mismatch → Make sure the protocol of FRONTEND_ORIGIN is consistent with the front end
• Reverse proxy → Add proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; to Nginx
• Cookie expired → Refresh the page and log in again

**First confirm which provider the server thinks it is using. ** Backend will print one of these three types when starting:

• EmailService: SMTP relay <host>:<port> from=<addr> —— SMTP (SMTP_HOST has a higher priority than Resend when it is not empty)
• EmailService: Resend API from=<addr> —— Go to Resend
• EmailService: DEV mode — codes printed to stdout … —— No provider is configured

docker compose -f docker-compose.selfhost.yml logs backend | grep "EmailService:"

If the line that should appear does not appear, it means that the environment variable has not entered the process - check .env and docker compose -f docker-compose.selfhost.yml exec backend env | grep -E 'RESEND_|SMTP_'. This line of startup log will not print any passwords.

Resend is the current provider

Possible reasons:

1. RESEND_API_KEY is not configured - the server will fall back silently and enter the verification code into its own stdout without reporting an error. Production deployment is easy
2. Resend API key is invalid/insufficient balance - the server log will have "failed to send verification code"
3. The domain name of RESEND_FROM_EMAIL is not verified by Resend - Resend will refuse to send it.
4. The email was sent but was judged as spam by the recipient's ISP - Check the Resend dashboard and spam directory

How to check:

• Search the Server log for "[DEV] Verification code for" - If there is one, it means that the Resend is not matched and the verification code is sent to stdout.
• Resend dashboard → Emails View sending records
• Confirm that the domain name of RESEND_FROM_EMAIL is in the "Verified Domains" list of Resend console

How to fix:

• API key is not configured → Follow the steps of [Login and Registration Configuration → How to configure Email](/vibe-coding-tools/multica/auth-setup#email--How does verification code login work) and restart the server after configuration
• The domain name is not verified → Go through the DNS verification process in the Resend console (add SPF / DKIM records)
• In emergency situations (such as internal testing) → copy the verification code printed by [DEV] from the server log

SMTP is the current providerThe SMTP path wraps each failure in stages, so the server log already tells you at which step the relay rejected the session. Search "failed to send verification email" / "failed to send invitation email" to see the specific errors:

Error log	Meaning	How to fix
smtp dial <host>:<port>: dial tcp …: connect: connection refused / i/o timeout	The backend container cannot connect to the relay - the host / port is wrong, the firewall is blocked, or the relay is not open	Confirm that it can be resolved and connected in the container: docker compose -f docker-compose.selfhost.yml exec backend nslookup <host> and nc -vz <host> <port>; Release the network from Multica host to relay
smtp starttls: x509: certificate signed by unknown authority (or certificate is not valid for any names)	The relay uses a private CA / self-signed certificate, which is not accepted by the container's trust store	Either install the CA into the container, or set SMTP_TLS_INSECURE=true after confirming that the relay is on a trusted network segment
smtp auth: 535 5.7.8 Authentication credentials invalid (or 534/530)	SMTP_USERNAME / SMTP_PASSWORD is incorrect, or the relay does not accept PLAIN authentication	Ask your mail administrator to review the service account credentials; Exchange anonymous internal relay should leave both blank
smtp MAIL FROM: 550 5.7.1 Client does not have permissions to send as this sender	relay does not accept RESEND_FROM_EMAIL as the envelope sender - Common "anonymous users not allowed" or DMARC alignment issues in Exchange	Change RESEND_FROM_EMAIL to a domain name accepted by relay; Grant the source IP on Exchange ms-Exch-SMTP-Accept-Any-Sender
smtp RCPT TO <addr>: 550 5.7.1 Unable to relay	The relay's receive connector does not allow this subnet to relay emails to external recipients (common when anonymous internal relays are sent to external domains)	Invite only internal domains, or add the Multica host's subnet to the Exchange "Anonymous Users → Relay" permission list

The error logs of MAIL FROM / RCPT TO / DATA all contain status codes returned by relay, which can be aligned with the logs on the Exchange / Postfix side. Verification codes and invitation tokens will not appear in these wrapped errors.

How to check:

• Search for "EmailService: SMTP relay" once when starting, and search for "failed to send" when running, depending on the specific stage.
• Test connectivity in the backend container: docker compose -f docker-compose.selfhost.yml exec backend sh -c 'nc -vz $SMTP_HOST $SMTP_PORT'
• Confirm that the environment variables have actually entered the process: docker compose -f docker-compose.selfhost.yml exec backend env | grep SMTP_ (This will bring out the password and only run on trusted terminals)

How to fix:

• The host / port is incorrect → change SMTP_HOST / SMTP_PORT and then restart backend; for supported relay modes, see [Login and Registration Configuration → Option B: SMTP relay] (/auth-setup)
• Certificate verification failed → Install the relay's CA into the container, or temporarily set SMTP_TLS_INSECURE=true on the trusted network segment
• Authentication failed → Review credentials; anonymous internal relay should leave both SMTP_USERNAME and SMTP_PASSWORD blank
• Unable to relay → Invite only internal domains, or grant relay permission to Multica host on Exchange receive connector

Possible reasons (mutually exclusive):

1. MULTICA_DEV_VERIFICATION_CODE is empty - fixed verification code is turned off by default
2. APP_ENV=production - This is the correct production configuration; fixed local test verification code will be ignored in production
3. The configured verification code is not 6 digits - this shortcut code only accepts 6 digits

How to check:

cat .env | grep -E 'APP_ENV|MULTICA_DEV_VERIFICATION_CODE'
docker exec <container> env | grep -E 'APP_ENV|MULTICA_DEV_VERIFICATION_CODE'

Check your email (including spam) to see if you have received the real verification code.

How to fix:

• Keep MULTICA_DEV_VERIFICATION_CODE empty in the production environment, and use the real verification code after configuring Resend.
• Local development or intranet testing can copy the generated verification code from the server log; if 888888 is required, set APP_ENV=development and MULTICA_DEV_VERIFICATION_CODE=888888. Do not enable fixed verification codes on public network instances (for details, see [Login and Registration Configuration → Fixed Local Test Verification Code] (/auth-setup#Fixed Local Test Verification Code))

Possible reasons:

1. rollup_task_usage_hourly() is not claimed —— The Usage/Runtime dashboard reads the derived table task_usage_hourly, which must be filled periodically by rollup_task_usage_hourly(). Starting from MUL-2957, the backend runs rollup in-process through the DB backend scheduler (sys_cron_executions); old versions of binary, migration 113 not applied, or all replicas being offline for a long time may cause this table to have no recent SUCCESS rows.
2. pg_cron is configured as a compatible path, but points to the wrong library - pg_cron.database_name defaults to postgres; if your Multica database name is not postgres, the scheduled task will not see rollup_task_usage_hourly() at all. The in-process scheduler does not rely on this, but if you deliberately remove the in-process scheduler and rely on pg_cron, the DB name must match.
3. handler is claimed, but an error is reported silently - for example, migration is not fully applied, resulting in missing SQL functions, or DB role / search_path is mismatched. Look at the FAILED audit line in sys_cron_executions.

How to check:

-- Confirm that the original data exists and the hourly table is empty
SELECT count(*) AS raw_rows FROM task_usage;
SELECT count(*) AS hourly_rows FROM task_usage_hourly;

--Look at the audit log of the in-process scheduler
SELECT plan_time, status, attempt, runner_id,
       error_code, error_msg, started_at, finished_at
  FROM sys_cron_executions
 WHERE job_name = 'rollup_task_usage_hourly'
 ORDER BY plan_time DESC
 LIMIT 20;

-- watermark —— If it is still 1970-01-01, it means rollup has never been run.
SELECT watermark_at FROM task_usage_hourly_rollup_state;

-- Compatible path: pg_cron has been registered before, confirm whether it is installed and whether the correct library is pointed out.
SELECT * FROM pg_available_extensions WHERE name = 'pg_cron';
SHOW shared_preload_libraries;
SELECT jobname, schedule, database, active FROM cron.job;

How to fix:

• Make sure the scheduler is actually running on at least one backend replica - a SUCCESS line should be added to rollup_task_usage_hourly in sys_cron_executions every 30 seconds.
• There is no problem in manually running the SQL verification function itself: SELECT rollup_task_usage_hourly(); - Refresh the dashboard; if the number comes out, the SQL layer is OK, and the problem lies in the scheduler claiming path.
• If migration 113_sys_cron_executions has not been applied, restart the backend and let migration run again, or manually migrate up.
• It’s okay if there are legacy pg_cron entries in the history - advisory lock 4246 is still held in the SQL function, and the application scheduler and pg_cron will not be double-written; to clear redundant items, see cron.unschedule in Self-host Quick Start → Usage Summary.

ERROR: refusing to drop legacy daily rollups:
  task_usage_hourly_rollup_state.watermark_at (1970-01-01 ...) trails
  task_usage latest event (...) by more than 01:00:00 — backfill is
  incomplete or pg_cron is not running. Run cmd/backfill_task_usage_hourly
  (and let pg_cron catch up) before re-running migrate

Possible cause: This is a fail-closed guard for migration 103. It requires that task_usage_hourly has tied the original task_usage before the old daily rollup is allowed to be discarded. As long as there is historical data in the database and the rollup watermark is still at epoch (which means that the history has not been backfilled into the hourly table), this guard will block it.

Starting from MUL-2957, the migrate command will automatically run an idempotent monthly slice backfill (advisory lock 4246 protection) before applying migration 103, so v0.3.4 → v0.3.5+ can be solved by directly upgrading once migrate up. If you still see this error, you are either using a pre-MUL-2957 binary, or the hook itself failed - look at the task_usage hourly rollup hook line earlier in the migrate log for the specific reason.

How to fix:

1. If you are running a binary before MUL-2957 and cannot upgrade the binary first, manually run an independent backfill on the same database (idempotent, can be interrupted, and can be retried):

   #DockerCompose
   docker compose -f docker-compose.selfhost.yml exec backend \
     ./backfill_task_usage_hourly --sleep-between-slices=2s
   
   #Kubernetes
   kubectl -n multica exec deploy/multica-backend -- \
     ./backfill_task_usage_hourly --sleep-between-slices=2s
   

2. Run the upgrade again - just restart the backend container, and migration will be automatically run at startup. When Guard sees the new watermark, 103 will pass.

3. The in-process scheduler will then continue to push the watermark - see Self-host Quick Start → Usage Summary.

--sleep-between-slices=2s is a restrained default on production libraries that are many years old. If you only want to keep the last N months and can accept permanently discarding older buckets, use --months-back N --force-partial.

Possible reasons:

1. Server port is occupied (default 8080)
2. Daemon health port is occupied (default 19514, each profile is offset by a hash value)
3. Web dev server port conflict (3000 / 5173)
4. Insufficient port permissions (the privileged port bound to < 1024 requires sudo)

How to check:

lsof -i :8080 # macOS / Linux
netstat -ano | findstr :8080 # Windows

How to fix:

• Kill the occupying process (kill -9 PID), or change the environment variable PORT=9000` to change the port
• Use 80 / 443 → Don’t bind directly, use reverse proxy (Nginx / Caddy) to forward to the high port

Components	Locations	Commands
Daemon	~/.multica/daemon.log (background mode) or foreground stdout	multica daemon logs -f --lines 100
Server (Docker)	container stdout	docker logs -f <container>
Server (systemd)	journal	journalctl -u multica-server -f
Front-end (dev)	Terminal where pnpm dev is located	View directly
Front-end (browser)	DevTools → Console	Press F12