Troubleshooting

Start with the agent’s logs

When an executor won’t connect, its own log is almost always the fastest answer — the platform can only tell you that nothing arrived, while the agent knows exactly why it was turned away.


docker logs qremotex          # Docker
kubectl logs deploy/qremotex  # Helm / Kubernetes

A rejection looks like this:


Gateway rejected connection [INVALID_KEY]: API key not recognised
  -- Check that QRAPTOR_API_KEY matches a key from Studio (Remote Executors -> API Keys),
     and that the whole key was copied. Retrying in 60s (attempt 1).

Three parts: the reason in brackets, what happened, and what to do about it. The table below is the same information, expanded.

ℹ️

The executor never gives up. Every rejection is retried — there’s no state it can get into where it stops trying and needs to be noticed. Problems that need you to change something back off to at least 60 seconds between attempts, so a fixed key or a newly activated gateway is picked up within a minute without anyone restarting anything.

Rejection reasons

Reason	What it means	What to do
`INVALID_KEY`	The key isn’t recognised.	Check `QRAPTOR_API_KEY` matches a key from Remote Executors → API Keys, and that the whole key was copied. Truncation is the usual cause.
`KEY_REVOKED`	The key was revoked in Studio.	Create a new key, then update `QRAPTOR_API_KEY`.
`KEY_EXPIRED`	The key passed its expiry date.	Create or rotate a key, then update `QRAPTOR_API_KEY`.
`KEY_INACTIVE`	The key is no longer usable — expired or revoked.	Create a new one in Studio.
`PROJECT_MISMATCH`	The key belongs to a different project than the URL points at.	Check the `exec-…` part of `QRAPTOR_GATEWAY_URL` matches the project the key came from.
`EXECUTOR_LIMIT`	The subscription is already running its maximum number of executors.	Stop one that’s idle, or upgrade the plan.
`PLAN_NOT_ALLOWED`	Remote executors aren’t enabled on this plan.	Upgrade to Pro or above.
`RATE_LIMITED`	Too many connection attempts with this key in a short window.	Nothing — it backs off and clears on its own.
`CLOCK_SKEW`	The host’s clock is too far out of sync for its requests to be verified.	Turn on automatic time sync (NTP).
`NONCE_REPLAY`	Usually two executors sharing one API key.	Give each executor its own key.
`SERVER_ERROR`	A platform-side error.	Nothing — it retries.
`GATEWAY_UNAVAILABLE`	Remote executors aren’t activated for this project, or the plan doesn’t include them.	Turn them on in Remote Executors, then allow a few minutes.

💡

GATEWAY_UNAVAILABLE immediately after activating is expected — the check is cached for about five minutes. Leave the executor running and it will connect by itself.

Symptoms without a rejection

Not every problem produces a rejection. These are the common ones.

The executor stays on “Not yet connected”

The agent has never successfully authenticated. Work through it in this order:

Is the agent running at all? docker ps, or check your process manager. A container that exited immediately usually means a missing required variable.
Can the host reach the gateway? It needs outbound HTTPS to your exec-… address. A proxy or egress filter that allows plain HTTPS but blocks WebSocket upgrades will fail here.
Read the log. By this point there’s almost certainly a rejection in it.

It was Online and went Offline

Offline means heartbeats stopped for about a minute. Check whether the container is still running, whether the host lost its network, and whether the key was revoked or expired. The agent reconnects on its own once the cause is gone.

An agent step didn’t run remotely

Nothing appears in execution history, because a step that never dispatched leaves no record.

Was an executor Online at the time? Draining and degraded-then-offline hosts are skipped.
Is the step actually set to Remote? The default is local — see Run work on an executor.
For a tool: was it pinned? An unpinned tool falls back to the platform and runs there, which looks like success in the agent but produces no executor record.

A code node fails with “no online executor”

That’s the intended behaviour, not a bug — a code node set to Remote will not silently run on the platform. Bring an executor online, or change the node to Auto or Local if platform execution is acceptable.

A remote script fails with `ModuleNotFoundError`

The package isn’t installed on the executor. Remote scripts use the executor’s Python environment, not the platform’s. Add it to QRAPTOR_EXTRA_PACKAGES or bake it into a custom image, then restart.

A remote script fails on `ctx`

ctx["db"], ctx["llm"] and ctx["config"] don’t exist on an executor. Use {'{{'} config.xxx {'}}'} templates, which are resolved on the platform before dispatch.

Two executors appeared for one host

Each API key binds to one executor. Sharing a key between hosts, or reusing one after deleting an executor, produces duplicates — and shows up as NONCE_REPLAY in the logs. Give every host its own key.

An HTTP tool times out against an internal service

The executor’s host can’t reach it. Test from that host directly rather than from your own machine — internal DNS and security groups often differ.

Getting more detail

Set LOG_LEVEL=DEBUG on the executor for verbose output covering the connection handshake, dispatch and per-job execution. Worth turning on while diagnosing and off again afterwards.

Security & networking — what the executor does and doesn’t have access to.