Endpoint auth

A running inference server that accepts anonymous requests is a free GPU for anyone who finds the URL. Amicu does not trust that the key is enforced; it tests.

The check

After apply starts an endpoint, it sends requests to GET /v1/models. This route is authenticated by both llama-server and vLLM. (llama-server's /health is public by design, so it cannot test the key.)

• A 2xx at step 1 or 2 means the endpoint is open. Amicu stops it and fails the apply. If the stop itself fails, the error names the command to run by hand.
• Any other answer to steps 1 and 2 (connection refused, 404, 5xx while the server warms up) is retried, then fails closed: an endpoint whose protection cannot be verified is not reported as deployed.
• Step 3 retries while the model is still loading (503). A 401 or 403 with the correct key fails immediately (endpoint rejected the configured API key) with no retry.
• Retry windows: remote endpoints up to 2 minutes for steps 1 and 2 and 10 minutes for step 3; local endpoints up to 15 minutes each.
• The request goes straight to the endpoint: no environment HTTP proxy is used, and redirects are not followed, so the key is never sent anywhere else.

Loopback endpoints with no api_key skip the check; there is nothing to protect and the endpoint is not reachable off the machine.

Providers with a gateway

Some providers put a gateway in front of the app that requires its own credential. amicu needs that credential to prove the runtime enforces its own key behind the gateway, so apply stops in preflight, before anything starts, if it is missing. Today this applies to Modal; see Modal: endpoint auth.

Calling the endpoint

After a successful apply, amicu prints the endpoint with ready-to-copy snippets. They name credentials by environment variable, never by value.

Local, with a key:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $LOCAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3.5-8B-Instruct", "messages": [{"role": "user", "content": "Hello"}]}'

A remote provider with a gateway needs the gateway headers too. See your provider page; for Modal, the example.

With the OpenAI Python SDK:

import os
from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key=os.environ["LOCAL_API_KEY"],
)
reply = client.chat.completions.create(
    model="Qwen/Qwen3.5-8B-Instruct",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

For a provider with a gateway, also pass its headers via default_headers. For Modal, see Modal: endpoint auth.

Related codes