deployment

A deployment is the resource that apply starts. It binds a model to a hardware_profile and selects a runtime.

Examples

llama.cpp on this machine:

resource "deployment" "support_local" {
  model    = model.qwen3_8b.id
  hardware = hardware_profile.local_dev.id
  runtime  = "llama_cpp"
  llama_cpp {
    context_length = 8192
    port           = 8000
    n_gpu_layers   = -1
  }
}

vLLM on a remote GPU provider (Modal here) with an endpoint key and autoscaling limits:

resource "model" "chat_prod" {
  source {
    registry  = "hugging_face"
    reference = "Qwen/Qwen3-8B"
  }
  architecture = "qwen3"
  modalities   = ["text"]
  artifact {
    format = "safetensors"
  }
  context_length = 32768
}

resource "hardware_profile" "gpu_a100" {
  provider = "modal"
  region   = "eu-west"
  accelerator {
    kind   = "gpu"
    model  = "a100-80gb"
    count  = 2
    memory = "80GiB"
  }
  memory = "64GiB"
}

resource "deployment" "chat_prod" {
  model    = model.chat_prod.id
  hardware = hardware_profile.gpu_a100.id
  runtime  = "vllm"
  api_key  = secret("CHAT_API_KEY")
  vllm {
    context_length         = 32768
    port                   = 8000
    tensor_parallel_size   = 2
    gpu_memory_utilization = 0.90
  }
  scaling {
    min_replicas     = 0
    max_replicas     = 1
    scaledown_window = "15m"
  }
}

Fields

scaling

Every field is optional. Omitted fields keep the provider's defaults (see Modal). On any other provider the block is an error (AMICU-CAP-009).

min_replicas must not exceed max_replicas. Invalid values are AMICU-CAP-008. Without max_replicas, a burst of requests can start many GPU containers.

llama_cpp

context_length, port, and n_gpu_layers are required. Everything else is optional; unset fields use llama-server's defaults.

speculative block:

multimodal block (llama.cpp only; a multimodal block inside vllm {} is a schema error):

vllm

context_length and port are required.

extra_args and env

extra_args and env are the escape hatch for flags that have no typed field.

resource "deployment" "tuned" {
  model    = model.qwen3_8b.id
  hardware = hardware_profile.local_dev.id
  runtime  = "llama_cpp"
  llama_cpp {
    context_length = 8192
    port           = 8001
    n_gpu_layers   = -1
    extra_args     = ["--no-warmup"]
    env {
      GGML_CUDA_DISABLE_GRAPHS = "1"
    }
  }
}

• A flag that duplicates a typed field is AMICU-CAP-002.
• A denylisted flag or variable is AMICU-CAP-003. This covers --host, --api-key, --api-key-file, TLS flags for vLLM, LLAMA_API_KEY, VLLM_API_KEY, and loader hooks such as LD_PRELOAD.
• Non-empty extra_args gives AMICU-CAP-004 (info): the memory fit result is unverified.
• Put secrets in api_key, never in env. An env value that looks like a secret gets AMICU-POLICY-006.

Notes

• The plan prints the exact launch command, so you can review every flag.
• Changing the model, hardware, runtime, or any setting is an update action on the next apply.