amicu

amicu vs using RunPod or Modal directly

RunPod and Modal are the platforms that actually run your GPUs, and their own consoles, CLIs and SDKs expose everything they offer. amicu is a layer on top for one job: deploying vLLM or llama.cpp servers there from HCL, with model fit checks, a plan before changes, recorded state and an endpoint authentication check. Going direct gives you full access to each platform and no extra tool. amicu gives you one workflow across providers, and it is a good fit when the job is specifically an LLM endpoint.

QuickstartAll comparisons

Side by side

amicu vs using RunPod or Modal directly
 RunPod or Modal directlyamicu
What it isGPU platforms with their own console, CLI and API or SDK.A CLI that deploys LLM servers to providers such as Modal and RunPod using your own account.
Platform featuresEverything the platform offers, such as RunPod Pods, Serverless and clusters, or Modal Python functions and sandboxes.Only what is needed to deploy and stop LLM servers.
How you define a deploymentConsole, provider CLI commands, or code (Modal defines GPUs and containers in Python).The same HCL schema for every provider.
Review before changeDepends on how you script it.plan diffs configuration against state; apply asks for approval.
Model fitYou pick the GPU and tune the serving engine yourself.validate checks weights plus KV cache against the accelerator memory.
CleanupYou track and delete resources yourself.State records stop identities, and destroy confirms each stop before forgetting a deployment.
Endpoint authYou configure it for each platform.Requires an api_key secret and verifies that anonymous requests are rejected.
Switching providersRewrite for the other platform's tools.Change the provider on the hardware profile; models and deployments use the same schema.
CredentialsManaged per platform.Read from environment variables; secrets are references and never stored in configuration or state.
LimitsFull control, including features amicu does not expose.Models must come from Hugging Face on remote providers; early and not stable.

Sources for the RunPod or Modal directly column: RunPod overview, runpodctl, Modal guide, Modal GPU guide. Details can change; check the official documentation.

When to choose RunPod or Modal directly

  • You use platform features amicu does not cover, such as training, clusters or custom Python functions.
  • You run a single provider and are happy scripting it yourself.
  • You need to tune the platform beyond what amicu exposes.

When to choose amicu

  • You want a reviewable plan and recorded state for each LLM endpoint.
  • You want model fit and endpoint auth checked before and after deploying.
  • You want the same configuration style for your machine and for GPU providers.

Questions

Does amicu replace the RunPod or Modal console?
No. amicu deploys and stops LLM servers. The platform consoles remain the place for everything else, including billing and account settings.
Do I need my own RunPod or Modal account?
Yes. amicu uses credentials you export in your shell, and talks only to the providers you deploy to.
Which providers does amicu support today?
This machine, plus Modal and RunPod as remote GPU providers. Other providers are in preview or planned; see the providers page.
Does amicu add cost?
The CLI is free and open source. You pay the provider for the GPU time you use.

Try amicu

Install the CLI and deploy a model on your machine or a GPU provider.

QuickstartProviders