LLM applications in production
One gateway for every model call, guardrails on both sides, and a measured choice of model per task.
How I deliver it at a customer
- One gateway for every callAll model traffic goes through the Luca gateway, which holds provider keys, rate limits and per-call logging in one place. It routes to more than one provider. Spend caps and a cost ledger are in early access.
- Right-size the model per taskClassification and extraction run on a small fast model; reasoning and drafting run on a large one. Both are measured on the customer's own examples before the choice is fixed.
- Guardrails on input and outputA safety classifier screens the prompt before any model sees it, and output is constrained to a schema the application validates.
- Evaluate every changeAn LLM judge scores answers on fixed metrics, with a fairness check, and a prompt or model change ships only if the scores hold.
- Observe in productionLatency, errors and token use per call go to Grafana and Loki, with alerts on regressions.
- Humans approve actionsAnything that changes a system passes an approval gate. Approval gates and role-based access on agent tools are in early access.
Tools I use
| Tool | What it does here | Status |
|---|---|---|
| Luca gateway | Multi-provider model gateway with keys, rate limits and logging. | Available |
| Claude (Anthropic API) | Primary reasoning model for agentic and drafting work. | Available |
| Luca evaluation judge | Six-metric LLM judge with a fairness harness. | Available |
| Spend caps, approval gates | Budget limits and human approval on agent actions. | Early access |
| Cloudflare Workers AI | Runs this public demo: Llama Guard 3 plus Llama 3.1 8B and Llama 3.3 70B. | Demo only |
Available means shipped and in use. Early access means built but not yet generally available. Demo only means it powers the public demo, not customer deployments.
Send one prompt. A safety classifier screens it first; if it passes, an 8B and a 70B model answer side by side with measured latency and token use, so the size and cost trade-off is visible.
Open the live demoOther answers
Gus IT LLC. Contact gus@gusit.de. Claude is a product of Anthropic PBC. Cloudflare and Workers AI are trademarks of Cloudflare, Inc. Llama is a trademark of Meta Platforms, Inc. Other names are trademarks of their respective owners. Gus IT LLC is an independent engineering firm and is not affiliated with or endorsed by these companies.