Forward Deployed Engineer, how I deliver it, requirement 8 of 12
Hybrid enterprise integration
I connect AI services to your identity, applications and data, and run the platform on Kubernetes on Azure Local.
The requirementIntegrate AI solutions with enterprise applications, platforms and cloud or hybrid environments.
How I deliver it at a customer
- Decide placement firstI start by deciding which data may leave the customer's environment and which model calls may use a hosted provider. The platform runs today on AKS on an Azure Local cluster, with CloudNativePG running Postgres. I have written a separate cloud AKS environment as Terraform and ArgoCD manifests, and it is not yet stood up.
- Wire your identityI set up single sign-on through Google or Microsoft Entra ID, using the Authorization Code flow with PKCE. The platform checks each ID token signature against the provider's published keys and can limit sign-in to approved email domains. On first sign-in, it links the identity to a matching account or creates one with the basic user role.
- Point the model gatewayI route model calls through one gateway that holds the provider keys and logs each call with model, tokens and latency. A route can switch between a hosted model and a local GPU model by configuration. A route can also fail closed, with no hosted fallback. Retrieval embeds on a local GPU by default and falls back to a hosted embedding model if that fails.
- Connect the applicationsThe platform's MCP endpoint ships switched off. When I turn it on, an assistant session reaches a set of platform tools over stdio MCP through a bridge. The server authorizes every call. Your own services call api.onluca.ai/v1 with issued keys. Each endpoint except the schema requires a named scope, and each key gets 120 requests per minute by default. Inside the platform, API tokens can be limited to named applications and carry their own rate limit.
- Handle secrets and trafficOn the Azure Local cluster today, the gateway gets its secrets from a Kubernetes secret. The cloud AKS environment I have written is set up to pull them from Azure Key Vault. It uses the CSI driver and a user-assigned managed identity. Public web traffic reaches the cluster through a Cloudflare tunnel that cloudflared opens from inside the cluster.
- Ship and roll backThe gateway deploy I use puts up a maintenance page and rolls out the new image. It rolls back if the rollout shows a real failure, such as a crash loop or a failed image pull. A smoke check then tests health, the database, the session endpoint, one model round trip and app loading. If it fails, the maintenance page stays up and the deploy is marked failed. An alert also posts to a webhook when one is configured. Prometheus and Grafana carry the metrics.
Tools I use
| Tool | What it does here | Status |
|---|---|---|
| Multi-provider model gatewayAvailable | Routes model calls through one place that holds the provider keys and logs each call. A route moves between a hosted model and a local GPU model by configuration. | Available |
| Kubernetes on Azure LocalAvailable | Runs the platform today on AKS on an Azure Local cluster, with CloudNativePG for Postgres and a Cloudflare tunnel for web traffic. Deploys roll back when a rollout shows a real failure and hold on a failed smoke check. | Available |
| Cloud AKS environmentRoadmap | A separate production environment on managed AKS, written as Terraform and ArgoCD manifests with prune and self-heal. The manifests pull secrets from Key Vault through the CSI driver. It is not yet stood up. | Roadmap |
| MCP tool bridge and gatewayEarly access | Gives an assistant session a set of platform tools over stdio MCP, and one server-side endpoint authorizes every call. That endpoint and an OAuth surface for other MCP clients sit behind one flag that ships off. | Early access |
| Cloudflare WorkersDemo only | Serves this page's demo and reports which Cloudflare data center handled your request. No language model runs in this demo. | Demo only |
Available means shipped and in use. Early access means built but not generally available. Roadmap means designed, not built. Demo only means it powers the public demo, not customer deployments.
Press the button to see which Cloudflare edge data center handled your request, with its TLS and HTTP details. No language model runs. The table shows where each layer runs in a customer deployment.
Open the live demoThe other answers
LLM applications in productionOn Luca, model calls normally go through a gateway that holds provider keys, records each chat call and sets a model per task.Prompt engineeringI write the output shape first, measure the prompt against each model's context window, and check replies against that shape.Retrieval-augmented generationChat answers over a customer's own documents, returned with the matched document names, after a check that the caller may use that persona.AI agents and orchestrationI build agents that call your systems through registered tools, still in early access, and workflows that start on business events.Request classification and routingI build automation that classifies each inbound request, plans bounded tasks, and holds high-risk tasks for approval; the dispatcher is in early access.AI workflow orchestrationI map the steps, API contracts, and data flows between a customer's systems and the models before I build the workflow.API and integration designI publish the API contract first, gate it with keys and scopes, then build integrations against it.Prototype to productionI take a prototype through generated tests and a two-approval merge gate to a production deploy that rolls back if the new pod keeps crashing or cannot start.Approvals and exception handlingIn chat, I hold each Gatekeeper tool marked destructive until the signed-in user approves it, with expiry and a record. This path is in early access.Evals and output validationI built a versioned rubric judge for a tutoring app, and I trace failed or slow model calls to their provider.Client-facing deliveryI turn engineering work into updates that client architects, security leads and business owners can each act on.
Built by Gus Assuncao of Gus IT LLC for a Forward Deployed Engineer application. Luca is the AI platform Gus IT builds and operates. Contact gus@gusit.de. Claude is a product of Anthropic PBC. Cloudflare and Workers AI are trademarks of Cloudflare, Inc. Llama is a trademark of Meta Platforms, Inc. Other names are trademarks of their respective owners. Gus IT LLC is an independent engineering firm and is not affiliated with or endorsed by these companies.