Private AI, in accounts you own.

Who it’s for

This engagement is for a technical team with a defined reason to control where AI workloads run and a named person willing to operate the result. The boundary might be data handling, ownership of accounts or the ability to replace a model without replacing the surrounding system.

You might have a local installation that staff now depend on, a requirement to account for data flows or a need to evaluate alternatives to a shared service. Scoping turns that requirement into an architecture, an operating responsibility and an acceptance test.

Owning a stack means owning its maintenance. If your team needs a hosted service with minimal operational involvement, start by testing that option. Model suitability is evaluated on your tasks; private deployment does not establish quality parity with another model.

A stack your team
can take over.

• Your environment. Provision the agreed stack in your cloud account or on your hardware. Describe compute, networking and deployment configuration as code, with a record of permitted external dependencies.
• Serving and access. Evaluate open-weight models against your tasks, then configure serving and an OpenAI-compatible API gateway with identity integration, user and team limits, and audit logging. Verify the endpoints your applications actually use.
• Applications. Provide an internal chat interface and API access for agreed tools. Retrieval over company knowledge is a separate scope when needed.
• Operation. Include secrets management, monitoring, backups, a restore exercise and a documented upgrade and model-swap procedure.

Hardware procurement and racking are outside the standard build and need a separate agreement. Pre-training models and making compliance determinations are outside this service. Your operator owns day-to-day operation after handover. Ongoing support is agreed per engagement.

If your environment requires full network isolation, raise it at scoping. Feasibility is assessed per engagement; it is not a standard part of this service.

When a simpler
alternative is better.

When the concern is how a provider uses or retains your data, evaluate a hosted API’s actual contract and configuration before choosing to self-host. If those terms and the processing location meet your requirements, compare its operating burden with the private option.

For limited, non-critical use, test a single-machine local setup before building a shared service. Identity, availability and recovery requirements should determine when that setup needs a more deliberate operating design.

Compare the options using your task quality, usage pattern and operating costs: compute, storage, backups, operator time and upgrades. Include idle capacity and peak demand. The decision does not assume that self-hosting costs less or that the same model fits every task.

See local, private cloud or hosted: how to choose for the five-option matrix, operating questions and synthetic break-even method.

Prerequisites

• A client-owned cloud account or hardware option, with an owner who can grant and revoke the access needed for the build.
• An identity provider, a user and group model, and an owner for service identities used by internal applications.
• A named internal operator for access, patching, backups, model changes and capacity monitoring.
• Representative tasks and a current baseline to compare model outputs, including difficult cases and unacceptable responses.
• Data classes and processing boundaries. Identify which material may reach which service, and whether prompts or responses may be retained.

Choose the model.
Then size the stack.

Evaluate candidate models on your tasks before fixing hardware sizing. Record output quality, memory needs and behaviour under the expected concurrency. Then test serving capacity and agree queueing, request limits and what users see when capacity is exhausted.

Build upward from compute and network controls through serving, pinned model artefacts, a gateway and applications. Monitoring and backup cover the stack. The gateway centralises routing, access decisions and limits; the application does not choose its own authority.

Private deployment needs an explicit data-flow review. Identify inference endpoints, telemetry, update checks, package downloads and backup destinations. For a deployment intended to keep prompts and documents within your environment, configure and test that boundary, and record each approved external flow.

For agents that must run in your environment, infrastructure access and action authority are separate decisions. A model’s ability to reach an API does not grant permission to use it.

Define the boundary.
Exercise the recovery.

Identity, secrets and logs

Interactive access goes through the agreed single sign-on integration. Internal applications use scoped service identities with an owner and a rotation procedure. Administrative access is separate, limited and recorded; the client holds the recovery credentials.

Usage logs record the agreed user or service identity, model version, time and request outcome. Whether prompt and response contents are retained is an explicit client decision, with access and retention rules. A usage log does not need to become an uncontrolled copy of sensitive prompts.

Backup and restore

The backup inventory includes configuration, required application state, vector stores if present and chat history if retained. Model recovery uses approved artefacts or pinned source references whose availability and licence terms have been checked. Separate the backup’s access from the running application’s credentials.

Test a restore into a clean environment before handover. Recover configuration and retained state, restore or reissue credentials through the agreed procedure, load the pinned model and verify identity, inference and application access. Record dependencies that would prevent recovery if an external source became unavailable.

Changes and capacity

A replacement model passes the agreed evaluation set before it replaces the current one. Stage it, compare the outputs and switch using a documented procedure. Retain the previous model and compatible configuration for rollback, subject to the agreed storage policy.

State queue limits, request timeouts and overload responses in the operating contract. Test them under the agreed workload. The client operator owns patch scheduling, capacity review and the decision to release a model or runtime update.

Illustrative example: an upgrade changes the model’s response format and breaks an internal tool. The operator pauses the rollout, restores the prior model and gateway configuration, checks the tool against the evaluation cases and records the failed change before trying again.

The accounts.
The instructions. The keys.

The handover includes infrastructure definitions and configuration in your repository, the architecture and data-flow document, and the model evaluation report on your tasks. It records supported API behaviour, external dependencies and the boundaries of the tested deployment.

Runbooks cover adding a user or team, adjusting limits, swapping a model, scaling capacity, restoring from backup, rotating secrets and investigating logs. The operator walkthrough includes a supervised restore drill and a review of the upgrade path.

Administrative accounts and credentials belong to the client. Handover includes an access inventory and the revocation or rotation of build access that is no longer needed. Continued support access is separately agreed, scoped and revocable.

What your team
operates.

Your operator manages users and service identities, limits, patching, backup schedules, restore exercises and capacity. Model changes go through the evaluation gate. A new data class, application or provider endpoint needs a review of the processing boundary.

Scoping assigns alert ownership, update decisions and recovery responsibilities. Document who can authorise an outage or a capacity change. Ongoing support is agreed per engagement.

A working install
is the beginning.

Agree the task set, current baseline, deployment boundary and acceptance thresholds before the build. Record quality and operational results separately. A deployment needs evidence that it can serve the intended work and that your team can recover and change it.

• Task quality: compare candidate outputs with the current baseline, side by side, using the agreed rubric and difficult cases.
• Capacity: measure throughput and latency on the selected hardware under the agreed concurrency. Check queueing and overload responses.
• Access: verify sign-on, a refused unauthorised user, scoped service credentials and removal of access.
• Logs and data flows: inspect the agreed fields, retention settings and destinations. Check that configured external flows match the approved design.
• Recovery: complete the restore drill on a clean environment and inspect the recovered state and dependencies.
• Change: exercise a model swap and rollback through the documented procedure, including the applications that depend on the API.

The report describes the measured configuration and task set. It does not promise the same result for a different model, workload or hardware. Changes need another evaluation and an updated operating record.

The business AI acceptance scorecard includes quality, concurrency, identity, restore and model-swap tests.

What to prepare

Bring data classes and processing rules, user count and expected concurrency, the identity provider, available cloud or hardware options, representative tasks and the operator’s name. Include an existing baseline and any application that depends on a particular API behaviour. Sensitive records can wait for an agreed access path.

Start with
the boundary.