An organisation wants AI to work with its documents while retaining control over processing conditions. It can use an API, run a model on its own servers or arrange dedicated hosting. Each option divides cost and operating responsibility differently.
The decision starts with the task. Extraction, document assistance and an agent connected to an ERP have different requirements. A representative trial defines the expected result before the model and environment are selected.
Define the service and its quality threshold
For a document assistant, the team prepares questions with known sources and answers. It adds conflicting documents and superseded versions, along with requests outside a user’s access. Evaluation examines the answer and citations, then behaviour under uncertainty.
Extraction tests cover missing fields and ambiguous documents. Agent tests also examine actions and recovery. Good general writing does not establish suitability for structured extraction or a tool-driven workflow.
Performance is part of the service. The organisation defines acceptable delay and concurrent demand, including document length. A three-page demonstration for one user does not represent a shared application processing large dossiers.
The evaluation set keeps data versions and scoring rules. It allows comparison of a new model or configuration with the previously accepted result.
Compare three deployment arrangements
Open weights can permit execution in a chosen environment, subject to the licence and required resources. They do not mean every model fits on a desktop. Memory must cover the weights and the runtime, including context needs.
| Arrangement | Organisational responsibility | Deciding factors |
|---|---|---|
| Provider API | Integration, submitted data and application controls | Contract, retention, availability and usage cost |
| On-site server | Installation and ongoing operations | Capacity, workload and operating skills |
| Dedicated hosted platform | Agreed division of administration | Isolation, technical access and service commitments |
A local server can meet an isolation requirement, but needs maintenance and a plan for failure. An API can simplify some operating work and provide larger models while creating dependence on its operator. Dedicated hosting offers another division that must be specified in the contract.
Infomaniak AI Services illustrates Swiss hosting. The Apertus model card illustrates published model information an organisation can inspect. A hosting offer and a model card answer different questions; the complete application still needs testing.
Examine the data path and access
Swiss residency is one criterion. Control also depends on the operator, subcontracting and administration. The Federal Council’s CLOUD Act report explains why storage location alone cannot settle foreign-access questions.
A no-training commitment answers a specific question. It does not necessarily define operational retention or staff access. The team checks the actual product, agreement and settings.
Local deployment also includes telemetry and updates, plus any external tools the application calls. A downloaded model can run inside software that sends information elsewhere. The trial must observe the whole exchange required for the task.
For a connected agent, permissions are a separate architectural decision. The code executing its requests checks actions, with human approval where the chosen consequences require it.
Assemble components and prepare replacements
An internal service connects user access with data preparation and model execution. It may also need retrieval and logs. Each component should have an identifiable version and responsible owner.
vLLM provides model inference and serving. Ollama offers another way to run and expose models. Selection depends on compatibility and workload. The user interface and directory integration remain separate design elements.
Document retrieval may use lexical or vector indexes, or combine both. It finds authorised passages for the current question. Persistent agent memory serves a different purpose by retaining information between sessions.
Modularity helps replacement when interfaces and data remain controlled. The team tests another model on the same evaluation set and prepares a fallback. Compatibility with an API does not guarantee identical answers.
Decide from the trial result
The pilot brings together quality and performance measurements with complete cost. An API comparison counts consumption and retries. A local service adds hardware and ongoing administration, including continuity arrangements. Both must provide comparable service.
Data control or a specific model capability may justify one option. Existing operating capacity may justify another. The trial makes those choices explainable and identifies what remains to be improved.
Offers and models will change. A defined use and versioned evaluation set allow the organisation to reconsider its choice without restarting the entire analysis.
Continue with selecting an open-weight model and sizing local infrastructure.