A practical guide for comparing Private AI on a client server, in a dedicated environment and in a hybrid architecture by tracing one real workflow.
Start with the workflow, not the server label
The question “where should AI run?” is an infrastructure decision only after one employee workflow, its data, response-time need and permitted actions are defined. A document search may fit an isolated environment, while another process needs an ERP connection and a human approval step. Record the decision with the workflow objective and acceptance criterion so the infrastructure choice can be tested under operating conditions.
Map the data source, transformation, model request, source retrieval, answer, logs and any write to another system. Mark which information may leave the client network and which must remain inside a controlled segment. The data owner confirms meaning, location and retention, while the technical owner preserves configuration evidence.
Without that map, labels create false confidence. An “on-premises” service can still depend on external identity or update channels, while a hosted environment may use strongly separated client resources. The real boundary is the data path rather than the marketing name. Run the failure case with test data and inspect the user outcome, logs and recovery path.
Assign a process owner, information-security owner and technical owner. Together they approve the business outcome, data boundary and support model, with a named owner for each decision. Choose deployment only after these requirements are written in a form that can be tested. Give the control an owner, review cadence and explicit action when the result does not meet the requirement.
Consider a buyer who wants to know whether the latest signed framework agreement allows the delivery date on a particular order to be moved. The agreement and any amendments are in the document archive. The enterprise resource planning system, or ERP, holds the date currently recorded against the order. A team can use this illustrative task to compare deployment options before choosing one.
A buyer should be able to open the clause behind the answer and see which signed version the assistant used. The date on the order should appear separately, with enough information to find the same entry in the ERP. If the clause does not cover that order, or a written amendment is missing, the question goes to the purchasing manager. A person decides whether to move the date and whether to contact the supplier. That description of success gives the technical team something concrete to test.
Define client-hosted, dedicated and hybrid models
In a client-hosted model, inference, the document index and integration services run in infrastructure controlled by the organisation. This can be physical hardware or a client-managed private environment. Control is direct, but capacity, updates and recovery remain operational responsibilities.
In a dedicated environment, a provider sets aside resources for one client in a data centre or on a cloud platform. Separation can mean an account, network, database, virtual machine or physical host. The architecture and contract must state exactly what is dedicated.
A hybrid design splits workflow stages. Documents and access checks may remain on the client network while inference uses a dedicated compute service with only the required context. Another pattern sends an anonymised task outside the network.
No model is automatically best. Client hosting can reduce data movement but increase maintenance risk. A dedicated environment can scale capacity but needs verifiable isolation. Hybrid deployment adds flexibility and also adds boundaries and connections that must be controlled.
Compare the same document question in all three models. Record where the source, index, model, answer, log and backup are located. The comparison exposes the operational difference instead of relying on broad labels.
Prepare a test order, the signed agreement, an older version and an appendix the buyer cannot open. Use the same buyer role in each environment. If the on-premises demonstration searches the archive with administrator access while the hosted demonstration uses an ordinary employee account, permissions alone could explain any difference in the answers. Keep the documents and permissions identical first. Then compare where each design retrieves the clause and how the buyer can check the conclusion against the original.
Use one end-to-end data path as the decision basis
Describe the path from an employee action to the final result: browser or application, identity check, API gateway, document retrieval, model service and the system receiving an approved outcome. Every arrow represents a network route, permission and potential logging point.
At each step, record the category and amount of data. A complete contract, three selected passages, a document identifier and a metric are different. Temporary caches and error messages can also contain sensitive content and belong in the map.
The map reveals unnecessary transfers. If a model needs three approved passages, there is no reason to send an entire collection. If a human can approve the proposed action, the model does not need final ERP write permission.
Use data minimisation, explicit retention and a deletion route. Include backups, monitoring platforms and support access. The path is incomplete until the team knows how information leaves the system as well as how it enters.
Illustrative example: an invoice review reads a PDF and supplier record locally, sends only required fields to dedicated inference, returns a structured proposal and presents it to an accountant. An unapproved proposal writes nothing to the accounting application.
Start with the order reference and use it to identify the relevant framework agreement. The signature status, revision and effective period distinguish the agreement in force from the other files. A recently uploaded document could be an unsigned proposal. Give the ERP connector read-only access to the delivery date for the order in question. Show the archive reference and the ERP date separately. The buyer can then spot an operational date that has already moved even though the available paperwork supplies no basis for the change.
Client-hosted deployment brings maintenance responsibility

Client infrastructure can fit a policy requiring documents and inference to remain in a controlled network, or a site with restricted external connectivity. Yet “running here” does not by itself guarantee secure configuration, access control or timely maintenance.
Measure CPU, GPU, memory, disk, spare capacity and power. Model size is only part of the load: indexing, concurrent users, encryption and logs also consume resources. Test the intended workflow and concurrency rather than one demonstration request.
Name who patches the operating system, updates the model, rotates certificates, watches storage and tests backups. A machine in the client’s office without a day-to-day owner becomes an outdated service rather than a controlled service.
Restrict and audit remote access. Support should use temporary approved access with multifactor authentication instead of a permanent shared account. Test updates in a separate environment and deliver them through a documented route.
Illustrative test: disconnect the external internet and repeat the core workflow. Verify identity, retrieval, inference, logging and the behaviour of any function that genuinely needs an external service.
In the on-premises trial the buyer opens the archive and the ERP in their usual working environment. When the answer points to a clause, the buyer follows the link using their own account and checks the passage in context. Try the question while the index is still processing a newly signed amendment. A service that reliably retrieves yesterday’s revision may be of little use on the day purchasing receives revised terms. Observe whether it identifies the version it actually searched and how the buyer can recognise a gap.
A dedicated environment must specify actual isolation

A dedicated environment can provide elastic compute and central maintenance, but “dedicated” is not a technical specification. Ask whether the client has a separate account, project, virtual network, encryption key, database, worker and log store.
Record the region, subprocessors, backup location and support-access procedure. Telemetry and fault diagnostics are also data flows. Compare contract wording with the deployed configuration and evidence.
Incorrect shared configuration can expose another environment’s names, logs or resource identifiers. A broad administrator identity can also span several clients. Isolation needs to cover identity, network, data and operations.
Use configuration review, access logs, encryption in transit and at rest, and separate secrets. A recovery test must show that only this client’s data can be restored without mixing environments.
Illustrative test: attempt to access another project resource from the client account. The attempt should be denied, recorded and reported to the person responsible. Then verify that emergency support access needs approval and expires at the agreed time.
Ask the provider of the dedicated environment to show where the excerpt for this order goes and where the technical log remains. The source link on the buyer’s screen should open the same signed version used in the on-premises trial. Include a support enquiry in the exercise. If an operator can locate a failed request from its identifier, there is no reason to send them a full contract or list of orders to diagnose that failure. Let the operator demonstrate the lookup with the test request.
Hybrid architecture sends minimum context across the boundary

A hybrid design keeps the most sensitive data on the client side and uses external or dedicated capacity for a specific stage. It can be useful when identity and retrieval must remain local but inference demand varies or needs specialised hardware.
The boundary service validates a schema, removes unnecessary fields, replaces identifiers and adds a correlation ID. The receiving service does not gain broad network access simply because it processes one request.
Hybrid systems add moving parts. A broken connection can leave unfinished work, retries can create duplicates, and a cache can retain context longer than intended. These failure modes belong in the design.
Use idempotency, time limits, an encrypted connection, allowed endpoints and a local failure mode. When the remote stage is unavailable, keep work in a protected queue or state clearly that no result was produced.
Illustrative example: local retrieval selects three procedure passages and replaces an employee ID with a temporary token. Dedicated inference drafts an answer; the local layer adds source links and checks current user rights before display.
In the hybrid trial, the local search service selects a clause the buyer can read. Check whether the clause and the relevant dates give the remote service enough information. Supplier contacts, images of signatures and unrelated order lines add nothing to this question. Equally, a clause may have a different meaning if a definition on the preceding page is omitted. Record that problem in the test results. The team can keep the task local or agree on a slightly wider set of permitted excerpts before repeating the trial.
Preserve identity and access rules in every model
An AI service must not bypass document permissions. Carry the user identity to retrieval and action layers. The model receives only passages the same user could open in the source system.
Define employee, reviewer, administrator, support and service-account roles. Each receives minimum permissions and a distinct authentication method. A service account does not need interactive administrator access, and an administrator should not reuse the service secret.
If an index ignores source permissions, an answer can combine material from departments that cannot see each other. Filtering the output after generation is too late because unauthorised content has already reached inference and logs.
Control starts before retrieval: check the role, filter candidate documents, record the decision and check again before an external action. Privileged writes need human approval or a separate strong authentication step.
Illustrative test: ask the same question in two roles. A finance employee receives the authorised procedure with its source, while another department receives a no-authorised-source response. Confirm that the forbidden passage is absent from error logs.
The buyer can open the framework agreement but lacks permission for the restricted appendix. An answer must not reveal that appendix by paraphrasing it. When the permitted material does not establish that a date change is allowed, the assistant can say so. The purchasing manager reviews the question using their own access and can request further documentation. Also test what happens after access is revoked. An old conversation or a previously issued source link should no longer display protected text to the buyer.
Control retention, logs and deletion
Set retention for source files, indexes, prompts, answers, feedback and technical logs. These can have different periods. A complete conversation history is not automatically necessary merely because it could help troubleshooting.
Logs should answer operational questions without copying unnecessary content. Correlation ID, time, model and index version, permission outcome and error code are often sufficient. Replace sensitive passages with references to protected sources.
Data can remain in caches, temporary files, crash dumps and backups. A deletion process covering only the main database leaves other copies. Evaluate the entire chain for every deployment option.
Maintain a data inventory, retention table, automated deletion and reviewed exception process. Document backup retention, and make sure restoration does not return deleted information to active use without reapplying deletion.
Illustrative test: create a request for a test identity, trigger deletion after the period, and check the active database, retrieval index, cache, logs and backup procedure. The result is evidence rather than a single “deleted” button.
Someone may later ask why the date in the answer differed from the date they can now see. The request time, the revision of the agreement and the time the ERP record was read help reconstruct what happened. Copying a complete email or contract into the technical log usually adds little to that investigation. If the business needs a record of the decision, the purchasing manager can record it in the designated case file with a reference to the document they actually reviewed.
Limit integrations, network routes and secrets
Enterprise AI often connects to document storage, ERP, email or a client portal. Every connection expands risk and maintenance. The deployment design must show where credentials live and which network segment can call each system.
Do not place secrets in prompts, source code or ordinary logs. Store them in approved secret management, assign them to one service and rotate them. Grant only the necessary operation, such as reading an approved document.
Broad two-way access increases impact. An initial stage can read data and prepare a proposal for human approval. Add writing only after schema validation, duplicate control and a read-back check.
Define allowed network directions, firewall rules, timeouts, certificate validation and correlated audit. Close unused routes instead of retaining them for possible future work.
Illustrative example: AI drafts an ERP record but cannot save it. An integration service validates the structure, approval and unique business ID, writes and reads back the result. See the guide to connecting Private AI with ERP and accounting.
For the purchasing question, the read-only ERP account needs the order reference and its planned delivery date. It needs no permission to amend the order. If the sources conflict, the assistant can prepare a question for the purchasing manager and include both references. The manager decides whether to look for a signed amendment and whether to contact the supplier. A person sends the message. If the supplier subsequently proposes a different date, the business can distinguish that later proposal from the material originally available to the buyer.
Version the model, index and configuration together
Deployment includes an update model. Identify who may change the model, system instruction, document splitting, retrieval parameters and security policy. Each can alter answers even when the user interface does not change.
Store the model identifier, configuration, index build date, source list and test results as a versioned set. This supports incident reproduction and comparison. A vendor model name without a precise version is insufficient.
An automatic update can break integration or language quality. A local system that is never updated accumulates vulnerable libraries. Use a controlled window, test environment and rollback plan.
Apply an approval flow, immutable artefact checksum and acceptance suite. After an update, test citations, access filters, prohibited actions, performance and recovery. Roll back the compatible set rather than one component.
Illustrative test: compare the same twenty questions before and after a model change. Assess source accuracy, refusal without evidence, role differences, latency and resource consumption, not only writing style.
Keep the older agreement in the test collection after a model or index update. It helps distinguish two different faults. The assistant may retrieve the old clause while a newer signed version is available, or it may find the correct clause and associate it with the wrong order. In the first case, check document selection and when the index was refreshed. In the second, examine the link between order and agreement. Repeat the original question under the same role. A source reference lets the buyer check which fault has been corrected.
Test recovery and loss of external connectivity

Every deployment has a different failure profile. Client hardware can fail, a dedicated environment can lose a region or account, and hybrid processing can lose the connection between stages. The workflow determines acceptable downtime and data loss.
Back up configuration, the index or its rebuild recipe, secret-recovery steps and audit evidence. The model file need not always be copied if it can be retrieved securely by checksum, but that path must be tested.
An untested recovery plan is an assumption. A restored environment may start yet fail to authenticate users, retrieve documents or reach an integration. Verification must execute an end-to-end scenario.
Set recovery targets, owners, communication and a manual fallback. In hybrid mode, distinguish functions that continue locally from functions that stop. Users must see a truthful status rather than an unverified answer.
Illustrative test: disable the external link, run a local document question, attempt a remote-model function and inspect the queue. After restoration, reuse the original request identifier, check the user’s current permission and complete the task once.
Interrupt connectivity after the clause has been retrieved but before the ERP returns the order date. The buyer should be able to tell which fact is missing and how to check the order manually. Once connectivity returns, read the current date again, check the buyer’s permissions and associate the response with the initial question. A colleague may already have amended the order during the outage. Showing the time of the new read makes it clear which state of the order the answer describes.
Use a comparison matrix and controlled rollout
Compare data boundaries, hardware, concurrent users, latency, integrations, maintenance skill, recovery target and cost categories. Mark figures as measured, vendor-supplied or illustrative.
Draw the same process in three architectures and use identical test scenarios. This avoids testing client hosting under production load while judging a dedicated environment from one demo. Use the same dataset and quality criteria.
Build a budget for each deployment option using the same workload as the basis for comparison. Costs include hardware or compute, storage, network, licences, implementation, monitoring, updates, recovery and staff work. Calculate them for the selected workflow.
Begin with read-only operation and a limited user group. Add integrations after acceptance tests, then controlled writing. For broader context, read the introduction to Private AI for a company.
The practical question is: where is data at each stage, who can access it, who maintains the component and how is a correct result proven after failure? Useful primary references include the NIST AI Risk Management Framework and the GDPR data-minimisation principle.
In the acceptance record, the purchasing manager names the clause the assistant should find, the date shown in the test ERP and the reason this case needs a person’s decision. Record what each environment actually returned and whether the buyer could open the source under their own account. A difference in response time can be weighed against operating costs. Disclosure of a restricted appendix, or an invitation to approve a date change without the required written agreement, calls for a correction before the team chooses that architecture.
Checklist before approving deployment. Select one real workflow and record starting data, expected answer, permitted sources, prohibited sources and the human approval point. For each of the three models, locate identity, document storage, index, inference, integration gateway, logs and backups. For every connection, document network direction, encryption, service account, secret storage, timeout and failure mode. Name who patches the operating system, model, libraries, certificates and index; who reviews access logs; and who authorises emergency support. Acceptance tests should cover two user roles, a question with an authorised source, a question without evidence, a prohibited document, connection failure, duplicate request, model update and full recovery. Compare the models with identical criteria: source accuracy, permission enforcement, expected refusal, latency, resource use and the operator’s ability to explain a failure. Separate implementation, hardware or compute, licences, storage, monitoring, updates, recovery tests and staff time in the cost table. Choose the architecture only after this evidence exists, and document which future change would trigger a new decision.
Operational evidence after the pilot. Approve deployment with a completed operating cycle as well as an architecture diagram. Retain the configuration version, test dataset identifier, user role, permission decision, selected source passages, model version, response time and final outcome. Repeat the cycle after a service restart, secret rotation and index rebuild. Any changed result must be explainable by a version or an approved configuration change. An operator should trace one request by correlation ID and locate the failing stage without opening unnecessary personal data. The process owner confirms that the outcome still serves the work, the security owner checks permissions and data boundaries, and the technical owner checks capacity, alerts and recovery. Store this evidence with the decision matrix and review it after a material change to the model, integration, source or workload.
For the CANIMAPS DIGITAL approach to tailoring the server environment, data access and integrations to an individual business, see Private AI for businesses.
Author: Rolands Zauls. Publisher: CANIMAPS DIGITAL.