Make an AI App Reliable: Access, Cost and Deployment

A document assistant that works on your laptop is a useful prototype. Operating it for other people introduces a different set of responsibilities: identity, permissions, request limits, provider failures, source updates, monitoring, and rollback.

This lesson turns those concerns into a deployment-readiness plan. The capstone remains an educational command-line program; it is not a web service or a production deployment package.

Establish trusted identity before retrieval

The most important distinction in the capstone’s audience exercise is where permissions come from. A command-line argument lets us simulate public and staff cases. In a deployed application, a client-provided staff flag must not grant access.

Authenticate users through an appropriate server-side mechanism, derive permissions from a trusted session or policy decision, and enforce document access before retrieval and context assembly. Check access again for source links or downloads. Protecting the answer while exposing the original private document would not preserve the boundary.

Plan for revocation. If a user loses access or a document is withdrawn, cached context, search indexes, and generated-answer caches must not continue to expose it. Source metadata and update/delete workflows are part of the application’s correctness.

Bound what the application can do

Set input length, upload size, accepted document formats, retrieved-context size, output limits, and request concurrency according to the workload. Validate formats before treating uploaded material as source text. A PDF parser, a web fetcher, and a model API are separate components with separate failure modes.

Do not let retrieved text grant tool permissions or choose arbitrary network destinations. The OWASP prompt-injection guidance is relevant even when a user never writes a malicious prompt: an external document can carry it.

Keep provider credentials on the server. Separate deployment secrets from the document corpus and logs. Give service accounts only the access required for their tasks. For additional operational context, see our guide to coding-agent secret leakage.

Handle failures without uncontrolled retries

A timeout, rate limit, invalid request, and unsupported output format should not all take the same path. Decide which failures can be retried, how many attempts are allowed, and what the user sees when the budget is exhausted.

Provider rate-limit guidance can inform backoff and request pacing. Respect applicable Retry-After instructions. Add jitter when appropriate so multiple clients do not retry in lockstep. Track the total operation budget rather than resetting it on every internal call.

The teaching example uses no automatic retry. That makes its failure behavior easy to inspect and prevents a beginner’s experiment from repeatedly spending on a bad request. Adding retries is a design change that needs its own tests.

A client timeout does not always prove that remote work stopped. Avoid promising a hard billing cap solely because a local timeout elapsed. Track actual provider usage and account-level controls where available.

Measure what helps you diagnose a problem

Useful events include a request ID, component durations, retrieval configuration, permitted source IDs, result status, provider status, retry count, model identifier, and usage information where returned. Record enough to distinguish slow retrieval from slow generation.

Avoid storing full questions, private source text, or credentials by default. Logs and traces are data stores that need access controls and retention rules. OpenTelemetry’s trace concepts explain how spans can represent stages of a request.

Watch quality alongside availability. A service can return HTTP 200 quickly while producing more unsupported answers after a source or model change. Use the evaluation set from the previous lesson as a release check, then add a reviewed process for learning from real failures.

Deploy in stages with a rollback path

To turn the CLI into a service, you would need an HTTP boundary, authentication, TLS termination, process management, resource limits, and a deployment environment. FastAPI’s deployment concepts describes concerns such as HTTPS, startup, restarts, replication, and memory. Those responsibilities still apply if you choose another framework.

First run the service in staging with synthetic or approved data. Test authentication, cross-user access, malformed inputs, provider outages, concurrency, and log contents. Validate backups and source-index rebuilds where the architecture requires them.

Keep the last working application, prompt, retrieval configuration, and compatible index snapshot identifiable. A rollback that restores code but leaves an incompatible vector index can remain broken. Model availability changes may require a planned replacement rather than a simple version rollback.

Our AWS deployment comparison discusses hosting choices. Choose hosting after defining the service’s operating requirements; the learning-path capstone does not require an AWS account.

Exercise: write and test a release gate

Create a short deployment record with the intended audience, permitted source types, maximum request budget, failure behavior, secrets location, logs retained, and rollback owner. Use explicit decisions rather than “secure by default” as a placeholder.

Write five staging checks: an unauthenticated request, a public user asking for staff data, a provider timeout, an oversized request, and a source permission being revoked. Record the expected response and which component enforces it. For a CLI-only project, label these as planned checks until a real service exists.

You have completed this lesson when every release gate has an owner and a way to verify it, and you can explain what the current prototype does not implement. Do not expose the downloadable CLI directly to the internet and describe a local test as deployment evidence.

Documentation checked November 8, 2026. This is a deployment-planning lesson; no hosted service or production load test is claimed.