Deploy on AWS

The three constraints that decide everything

The general model is on the Docker Compose and Kubernetes pages; this page is only the AWS projection of it.

• One data directory, one writer — and the advisory lock cannot be trusted on network filesystems. On EFS (NFSv4.1) the lock does hold in the happy path — a second server against the same directory is refused — but the guarantee rides on an NFS lease. A client partitioned for longer than the lease loses the lock silently: a second writer acquires it and starts serving while the first is still alive and writing. That failure class does not exist on a single-attached EBS volume, which is why the storage answer is EBS only.
• Availability is restore time. Deploys are stop-then-start, there is no failover, and the availability story is snapshots plus (optionally) a replica to promote by hand — see the promotion runbook. No multi-AZ active-active exists to configure.
• TLS belongs to the layer in front. A bearer token is the whole credential, so the server must never be reachable over plain HTTP from outside. On AWS that layer is an ALB with an ACM certificate; the server behind it speaks HTTP on 8248 to the ALB alone.

Compute: EC2 or EKS, and why not the rest

Anything that cannot keep one EBS volume attached to one long-lived process is out.

EC2

• Keep the data directory on its own gp3 volume, separate from the root volume: snapshots, restores, and instance replacement then never touch the OS disk. Format it (XFS works well), mount it, and create the data directory owned by uid 65532 before the first start — the image is the binary alone on scratch, with no shell and no mkdir, so the host does this once:

  mkfs -t xfs /dev/nvme1n1 && mount /dev/nvme1n1 /mnt/taguru
  mkdir -p /mnt/taguru/data && chown 65532:65532 /mnt/taguru/data

• No SSH required. An instance profile with AmazonSSMManagedInstanceCore gives Session Manager access and — the same policy — lets the boot script read the API token from SSM Parameter Store (below).
• docker run --restart unless-stopped (or a systemd unit with Restart=on-failure) is the restart policy — restarts are the host's job, not the load balancer's (see the probe split).
• Instance size: memory ≈ TAGURU_CACHE_BYTES (default 512 MiB) + pinned contexts + headroom — a t3.small runs the default profile; measure with taguru estimate before scaling up.
• EBS is AZ-bound. Replacing a dead instance means recreating in the same AZ and reattaching the volume — or restoring a snapshot into another AZ (snapshots are regional). Script whichever walk you choose, and rehearse it.

EKS

• deploy/kubernetes.yaml applies unchanged. Everything the Kubernetes page says — the probe wiring, fsGroup: 65532 (which makes the volume-ownership step automatic, unlike raw EC2), the grace period — holds on EKS. The AWS-specific work is only the storage and credential plumbing:

  aws eks create-addon --cluster-name CLUSTER --addon-name aws-ebs-csi-driver

  plus a gp3 StorageClass marked default (the CSI driver provisions topology-aware, so WaitForFirstConsumer places the volume in the pod's AZ):

  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    name: gp3
    annotations: { storageclass.kubernetes.io/is-default-class: "true" }
  provisioner: ebs.csi.aws.com
  volumeBindingMode: WaitForFirstConsumer
  parameters: { type: gp3 }

• The CSI controller needs its own AWS identity — attaching a policy to the node role is not enough. Managed node groups default to an IMDS hop limit of 1, which pods cannot reach, so the controller crash-loops with "no EC2 IMDS role found". Give it EKS Pod Identity (the eks-pod-identity-agent add-on plus an association handing a role with AmazonEBSCSIDriverPolicy to kube-system/ebs-csi-controller-sa) or IRSA.
• A Recreate rollout on the same node costs seconds — the EBS volume detaches and reattaches without leaving the instance. A reschedule to another node in the same AZ adds the detach/attach round trip; another AZ needs the snapshot walk (RWO EBS cannot follow the pod across AZs).
• Fronting with an ALB: the standard route is the AWS Load Balancer Controller (Ingress → ALB, target-type: ip, certificate from ACM), with the same health-check and timeout rules as below. It needs its own Pod Identity/IRSA role — the same pattern as the CSI controller.
• Grace vs. drain: terminationGracePeriodSeconds: 60 in the manifest covers the drain plus the final flush. Keep the ALB target group's deregistration delay at or below it — a drain window longer than the pod's remaining lifetime just resets whatever is still in flight when the pod dies.

The stateless and read-pool variants (kubernetes-stateless.yaml, kubernetes-replicas.yaml) work on EKS the same way, with TAGURU_REPLICATE_URL pointed at S3 and the pod's bucket credentials granted via Pod Identity/IRSA — see the Kubernetes page.

The image: mirror ghcr.io through ECR

Keep pulls inside AWS and survive a ghcr.io outage without changing what you run.

• ECR's pull-through cache mirrors ghcr.io on first pull. ghcr requires upstream credentials even for public images: store a GitHub token (read:packages only) in Secrets Manager under the required ecr-pullthroughcache/ prefix, then

  aws ecr create-pull-through-cache-rule \
    --ecr-repository-prefix ghcr \
    --upstream-registry-url ghcr.io \
    --credential-arn arn:aws:secretsmanager:…:secret:ecr-pullthroughcache/ghcr-…

• The image reference becomes ACCOUNT.dkr.ecr.REGION.amazonaws.com/ghcr/t0k0sh1/taguru:VERSION. The cache stores the same manifest, so a digest pin carries over unchanged — pin exactly as the image guidance says, digest and all.
• Pullers need only AmazonEC2ContainerRegistryReadOnly once the cache is populated; the first pull of a new tag additionally needs ecr:BatchImportUpstreamImage — grant that to whatever refreshes versions (CI, an operator role), not to every node.

Secrets: the token never touches a manifest

TAGURU_API_TOKENS (and TAGURU_KEY_SCOPES, provider keys, …) come from a secret store at boot.

• EC2: put the token in SSM Parameter Store as a SecureString (or in Secrets Manager) and read it in the boot script with the instance role — AmazonSSMManagedInstanceCore already includes ssm:GetParameter:

  TOKENS=$(aws ssm get-parameter --name /taguru/api-tokens \
    --with-decryption --query Parameter.Value --output text)
  docker run -d --restart unless-stopped -p 8248:8248 \
    -e TAGURU_API_TOKENS="$TOKENS" -e TAGURU_LOG_FORMAT=json \
    -v /mnt/taguru/data:/data ACCOUNT.dkr.ecr.REGION.amazonaws.com/ghcr/t0k0sh1/taguru:VERSION

• EKS: External Secrets Operator or the Secrets Store CSI driver materializes the Secrets Manager entry as the taguru-keys Secret the manifest already consumes. For live key rotation, mount it as a file and pass --config — the manifest's comments carry the exact steps.
• TAGURU_PUBLIC_URL (for claude.ai-connector OAuth on /mcp) is plain configuration, not a secret — set it beside the rest of the environment.

Network: ALB in front, one port behind

Two security groups and a strict division of labor between the two health endpoints.

• Security groups: the ALB's accepts 443 from clients; the instance's (or the cluster's, for a NodePort/IP target) accepts 8248 from the ALB's security group only. Nothing else reaches the server — a bearer token is the whole credential, and the token never travels unencrypted because TLS terminates at the ALB (ACM issues and renews the certificate).
• Target group health check: GET /health — and understand what the ALB can and cannot do about a failure:

  Signal	Endpoint	Who acts
  Routing: is this target serving?	/health (503 while the write path is degraded)	The ALB routes away and back — it never restarts anything.
  Restart: is the process wedged?	/live	Docker's restart policy, systemd, or the kubelet's liveness probe. The ALB has no restart lever, so pointing its check at /live would just mask write-path failures.

• Idle timeout is a real ceiling. The server always answers within TAGURU_REQUEST_TIMEOUT_SECS (30s default) — embedding-provider attempts are cut at the request's remaining budget, and after repeated provider failures a circuit breaker fails fast instead of hanging at all. With the defaults (30s budget, 60s ALB idle timeout) nothing ever hits the ALB limit. But the budgets are coupled: if you raise TAGURU_REQUEST_TIMEOUT_SECS past 60 — say, to give a slow embedding provider its full TAGURU_EMBED_TIMEOUT_SECS ceiling — the ALB idle timeout is the knob that must move with it, or the ALB returns 504 at the idle timeout while the server is still mid-request.
• Deregistration delay ≈ the request budget. In-flight requests get at most the deregistration delay to finish once a target starts draining; connections that outlive it are reset. The 300s default is safe but slows every deploy — with a 30s request budget, a 30–60s delay drains everything a target could still be serving.
• Once the server is reachable beyond the VPC, set TAGURU_RATE_LIMIT_PER_MIN; WAF on the ALB is optional on top.

Hosting the model side on AWS too — Bedrock driving the agent through taguru-mcp or the Converse API, Titan/Cohere behind the embedding bridge — is the Amazon Bedrock page; managed agent runtimes register https://your-host/mcp through this same ALB.

Backups and DR: DLM snapshots + an export lane

Every writer is fsync+rename, so an EBS snapshot is safe at any instant — even mid-write, with the server running.

1. Automate snapshots with DLM (or AWS Backup): a lifecycle policy targeting the data volume's tag, on whatever cadence bounds your RPO. POST /flush first when you want a snapshot to contain this second's writes rather than the last flush interval's.
2. Rehearse the restore: create a volume from the snapshot, attach it, and run the image against it — XFS twins need -o nouuid to mount beside their original:

   mount -o nouuid /dev/nvme2n1 /mnt/restore
   docker run --rm -v /mnt/restore/data:/data \
     ACCOUNT.dkr.ecr.REGION.amazonaws.com/ghcr/t0k0sh1/taguru:VERSION inspect /data

   taguru inspect runs the same fully-validating load + WAL replay the server boots with; a non-zero exit means the copy is corrupt, and a snapshot you have never restored is not a backup.
3. Keep a second, environment-independent lane: taguru export --url against the running server writes portable JSONL batch streams — ship them to S3 on a schedule. They restore anywhere through taguru import, across versions and machines. Note the export's own caveat: each context's stream is internally consistent, but a multi-context export is not one point in time — the snapshot lane is.
4. DR is a copy, not a mirror: snapshots are regional and copy cross-region (DLM can schedule the copy); S3 replicates the export lane. EBS itself is AZ-bound — recovery into another AZ or region is always snapshot-restore, never reattach.

With TAGURU_REPLICATE_URL pointed at an S3 bucket the data directory also ships continuously (RPO ≈ seconds; recover with taguru restore) — that lane and the promotion runbook are on the Docker Compose and architecture pages, and they complement, not replace, the snapshot you can hand to taguru inspect.

Observability

Three surfaces, three AWS sinks — nothing needs shell access to the scratch container.

• /metrics (Prometheus text) → the CloudWatch agent's Prometheus scrape, or Amazon Managed Prometheus via an ADOT collector. The health-story metrics to alert on are the flush failures behind /health, and — if the embedding tier is wired — the circuit-breaker family (taguru_embedding_breaker_state, …) described on the Bedrock page.
• TAGURU_LOG_FORMAT=json → one JSON object per line, which the awslogs Docker log driver or the CloudWatch agent forwards to CloudWatch Logs with fields intact.
• OTEL_EXPORTER_OTLP_ENDPOINT → an ADOT collector → X-Ray (or any OTLP backend): the full retrieval span tree from Tracing, with no instrumentation to add.

Scale-out: shards, not replicas

Write scale is disjoint context sets on independent instances — the model is on the Kubernetes page.

• Each shard is a complete stack of this page: its own instance (or pod), its own EBS volume, its own snapshots. Contexts that must be searched together (groups, cross-context search reach only their own shard) cohabit on one shard.
• In front, taguru router is stateless — run it anywhere, give clients one hostname. If clients address shards directly instead, give each shard its own hostname and target group (host-based ALB rules): context names live in the path, so path-based routing cannot split them.
• Read scale is the replica pool tailing an S3 bucket (the read pool) — replicas are read-only and need only bucket credentials, not EBS.