Deploy on Google Cloud

The three constraints that decide everything

The general model is on the Docker Compose and Kubernetes pages; this page is only the Google Cloud projection of it.

• One data directory, one writer — and network filesystems are out. On Google Cloud that means Persistent Disk only, never Filestore. Filestore is NFS, the same protocol the AWS page rules out for EFS: a second writer isn't refused cleanly, it's refused (or worse, silently tolerated at the protocol level) through an NFS lock that a partition can hand away. A zonal or regional Persistent Disk enforces single-attach at the platform layer, before the advisory lock ever has to.
• Availability is restore time — and Google Cloud has a real upgrade to it. Deploys are stop-then-start, there is no failover, and the baseline story is snapshots plus (optionally) a replica to promote by hand — see the promotion runbook. A Regional Persistent Disk changes the baseline itself: it replicates synchronously to a second zone in the same region while staying a single-attach block device, and on a zone outage --force-attach moves it to a VM in the surviving zone with no snapshot restore in the path — RPO effectively zero, not "since the last snapshot." See the failover procedure below.
• TLS belongs to the layer in front. A bearer token is the whole credential, so the server must never be reachable over plain HTTP from outside. On Google Cloud that layer is the external Application Load Balancer with a Google-managed certificate; the server behind it speaks HTTP on 8248 to the load balancer alone.

Compute: GCE or GKE, and why not Cloud Run

Anything that cannot keep one block device attached to one long-lived process is out — verified against the platform, not assumed from its marketing copy.

GCE VM

• Keep the data directory on its own Persistent Disk, separate from the boot disk: snapshots, restores, and VM replacement then never touch the OS. Format, mount, and create the data directory owned by uid 65532 before the first start — the image is the binary alone on scratch, with no shell and no mkdir, so the host does this once:

  mkfs.ext4 -F /dev/disk/by-id/google-taguru-data
  mkdir -p /mnt/taguru
  mount /dev/disk/by-id/google-taguru-data /mnt/taguru
  mkdir -p /mnt/taguru/data && chown 65532:65532 /mnt/taguru/data
  echo '/dev/disk/by-id/google-taguru-data /mnt/taguru ext4 discard,defaults,nofail 0 2' >> /etc/fstab

• docker run --restart unless-stopped (or a systemd unit with Restart=on-failure) is the restart policy — restarts are the host's job, not the load balancer's (see the probe split).
• VM size: memory ≈ TAGURU_CACHE_BYTES (default 512 MiB) + pinned contexts + headroom — an e2-small runs the default profile; measure with taguru estimate before scaling up. A shared-core machine type is fine for the server itself, but building anything on the VM (not needed for normal operation — the image ships prebuilt) is a different story: a cargo build --release on an e2-small (2 vCPU burstable, 2 GiB RAM) was still linking the final binary past the half-hour mark, memory dropping under 100 MiB free at points. Extracting the published image's own binary (docker create + docker cp, matched to the VM's architecture) and copying it over took under a minute and is what any failover rehearsal should do — never compile on the box you're trying to bring back up.
• Zone recovery is a force-attach if the disk is a Regional PD. A zonal disk is zone-bound and recovery into another zone is a snapshot restore; a Regional PD detaches from the dead VM (or is force-attached over its objection) and attaches to a standby in its replica zone, data intact — see the failover procedure.

GKE

• deploy/kubernetes.yaml applies unchanged — verified, not assumed. Applied verbatim to a fresh Autopilot cluster: the pod reached 1/1 Running with readOnlyRootFilesystem, runAsUser: 65532, and fsGroup: 65532 all honored exactly as written, and GET /health/GET /live both answered 200 immediately. The only changes Autopilot made were additive: it defaulted an ephemeral-storage request/limit the manifest doesn't set, and injected seccompProfile: RuntimeDefault plus a dropped NET_RAW capability. Nothing the manifest specifies was overridden or rejected.
• Never let the PVC land on the Filestore or GCS FUSE storage classes. A GKE cluster ships standard-rwo/premium-rwo (Persistent Disk) alongside enterprise-multishare-rwx and GCS FUSE-backed classes — picking one of the latter appears to work, one replica comes up and serves, and the failure arrives later wearing the same costume the Azure page documents for azurefile: a second pod against the same network-backed volume surfaces as a permissions or lock error, not an obvious storage misconfiguration. Leave storageClassName unset (it resolves to the PD-backed default) or name a PD class explicitly.
• Two guards, two layers. On a Persistent Disk, a second pod on another node is refused by the platform (the volume stays attached where it is), and a second pod on the same node — which ReadWriteOnce does not prevent — is refused by the server's own lock with an explicit message (data directory is held by another taguru process). Both refusals are loud and correct; neither exists on a network-backed share.
• Regional PD as a GKE StorageClass: verified live —

  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    name: regional-pd
  provisioner: pd.csi.storage.gke.io
  parameters:
    type: pd-balanced
    replication-type: regional-pd
  volumeBindingMode: WaitForFirstConsumer

  A PVC naming this class provisioned and bound cleanly on Autopilot with no other changes to the manifest — only the PVC's storageClassName needs to change to move a deployment from zonal to regional storage.
• Fronting with an external Application Load Balancer: a BackendConfig pointed at GET /health plus a standard Ingress is enough — see the section below for the verified health-check wiring.
• Grace vs. drain: terminationGracePeriodSeconds: 60 in the manifest covers the drain plus the final flush. Keep the load balancer's connection-draining timeout at or below it — a drain window longer than the pod's remaining lifetime just resets whatever is still in flight when the pod dies.

The stateless and read-pool variants (kubernetes-stateless.yaml, kubernetes-replicas.yaml) work on GKE the same way, with TAGURU_REPLICATE_URL pointed at a GCS bucket and the pod's credentials granted via Workload Identity — see the Kubernetes page and Backups and DR below for the bucket side, verified end to end.

The image: mirror ghcr.io through Artifact Registry

Keep pulls inside Google Cloud and survive a ghcr.io outage without changing what you run.

• Artifact Registry's remote repository mode mirrors ghcr.io on first pull — verified live, a cold pull through a freshly created remote repository resolved and cached the published image with no extra configuration:

  gcloud artifacts repositories create taguru-mirror \
    --repository-format=docker --mode=remote-repository \
    --remote-docker-repo=https://ghcr.io \
    --location=REGION

  The image reference becomes REGION-docker.pkg.dev/PROJECT/taguru-mirror/t0k0sh1/taguru:VERSION. The cache stores the same manifest, so a digest pin carries over unchanged — pin exactly as the image guidance says.
• Pullers authenticate with the node's (or pod's, via Workload Identity) service account holding roles/artifactregistry.reader — no registry passwords in the boot script; on GKE, the default node service account already has this once the repository exists in the same project.

Persistent Disk, and the Regional PD upgrade

A zonal disk is the default fit; a Regional PD is the one change that turns a zone outage into a reattach instead of a restore.

• pd-balanced is the default choice for the data directory — pd-ssd if the write volume genuinely needs the extra IOPS, measured, not assumed.
• A Regional PD is created with two replica zones, not a region:

  gcloud compute disks create taguru-data \
    --region=REGION --replica-zones=ZONE-A,ZONE-B \
    --type=pd-balanced --size=10GB

  It attaches, formats, and mounts exactly like a zonal disk — nothing about the server's use of /data changes. The difference only shows up when a zone goes away.
• On GKE, the same trade is one StorageClass parameter (replication-type: regional-pd, shown in Compute above) — verified to provision and bind without touching the deployment manifest at all.
• Regional PD is not a backup. It is one disk with one write history, replicated — a bad write replicates too. Snapshots and the export lane (below) are still the only defense against corruption or accidental deletion; Regional PD only removes zone failure from the list of things a restore is needed for.

Regional PD failover: the procedure

Rehearsed on a live VM pair, not described from the API reference.

1. Normal operation: the Regional PD is attached read-write to one VM, in one of its two replica zones — exactly like a zonal disk from the server's point of view. POST /flush before a planned failover so the surviving zone's copy is current to the second, not the last flush interval.
2. The zone with the attached VM becomes unavailable (an outage, or a deliberate drain for maintenance). The disk is still attached to the now-unreachable VM as far as the platform's bookkeeping is concerned.
3. Force-attach from a VM in the surviving zone:

   gcloud compute instances attach-disk taguru-vm-standby \
     --zone=ZONE-B --disk=taguru-data --disk-scope=regional \
     --force-attach

   --force-attach succeeds even though the disk shows as attached elsewhere, and the platform keeps trying to detach it from the dead VM in the background — the standby does not wait on that.
4. Mount and start the server on the standby exactly as on the primary — same UID, same mount path. No snapshot restore, no taguru restore: the block device itself is current as of the last synchronously-replicated write.
5. Verify with taguru inspect /data before serving traffic — the same fully-validating load + WAL replay the server boots with. A non-zero exit means investigate before routing anything at the standby.
6. Rehearse this before an outage forces it. The mechanics are simple, but the first time anyone runs --force-attach should not be during an actual zone loss — the flag name alone invites hesitation at 3 a.m.

On GKE, the same disk resource is what a regional-pd StorageClass provisions underneath; the CSI driver performs the equivalent force-attach when the scheduler moves the pod to a node in the surviving zone after the original node becomes NotReady — the manual procedure above is what to reach for on a bare VM, or to understand what the driver is doing on your behalf.

Secrets: Secret Manager feeds the boot

TAGURU_API_TOKENS (and TAGURU_KEY_SCOPES, the embedding key, …) come from Secret Manager at boot; nothing lands in a manifest or a template.

• GCE VM: the attached service account plus roles/secretmanager.secretAccessor is the whole credential chain — the boot script reads the secret directly, no token exchange to write by hand. Write it to a permission-restricted file rather than -e: a command-line environment variable lands in docker inspect and any other user's ps output on the same host, not just shell history.

  install -m 600 /dev/null /etc/taguru.env
  echo "TAGURU_API_TOKENS=$(gcloud secrets versions access latest --secret=taguru-api-tokens)" >> /etc/taguru.env
  echo "TAGURU_LOG_FORMAT=json" >> /etc/taguru.env
  docker run -d --restart unless-stopped -p 8248:8248 \
    --env-file /etc/taguru.env \
    -v /mnt/taguru/data:/data REGION-docker.pkg.dev/PROJECT/taguru-mirror/t0k0sh1/taguru:VERSION

• GKE: the Secret Manager add-on (gcloud container clusters update --enable-secret-manager) mounts a secret directly through a SecretProviderClass — verified live, including one trap that cost real debugging time: the CSI driver name is secrets-store-gke.csi.k8s.io, not the community secrets-store.csi.k8s.io most tutorials reference. Get the driver name wrong and the pod sticks in ContainerCreating with driver name … not found in the list of registered CSI drivers — a message that reads like the add-on isn't enabled at all, when it is.

  apiVersion: secrets-store.csi.x-k8s.io/v1
  kind: SecretProviderClass
  metadata:
    name: taguru-secrets
  spec:
    provider: gke
    parameters:
      secrets: |
        - resourceName: "projects/PROJECT_NUMBER/secrets/taguru-api-tokens/versions/latest"
          path: "taguru-api-tokens"

  Grant access straight to the pod's Kubernetes service account, no separate Google service account needed:

  gcloud secrets add-iam-policy-binding taguru-api-tokens \
    --role=roles/secretmanager.secretAccessor \
    --member="principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/KSA_NAME"

  Mount it as a file and pass --config for live key rotation — the manifest's comments carry the exact steps.
• TAGURU_PUBLIC_URL (for claude.ai-connector OAuth on /mcp) is plain configuration, not a secret — set it beside the rest of the environment.

Embeddings: Vertex AI needs a bridge

Verified against the live API, not the desk-analysis assumption: Vertex's OpenAI-compatible surface covers chat, not embeddings.

• The OpenAI-compatible /embeddings endpoint does not work. Vertex publishes an OpenAI-compatible surface at …/endpoints/openapi/chat/completions that answers real chat requests correctly — verified live. The equivalent …/endpoints/openapi/embeddings returned 500 INTERNAL for every embedding model tried (gemini-embedding-001, text-embedding-005, with and without the google/ publisher prefix). The embed client (src/embedding.rs) speaks exactly this OpenAI-shaped protocol and nothing else, so a bridge is required — this is the one cloud, alongside Bedrock, where the model has to sit behind a proxy rather than take the client's request directly.
• LiteLLM bridges it cleanly — for one model. Pointed at vertex_ai/gemini-embedding-001 with Application Default Credentials, LiteLLM's OpenAI-compatible /v1/embeddings returned correct, distinct vectors for distinct inputs, and fed a real taguru calibrate run without any client-side changes.

  model_list:
    - model_name: gemini-embedding-001
      litellm_params:
        model: vertex_ai/gemini-embedding-001
        vertex_project: PROJECT
        vertex_location: REGION
        dimensions: 768   # see the note below — pin this

• text-embedding-005 through the same bridge is broken, not just untested. Verified live: every request to vertex_ai/text-embedding-005 through LiteLLM 1.95.0 returned the identical vector regardless of input text — cosine 1.0 between embeddings of completely unrelated strings. The native Vertex :predict endpoint embeds this model correctly; the failure is specific to this LiteLLM version's handling of it. Use gemini-embedding-001 until this is independently confirmed fixed, and treat "the bridge returns 200 with plausible-looking numbers" as insufficient — diff two embeddings of clearly different inputs before trusting a bridge configuration.
• Pin dimensions in the bridge config — the client never asks for one. gemini-embedding-001 defaults to 3072 dimensions; the embed client's request body is {"model", "input"} only, with no dimensions field to negotiate a smaller width. Setting dimensions: N in LiteLLM's litellm_params (verified: it maps to Vertex's own outputDimensionality) fixes the width server-side with no code change needed — pick a width once, before the first gloss is embedded, since changing it later means re-embedding everything.
• Calibrate before trusting any floor for this model — measured overlap, not a clean number. A real taguru calibrate run against gemini-embedding-001, at both 768 and 3072 dimensions, over a small mixed Japanese corpus (sake brands, a mountain, an author, a castle) returned OVERLAP both times: the best non-expected candidate's cosine landed inside the range of expected-match cosines, most often because a subject's own gloss text names its related concept directly (a brand's gloss reads "…産地。秋田県", and "秋田県" is itself a concept in the corpus). Unlike Bedrock's Titan V2, where calibration reliably finds a clean gap around 0.2, Vertex embeddings on a small corpus did not separate cleanly at either width tried. This is exactly the case taguru calibrate is built to refuse to paper over — see the calibration walkthrough. Run it against your own, larger corpus before picking TAGURU_SEMANTIC_FLOOR; no table entry here would be honest.
• Document extraction (taguru extract) can use Vertex's OpenAI-compatible /chat/completions directly for one-shot runs — TAGURU_EXTRACT_API_KEY=$(gcloud auth print-access-token) — since that surface works and the credential only needs to last one run. Steady use should still go through the same LiteLLM bridge, since a raw access token expires in about an hour and TAGURU_EXTRACT_API_KEY is read once at process start. Model credentials stay in the offline producer's environment, never on the server — see Document extraction.

The agent side needs none of this: Claude on Vertex AI (CLAUDE_CODE_USE_VERTEX=1) drives taguru-mcp over stdio unchanged, and a managed agent runtime registering https://your-host/mcp rides through the same load balancer this page fronts the server with.

Network: external load balancer in front, one port behind

One firewall rule and a strict division of labor between the two health endpoints — verified against a live backend.

• Reachability — the source ranges differ by load balancer type. The frontend accepts 443 from clients; the backend's firewall rule accepts 8248 from the load balancer itself, and what that means depends on which external Application Load Balancer is in front. A global one (what GKE's default Ingress class provisions, and what this page's own testing used) sends both health checks and proxied traffic from Google's well-known ranges (130.211.0.0/22, 35.191.0.0/16) — allow those two and nothing else reaches the server. A regional external Application Load Balancer is Envoy-based: health checks still come from those same ranges, but proxied traffic to the backend originates from your VPC's proxy-only subnet instead — allowing only the health-check ranges leaves the health check passing while every real request is dropped by the firewall. Allow the proxy-only subnet's CIDR too if you provision a regional one. Either way, a bearer token is the whole credential, and it never travels unencrypted because TLS terminates at the load balancer.
• A BackendConfig pointed at GET /health, on GKE — verified live: with a custom health check configured this way, the backend service reported healthState: HEALTHY against the running pod within seconds. Then keep the division of labor straight:

  Signal	Endpoint	Who acts
  Routing: is this backend serving?	/health (503 while the write path is degraded)	The load balancer routes away and back — it never restarts anything.
  Restart: is the process wedged?	/live	Docker's restart policy, systemd, or the kubelet's liveness probe. The load balancer has no restart lever, so pointing its check at /live would just mask write-path failures.

• The backend service timeout defaults to 30 seconds — verified via gcloud compute backend-services create --help — which happens to equal TAGURU_REQUEST_TIMEOUT_SECS's own default. That's a coincidence worth not relying on: the server always answers within its own request budget (embedding attempts are cut at the request's remaining time, not their own nominal ceiling), so with both left at their defaults nothing ever hits the load balancer's limit. Raise TAGURU_REQUEST_TIMEOUT_SECS — say, to give a slow embedding bridge more room across retries — and --timeout on the backend service has to move with it, or the load balancer returns 504 while the server is still mid-request and would have answered.
• Connection draining ≥ the request budget. When a backend leaves the pool, in-flight requests get exactly the configured drain timeout to finish, then the connection is cut. Size it to TAGURU_REQUEST_TIMEOUT_SECS or above, and keep it at or below the pod's grace period on GKE (see grace vs. drain).
• The Google-managed certificate needs a real, resolvable hostname before it will issue — point DNS at the load balancer's static IP first. On GKE that's a ManagedCertificate resource referenced from the Ingress; on a VM fronted by a manually configured load balancer it's the same certificate manager under a plain gcloud call instead:

  gcloud compute ssl-certificates create taguru-cert \
    --domains=HOST --global

  Either path stays Provisioning until the domain resolves and the challenge succeeds, which can take longer than the first impatient check.
• Once the server is reachable beyond the VPC, set TAGURU_RATE_LIMIT_PER_MIN; Cloud Armor in front is optional on top.

Backups and DR: snapshot schedules + an export lane

Every writer is fsync+rename, so a Persistent Disk snapshot is safe at any instant — even mid-write, with the server running.

1. Automate snapshots with a resource policy — verified CLI shape:

   gcloud compute resource-policies create snapshot-schedule taguru-snap \
     --region=REGION --daily-schedule --start-time=13:00 \
     --max-retention-days=14

   POST /flush first when you want a snapshot to contain this second's writes rather than the last flush interval's.
2. Rehearse the restore: create a disk from the snapshot, attach it to a running VM, mount it, and run the image against it:

   gcloud compute disks create taguru-restore-test --source-snapshot=SNAPSHOT --zone=ZONE
   gcloud compute instances attach-disk RESTORE_VM --zone=ZONE --disk=taguru-restore-test
   mkdir -p /mnt/restore && mount /dev/disk/by-id/google-taguru-restore-test /mnt/restore
   docker run --rm -v /mnt/restore/data:/data \
     REGION-docker.pkg.dev/PROJECT/taguru-mirror/t0k0sh1/taguru:VERSION inspect /data

   taguru inspect runs the same fully-validating load + WAL replay the server boots with; a non-zero exit means the copy is corrupt, and a snapshot you have never restored is not a backup.
3. Regional PD is the zone lane, snapshots are the region and corruption lane — see the failover procedure for the former; a Regional PD is still one write history, so it does not replace the latter.
4. Keep a second, environment-independent lane: taguru export --url against the running server writes portable JSONL batch streams — ship them to Cloud Storage on a schedule. They restore anywhere through taguru import, across versions and machines. Note the export's own caveat: each context's stream is internally consistent, but a multi-context export is not one point in time — the snapshot lane is.

With TAGURU_REPLICATE_URL=gs://BUCKET the data directory also ships continuously — verified end to end: a running server shipped its first generation to a bucket within a second of boot, and taguru restore --out DIR gs://BUCKET into an empty directory followed by taguru inspect came back clean, no corruption, no manual repair. Credentials ride Application Default Credentials on a VM or Workload Identity on GKE — nothing cloud-specific to configure beyond the bucket URL. This lane complements, not replaces, the snapshot you can hand to taguru inspect — see the promotion runbook for the read-replica side of the same mechanism.

Observability

Three surfaces, three Google Cloud sinks — verified against a live GKE deployment, including one nesting detail worth knowing before it surprises you.

• /metrics (Prometheus text format, confirmed against a live pod) → Google Cloud Managed Service for Prometheus, via a PodMonitoring resource on GKE or the Ops Agent's Prometheus receiver on a VM. The health-story metrics to alert on are the flush failures behind /health and — with the embedding tier wired — the circuit-breaker family described on the Bedrock page.
• TAGURU_LOG_FORMAT=json → Cloud Logging ingests it correctly, but not entirely the way the field names suggest. Verified against real log entries from a GKE pod: the level field does get mapped automatically to Cloud Logging's severity (a WARN line surfaces as severity: WARNING in a gcloud logging read query) — but the human-readable text stays nested at jsonPayload.fields.message, not the top-level jsonPayload.message Cloud Logging's console uses for its default one-line summary. Filtering and alerting on severity works out of the box; expect to expand each entry (or write a log-based view) to read the message itself.
• OTEL_EXPORTER_OTLP_ENDPOINT → an OpenTelemetry collector → Cloud Trace: the full retrieval span tree from Tracing, with no instrumentation to add.

Scale-out: shards, not replicas

Write scale is disjoint context sets on independent instances — the model is on the Kubernetes page.

• Each shard is a complete stack of this page: its own VM (or pod), its own Persistent Disk, its own snapshots. Contexts that must be searched together (groups, cross-context search reach only their own shard) cohabit on one shard.
• In front, taguru router is stateless — run it anywhere, give clients one hostname. If clients address shards directly instead, give each shard its own hostname behind the load balancer: context names live in the path, so path-based routing cannot split them.
• Read scale is the replica pool tailing a bucket (the read pool) — replicas are read-only and need only storage credentials, not a disk, verified via the same TAGURU_REPLICATE_URL/taguru restore path as Backups and DR above.