feat(terraform): add GCP onyx composition and README (#15139)

deployment/terraform/modules/gcp/README.md

@@ -0,0 +1,422 @@
+# Onyx GCP modules
+
+## Status
+
+These modules pass validation against the `google` provider schema. Each one
+has a `terraform test` suite that plans it against a mocked provider. No one
+has applied them to a live project yet. Treat the first deployment as a first
+deployment.
+
+## Overview
+
+This directory contains Terraform modules to provision the core GCP
+infrastructure for Onyx:
+
+- `vpc`: a custom-mode VPC with one subnet, secondary ranges for pods and
+  services, Cloud NAT for egress, flow logs, and Private Service Access for
+  Cloud SQL and Memorystore
+- `gke`: a regional GKE Standard cluster with Workload Identity, private
+  nodes, pools for the application and the document index, optional GPU and
+  sandbox pools, and the Onyx namespace and service account
+- `postgres`: a Cloud SQL for PostgreSQL instance with a private IP only, and
+  five alert policies
+- `redis`: a Memorystore for Redis instance with AUTH on, and five alert
+  policies
+- `gcs`: a bucket for the Onyx file store, with versioning, lifecycle rules
+  and public access prevention
+- `cloud-armor`: a Cloud Armor policy with the OWASP Core Rule Set, two rate
+  limits, and optional IP and country rules
+- `onyx`: a higher-level composition that wires the above together
+
+Use the `onyx` module for a working cluster with sane defaults. Use the
+individual modules when you need more control.
+
+These mirror the AWS modules in `../aws` and the Azure modules in `../azure`.
+Read [Differences from the AWS and Azure
+modules](#differences-from-the-aws-and-azure-modules) before you port a
+configuration across.
+
+## Consuming these modules from another repository
+
+The quickstart below uses local paths. To consume the modules from a different
+repository, point `source` at this repository and pin a ref:
+
+```hcl
+module "vpc" {
+  source     = "git::https://github.com/onyx-dot-app/onyx.git//deployment/terraform/modules/gcp/vpc?ref=tf-gcp/v1.0.0"
+  project_id = "my-project"
+  region     = "us-east1"
+}
+```
+
+GCP releases have tags of the form `tf-gcp/vX.Y.Z`. Their versions are
+independent of the AWS modules (`tf/`), the Azure modules (`tf-azure/`) and
+Onyx product releases. A commit sha also works as a `ref`. Automated consumers
+should use a sha: no one can move a sha, but someone can move a tag.
+
+Pin something. Without a `ref`, Terraform tracks the default branch, and an
+unrelated merge can change your infrastructure.
+
+## Quickstart (copy/paste)
+
+This module declares no `provider` block. Your root module configures the
+`google` provider and the `kubernetes` provider.
+
+```hcl
+locals {
+  project_id = "my-project"
+  region     = "us-east1"
+  # Supply this from a secret store or TF_VAR_ in anything but a scratch stack.
+  postgres_password = "your-postgres-password"
+}
+
+provider "google" {
+  project = local.project_id
+  region  = local.region
+}
+
+module "onyx" {
+  source = "./modules/gcp/onyx"
+
+  name       = "onyx"
+  project_id = local.project_id
+  region     = local.region
+  size       = "medium"
+
+  postgres_password = local.postgres_password
+
+  # Required. A public API server with no authorized networks is open to every
+  # address on the internet. The module refuses to build one unless you
+  # restrict it, make it private, or say the exposure is intended.
+  master_authorized_networks = [
+    { cidr_block = "203.0.113.0/24", display_name = "office" },
+  ]
+}
+
+# An access token for whoever runs Terraform. It is not keyed on the cluster,
+# so Terraform can read it at plan time, before the cluster exists.
+data "google_client_config" "default" {}
+
+# The composition creates a namespace and a service account, so the kubernetes
+# provider needs the cluster. It reads the address and CA from the module.
+provider "kubernetes" {
+  host                   = module.onyx.cluster_endpoint
+  cluster_ca_certificate = base64decode(module.onyx.cluster_ca_certificate)
+  token                  = data.google_client_config.default.access_token
+}
+
+output "gcs_bucket_name" {
+  value = module.onyx.gcs_bucket_name
+}
+
+output "postgres_host" {
+  value = module.onyx.postgres_host
+}
+
+output "redis_host" {
+  value = module.onyx.redis_host
+}
+```
+
+Then:
+
+```bash
+terraform init
+terraform apply
+```
+
+Configure the `kubernetes` provider from the module outputs, as above. Do not
+use a `data "google_container_cluster"` keyed on the cluster name: on the first
+apply the cluster does not exist, and the read fails at plan. Do not add
+`depends_on` to that data source or to the module either. It defers reads to
+apply time, and plan values that must be known become unknown. Module outputs
+carry the dependency without either problem. The Azure modules use the same
+pattern for the same reason.
+
+## T-shirt sizing
+
+`size` sets every compute and data-plane knob together. Any individual sizing
+variable set to a non-null value overrides its tier default.
+
+| | small | medium | large |
+|---|---|---|---|
+| main pool machine | `n2-standard-8` | `n2-standard-16` | `n2-standard-16` |
+| main pool nodes | 1-3 | 1-5 | 2-8 |
+| index pool machine | `n2-highmem-4` | `n2-highmem-8` | `n2-highmem-16` |
+| index pool disk | 256 GB | 512 GB | 1024 GB |
+| database tier | `db-custom-2-8192` | `db-custom-2-8192` | `db-custom-4-16384` |
+| database disk | 64 GB | 128 GB | 256 GB |
+| cache | 5 GB | 10 GB | 20 GB |
+
+Roughly: small suits pilots and small teams, medium a department or company,
+large an org-wide deployment. Node counts are for the whole pool, not per zone.
+The index pool is memory-optimised at every tier because on GCP it carries the
+document index itself.
+
+### Using an existing network
+
+```hcl
+module "onyx" {
+  source = "./modules/gcp/onyx"
+
+  project_id     = "my-project"
+  region         = "us-east1"
+  create_network = false
+
+  network_id          = "projects/my-project/global/networks/shared"
+  subnet_id           = "projects/my-project/regions/us-east1/subnetworks/gke"
+  pods_range_name     = "gke-pods"
+  services_range_name = "gke-services"
+
+  postgres_password          = local.postgres_password
+  master_authorized_networks = [{ cidr_block = "203.0.113.0/24" }]
+}
+```
+
+The network must already have a Private Service Access connection, or Cloud
+SQL and Memorystore fail to create. The nodes have no public IPs, so the
+subnet also needs a Cloud NAT for egress. `enable_flow_logs` does nothing here;
+configure flow logs on your own subnet.
+
+## What each module does
+
+### `onyx`
+
+Enables the project APIs, then wires the modules below together with t-shirt
+sizing. Its outputs carry everything the Helm chart needs. One
+`deletion_protection` switch guards the cluster, database, cache, bucket and
+Cloud Armor policy.
+
+### `vpc`
+
+A custom-mode network and one subnet. The subnet has secondary ranges for pods
+and services, and Private Google Access, so nodes reach Google APIs without the
+NAT. Cloud NAT gives private nodes egress. The module reserves a Private
+Service Access range and peers it. Its `private_service_access_network_id`
+output arrives only once the peering exists, so Cloud SQL and Memorystore wait
+for it.
+
+### `gke`
+
+A regional GKE Standard cluster with Dataplane V2, private nodes, Workload
+Identity and the Gateway API. The nodes run as a dedicated service account with
+the minimum roles. It creates a main pool, a tainted document-index pool, and
+optional GPU and Craft sandbox pools. It also creates the `onyx` namespace and
+the `onyx-workload-access` service account.
+
+### `postgres`
+
+A Cloud SQL for PostgreSQL instance with a private IP only, TLS enforced,
+backups and point-in-time recovery. It creates the `onyx` database. It sets the
+password of the `postgres` user that Cloud SQL ships.
+
+### `redis`
+
+A Memorystore for Redis instance on the Private Service Access range. AUTH is
+always on. The eviction policy is `volatile-lru`, because Celery broker keys
+carry no TTL and an `allkeys-*` policy would drop queued tasks. RDB snapshots
+are on.
+
+### `gcs`
+
+A bucket with uniform access, public access prevention, versioning, and a
+lifecycle rule for old versions. It grants the workload identity principal
+object access and the narrow read that Onyx needs at startup.
+
+### `cloud-armor`
+
+A backend security policy with the OWASP Core Rule Set at sensitivity 1, an API
+rate limit, a global rate limit and Adaptive Protection. It takes effect only on
+an L7 load balancer; see below.
+
+## Differences from the AWS and Azure modules
+
+Most of these follow from the platform rather than from taste.
+
+| | AWS | Azure | GCP |
+|---|---|---|---|
+| document index | managed OpenSearch, or in-cluster | in-cluster only | in-cluster only; GCP has no managed OpenSearch |
+| cache | ElastiCache | Azure Managed Redis, off by default | Memorystore for Redis, on by default |
+| pod identity | IRSA: assume a role | federate a managed identity to a service account | grant IAM roles straight to the Kubernetes service account |
+| WAF attachment | ALB | Application Gateway | L7 load balancer through a BackendConfig or GCPBackendPolicy |
+| flow logs | on, to CloudWatch | opt-in, needs a storage account | on, to Cloud Logging |
+| project APIs | none | none | enabled by the composition |
+| provider config | declared inside the composition | declared by your root module | declared by your root module |
+
+**Managed Redis works here with no extra settings.** Onyx uses database 0 for
+the app, 14 for Celery results and 15 for the Celery broker. Memorystore for
+Redis is one primary endpoint with the default 16 databases. The Onyx defaults
+work unchanged. Azure Managed Redis has only database 0, and the Redis Cluster
+products shard keys, which breaks Celery with `CROSSSLOT`. So the composition
+turns Memorystore on by default. Set `enable_redis = false` to use the
+in-cluster Redis.
+
+**The document index runs in the cluster.** GCP has no managed OpenSearch. The
+chart's OpenSearch subchart runs on the tainted index pool.
+
+**Workload Identity needs no Google service account.** GKE Workload Identity
+Federation lets IAM name a Kubernetes service account directly. The bucket
+grant goes to the `workload_identity_principal` output. There is no Google
+service account to impersonate and no annotation on the Kubernetes service
+account. Use the same output to grant access to other resources.
+
+**Cloud Armor attaches only to an L7 load balancer.** Put the
+`cloud_armor_policy_name` output in a GKE `BackendConfig`
+(`spec.securityPolicy.name`) for Ingress, or in a `GCPBackendPolicy`
+(`spec.default.securityPolicy`) for Gateway. The chart's ingress-nginx
+controller sits behind a Service of type `LoadBalancer`, which is an L4
+passthrough load balancer. That load balancer cannot carry the policy. The
+policy then protects nothing, and nothing reports it. This is the same trap as
+the Azure WAF policy without an Application Gateway.
+
+**Flow logs are on by default.** GCP writes them to Cloud Logging and needs no
+extra infrastructure. Azure needs a Network Watcher and a storage account, so
+it keeps them off.
+
+**The composition enables the project APIs.** A new project has most of them
+off. `enable_project_apis` turns on Compute, GKE, Cloud SQL Admin, Memorystore,
+Service Networking, Storage, Monitoring, Logging, IAM and Resource Manager. It
+never turns them off on destroy, because other workloads may use them.
+
+## Installing the Onyx Helm chart (after Terraform)
+
+```bash
+gcloud container clusters get-credentials "$(terraform output -raw cluster_name)" \
+  --location "$(terraform output -raw location)" --project my-project
+```
+
+**Install into the `onyx` namespace.** A Kubernetes service account belongs to
+one namespace, and the `gke` module creates the one with the bucket grant in
+`onyx`. A release in a different namespace references an account that does not
+exist there, and the API and Celery pods never start.
+
+Terraform creates the namespace itself, because the service account needs it
+and the Helm install happens afterwards. So the release joins that namespace
+and does not create one:
+
+```bash
+helm install onyx onyx/onyx --namespace onyx -f values.yaml
+```
+
+Set `create_workload_namespace = false` if something else already creates it.
+
+Four pieces of chart configuration are necessary. Their defaults do not work
+here, so a plain `helm install` does not use the infrastructure that Terraform
+built.
+
+### 1. Point the workloads at the workload service account
+
+The chart runs every Onyx pod, including the API server and the Celery
+workers, as `serviceAccount.name`. The bucket grant names that account, so no
+pod label and no annotation are necessary. Without it, the pods run as
+`default` and every file store call gets `403`.
+
+```yaml
+serviceAccount:
+  create: false
+  name: onyx-workload-access   # the gke module's default, in namespace onyx
+```
+
+### 2. Point the file store at the bucket
+
+Turn off the bundled MinIO and set the GCS keys. The chart copies every
+non-empty `configMap` key into the pod environment, so no chart change is
+necessary.
+
+```yaml
+minio:
+  enabled: false
+
+configMap:
+  FILE_STORE_BACKEND: "gcs"
+  GCS_FILE_STORE_BUCKET_NAME: "<gcs_bucket_name output>"
+  GCS_PROJECT_ID: "<gcs_project_id output>"
+```
+
+Leave `GCS_SERVICE_ACCOUNT_KEY_PATH` and `GCS_SERVICE_ACCOUNT_KEY_JSON` unset.
+Onyx then uses Application Default Credentials, which pick up the workload
+identity from step 1.
+
+### 3. Point Onyx at Cloud SQL and Memorystore
+
+```yaml
+postgresql:
+  enabled: false
+
+redis:
+  enabled: false
+
+configMap:
+  POSTGRES_HOST: "<postgres_host output>"
+  POSTGRES_PORT: "5432"
+  POSTGRES_DB: "onyx"            # postgres_db_name output
+  REDIS_HOST: "<redis_host output>"
+  REDIS_PORT: "6379"             # redis_port output
+
+auth:
+  postgresql:
+    existingSecret: "onyx-postgresql"   # keys: username, password
+  redis:
+    existingSecret: "onyx-redis"        # key: redis_password = redis_auth_string output
+```
+
+Set `POSTGRES_DB`. Without it, Onyx uses the `postgres` database that Cloud SQL
+ships, and the `onyx` database stays empty.
+
+Create the two secrets in the `onyx` namespace from the Terraform outputs.
+Do not put the passwords in `values.yaml`. Cloud SQL accepts only encrypted
+connections, and Onyx encrypts by default, so no TLS setting is necessary. With
+`redis_transit_encryption_enabled = true`, also set `REDIS_PORT: "6378"` and
+the chart's `redisTls` values, with the CA from the redis module's
+`server_ca_certs` output.
+
+### 4. Send the document index to its own node pool
+
+The `gke` module taints the index pool `document-index=true:NoSchedule`, so
+nothing else lands on it. The OpenSearch subchart sets no tolerations of its
+own. Without this, it goes onto the main pool and the index pool stays empty:
+
+```yaml
+opensearch:
+  nodeSelector:
+    onyx.app/workload: document-index
+  tolerations:
+    - key: document-index
+      operator: Equal
+      value: "true"
+      effect: NoSchedule
+```
+
+Set `index_node_pool_enabled = false` to run the index on the main pool and
+skip this.
+
+## Testing
+
+Every module has a `terraform test` suite that plans it against a mocked
+provider. The suites need no GCP project and no credentials:
+
+```bash
+cd deployment/terraform/modules/gcp/vpc
+terraform init -backend=false
+terraform test
+```
+
+They cover the wiring and the input validation. Without them, a wrong argument
+name or an out-of-range value shows only at apply. They do not prove that the
+modules work against GCP. Only an apply does that.
+
+## Security
+
+- The bucket has uniform access and enforced public access prevention, so no
+  object can become public.
+- The database and cache have private IPs only. The database refuses
+  unencrypted connections. The cache always requires AUTH.
+- Nodes have no public IPs, run as a dedicated service account with the
+  minimum roles, and use shielded VMs. Pods see only their own identity through
+  the GKE metadata server.
+- The API server will not be built open. Set `master_authorized_networks`, or
+  `private_endpoint_enabled`, or `allow_unrestricted_api_server_access` to
+  record that an open control plane is what you meant.
+- The database will not be built without a password of at least 8 characters.
+- `deletion_protection` is on by default for the cluster, database, cache,
+  bucket and Cloud Armor policy. Set it to `false` and apply before a destroy.

deployment/terraform/modules/gcp/onyx/main.tf

@@ -0,0 +1,266 @@
+locals {
+  workspace     = terraform.workspace
+  merged_labels = merge(var.labels, { tenant = var.name, environment = local.workspace })
+
+  resource_prefix = "${var.name}-${local.workspace}"
+  cluster_name    = local.resource_prefix
+  postgres_name   = "${var.name}-postgres-${local.workspace}"
+  redis_name      = "${var.name}-redis-${local.workspace}"
+
+  # Bucket names are globally unique across every GCP customer, so the
+  # readable prefix is truncated and a short digest of the full identity is
+  # appended to keep collisions unlikely.
+  bucket_name_digest    = substr(sha1("${var.project_id}-${var.name}-${local.workspace}-${var.region}"), 0, 8)
+  bucket_name_sanitized = replace(lower("${local.resource_prefix}-files"), "/[^a-z0-9-]/", "-")
+  # 54 + "-" + 8 = 63, the most a bucket name without dots may have.
+  bucket_name = coalesce(
+    var.bucket_name,
+    "${substr(local.bucket_name_sanitized, 0, 54)}-${local.bucket_name_digest}",
+  )
+
+  project_apis = var.enable_project_apis ? toset([
+    "compute.googleapis.com",
+    "container.googleapis.com",
+    "sqladmin.googleapis.com",
+    "redis.googleapis.com",
+    "servicenetworking.googleapis.com",
+    "storage.googleapis.com",
+    "monitoring.googleapis.com",
+    "logging.googleapis.com",
+    "iam.googleapis.com",
+    # Project IAM grants for the node service account go through this API.
+    "cloudresourcemanager.googleapis.com",
+  ]) : toset([])
+
+  # T-shirt size defaults, chosen so each tier lands on the GCP machine closest
+  # to what the AWS and Azure compositions pick. The index pool is
+  # memory-optimised because on GCP it carries the document index itself:
+  # there is no managed OpenSearch to move that load off the cluster.
+  size_defaults = {
+    small = {
+      main_node_machine_type  = "n2-standard-8"
+      main_node_min_count     = 1
+      main_node_max_count     = 3
+      index_node_machine_type = "n2-highmem-4"
+      index_node_disk_size_gb = 256
+      postgres_tier           = "db-custom-2-8192"
+      postgres_disk_size_gb   = 64
+      redis_memory_size_gb    = 5
+    }
+    medium = {
+      main_node_machine_type  = "n2-standard-16"
+      main_node_min_count     = 1
+      main_node_max_count     = 5
+      index_node_machine_type = "n2-highmem-8"
+      index_node_disk_size_gb = 512
+      postgres_tier           = "db-custom-2-8192"
+      postgres_disk_size_gb   = 128
+      redis_memory_size_gb    = 10
+    }
+    large = {
+      main_node_machine_type  = "n2-standard-16"
+      main_node_min_count     = 2
+      main_node_max_count     = 8
+      index_node_machine_type = "n2-highmem-16"
+      index_node_disk_size_gb = 1024
+      postgres_tier           = "db-custom-4-16384"
+      postgres_disk_size_gb   = 256
+      redis_memory_size_gb    = 20
+    }
+  }
+  sizing = local.size_defaults[var.size]
+
+  # An explicitly set variable always wins over its tier default.
+  main_node_machine_type  = coalesce(var.main_node_machine_type, local.sizing.main_node_machine_type)
+  main_node_min_count     = coalesce(var.main_node_min_count, local.sizing.main_node_min_count)
+  main_node_max_count     = coalesce(var.main_node_max_count, local.sizing.main_node_max_count)
+  index_node_machine_type = coalesce(var.index_node_machine_type, local.sizing.index_node_machine_type)
+  index_node_disk_size_gb = coalesce(var.index_node_disk_size_gb, local.sizing.index_node_disk_size_gb)
+  postgres_tier           = coalesce(var.postgres_tier, local.sizing.postgres_tier)
+  postgres_disk_size_gb   = coalesce(var.postgres_disk_size_gb, local.sizing.postgres_disk_size_gb)
+  redis_memory_size_gb    = coalesce(var.redis_memory_size_gb, local.sizing.redis_memory_size_gb)
+
+  # The index label and taint are the gke module's defaults, which the chart
+  # values in the README select and tolerate. The gke module drops this pool
+  # when index_node_pool_enabled is false.
+  node_pools = {
+    main = {
+      machine_type = local.main_node_machine_type
+      min_count    = local.main_node_min_count
+      max_count    = local.main_node_max_count
+      disk_size_gb = var.main_node_disk_size_gb
+    }
+    index = {
+      machine_type = local.index_node_machine_type
+      min_count    = var.index_node_min_count
+      max_count    = var.index_node_max_count
+      disk_size_gb = local.index_node_disk_size_gb
+      labels       = { "onyx.app/workload" = "document-index" }
+      taints       = [{ key = "document-index", value = "true", effect = "NO_SCHEDULE" }]
+    }
+  }
+
+  # Read per module rather than passed as the bare variable, so the tests can
+  # see that every guarded resource follows the one switch.
+  deletion_protection = {
+    gke         = var.deletion_protection
+    postgres    = var.deletion_protection
+    redis       = var.deletion_protection
+    gcs         = var.deletion_protection
+    cloud_armor = var.deletion_protection
+  }
+}
+
+resource "google_project_service" "this" {
+  for_each = local.project_apis
+
+  project = var.project_id
+  service = each.value
+
+  # Other workloads in the project may use the same APIs, and turning one off
+  # on destroy breaks them.
+  disable_on_destroy = false
+}
+
+locals {
+  # Reading the project back off the API resources is what orders every
+  # module after them. A depends_on on the modules would do it too, but it
+  # defers the gke module's project lookup to apply, which makes the bucket
+  # grant's principal unknown at plan and fails its for_each.
+  project_id = coalesce(one(distinct([for s in google_project_service.this : s.project])), var.project_id)
+
+  network_id          = var.create_network ? module.vpc[0].network_id : var.network_id
+  subnet_id           = var.create_network ? module.vpc[0].subnet_id : var.subnet_id
+  pods_range_name     = var.create_network ? module.vpc[0].pods_range_name : var.pods_range_name
+  services_range_name = var.create_network ? module.vpc[0].services_range_name : var.services_range_name
+
+  # Cloud SQL and Memorystore take the id that arrives only once the Private
+  # Service Access peering exists; network_id would let them race it.
+  private_service_network_id = var.create_network ? module.vpc[0].private_service_access_network_id : var.network_id
+
+  bucket_object_admin_members = [module.gke.workload_identity_principal]
+}
+
+module "vpc" {
+  source = "../vpc"
+  count  = var.create_network ? 1 : 0
+
+  name       = local.resource_prefix
+  project_id = local.project_id
+  region     = var.region
+  labels     = local.merged_labels
+
+  subnet_cidr       = var.subnet_cidr
+  pods_cidr         = var.pods_cidr
+  services_cidr     = var.services_cidr
+  psa_prefix_length = var.psa_prefix_length
+  enable_flow_logs  = var.enable_flow_logs
+}
+
+module "gke" {
+  source = "../gke"
+
+  name                = local.cluster_name
+  project_id          = local.project_id
+  region              = var.region
+  kubernetes_version  = var.kubernetes_version
+  release_channel     = var.release_channel
+  deletion_protection = local.deletion_protection.gke
+  labels              = local.merged_labels
+
+  network_id          = local.network_id
+  subnet_id           = local.subnet_id
+  pods_range_name     = local.pods_range_name
+  services_range_name = local.services_range_name
+
+  private_endpoint_enabled             = var.private_endpoint_enabled
+  master_authorized_networks           = var.master_authorized_networks
+  allow_unrestricted_api_server_access = var.allow_unrestricted_api_server_access
+
+  node_pools               = local.node_pools
+  index_node_pool_enabled  = var.index_node_pool_enabled
+  enable_gpu_node_pool     = var.enable_gpu_node_pool
+  enable_sandbox_node_pool = var.enable_sandbox_node_pool
+
+  create_workload_namespace     = var.create_workload_namespace
+  workload_namespace            = var.workload_namespace
+  workload_service_account_name = var.workload_service_account_name
+}
+
+module "gcs" {
+  source = "../gcs"
+
+  name                = local.bucket_name
+  project_id          = local.project_id
+  location            = upper(var.region)
+  force_destroy       = var.bucket_force_destroy
+  deletion_protection = local.deletion_protection.gcs
+  labels              = local.merged_labels
+
+  # Built from inputs inside the gke module, so it is known at plan and safe
+  # as a for_each key there.
+  object_admin_members = local.bucket_object_admin_members
+}
+
+module "postgres" {
+  source = "../postgres"
+
+  name                = local.postgres_name
+  project_id          = local.project_id
+  region              = var.region
+  network_id          = local.private_service_network_id
+  deletion_protection = local.deletion_protection.postgres
+  labels              = local.merged_labels
+
+  database_version  = var.postgres_database_version
+  tier              = local.postgres_tier
+  disk_size_gb      = local.postgres_disk_size_gb
+  availability_type = var.postgres_availability_type
+  max_connections   = var.postgres_max_connections
+
+  db_name  = var.postgres_db_name
+  username = var.postgres_username
+  password = var.postgres_password
+
+  enable_alerts         = var.enable_alerts
+  notification_channels = var.notification_channels
+}
+
+module "redis" {
+  source = "../redis"
+  count  = var.enable_redis ? 1 : 0
+
+  name                = local.redis_name
+  project_id          = local.project_id
+  region              = var.region
+  network_id          = local.private_service_network_id
+  deletion_protection = local.deletion_protection.redis
+  labels              = local.merged_labels
+
+  tier                       = var.redis_tier
+  memory_size_gb             = local.redis_memory_size_gb
+  transit_encryption_enabled = var.redis_transit_encryption_enabled
+
+  enable_alerts         = var.enable_alerts
+  notification_channels = var.notification_channels
+}
+
+module "cloud_armor" {
+  source = "../cloud-armor"
+  count  = var.enable_cloud_armor ? 1 : 0
+
+  name                = local.resource_prefix
+  project_id          = local.project_id
+  deletion_protection = local.deletion_protection.cloud_armor
+  labels              = local.merged_labels
+
+  preview                     = var.cloud_armor_preview
+  sensitivity                 = var.cloud_armor_sensitivity
+  allowed_ip_cidrs            = var.cloud_armor_allowed_ip_cidrs
+  blocked_ip_cidrs            = var.cloud_armor_blocked_ip_cidrs
+  rate_limit_exempt_ip_cidrs  = var.cloud_armor_rate_limit_exempt_ip_cidrs
+  geo_restriction_countries   = var.cloud_armor_geo_restriction_countries
+  rate_limit_threshold        = var.cloud_armor_rate_limit_threshold
+  api_rate_limit_threshold    = var.cloud_armor_api_rate_limit_threshold
+  adaptive_protection_enabled = var.cloud_armor_adaptive_protection_enabled
+}

deployment/terraform/modules/gcp/onyx/outputs.tf

@@ -0,0 +1,114 @@
+output "cluster_name" {
+  description = "Name of the GKE cluster"
+  value       = module.gke.cluster_name
+}
+
+output "location" {
+  description = "Region of the cluster. gcloud container clusters get-credentials takes this as --location."
+  value       = module.gke.location
+}
+
+output "cluster_endpoint" {
+  description = "API server URL for the kubernetes and helm providers"
+  value       = module.gke.endpoint
+  sensitive   = true
+}
+
+# A CA certificate is public by design, so this is not marked sensitive.
+output "cluster_ca_certificate" {
+  description = "Cluster CA certificate, base64 encoded"
+  value       = module.gke.cluster_ca_certificate
+}
+
+output "workload_identity_principal" {
+  description = "IAM member for the workload service account, for granting it roles on resources outside this module"
+  value       = module.gke.workload_identity_principal
+}
+
+output "workload_namespace" {
+  description = "Namespace of the workload service account. Install the Helm release into it."
+  value       = module.gke.workload_namespace
+}
+
+output "workload_service_account_name" {
+  description = "Kubernetes service account that holds the bucket grant. Set it as the chart's serviceAccount.name."
+  value       = module.gke.workload_service_account_name
+}
+
+output "node_service_account_email" {
+  description = "Service account the nodes run as. Grant it roles/artifactregistry.reader on a registry in another project to pull from it."
+  value       = module.gke.node_service_account_email
+}
+
+# With a supplied network these are read off the ids given, so they name what
+# the cluster joined either way.
+output "network_name" {
+  description = "Name of the VPC network the cluster joins"
+  value       = var.create_network ? module.vpc[0].network_name : element(reverse(split("/", var.network_id)), 0)
+}
+
+output "subnet_name" {
+  description = "Name of the subnet the nodes join"
+  value       = var.create_network ? module.vpc[0].subnet_name : element(reverse(split("/", var.subnet_id)), 0)
+}
+
+# --- Values the Helm chart needs ---------------------------------------------
+
+output "gcs_bucket_name" {
+  description = "Set as GCS_FILE_STORE_BUCKET_NAME"
+  value       = module.gcs.bucket_name
+}
+
+output "gcs_project_id" {
+  description = "Set as GCS_PROJECT_ID"
+  value       = var.project_id
+}
+
+output "postgres_host" {
+  description = "Private IP of the database. Set as POSTGRES_HOST."
+  value       = module.postgres.host
+}
+
+output "postgres_port" {
+  description = "Database port. Set as POSTGRES_PORT."
+  value       = module.postgres.port
+}
+
+output "postgres_db_name" {
+  description = "Database name. Set as POSTGRES_DB; Onyx falls back to \"postgres\" without it."
+  value       = module.postgres.db_name
+}
+
+output "postgres_username" {
+  description = "Login Onyx connects as. Set as POSTGRES_USER."
+  value       = module.postgres.username
+  sensitive   = true
+}
+
+output "redis_host" {
+  description = "Private IP of the cache, null when enable_redis is false. Set as REDIS_HOST."
+  value       = one(module.redis[*].host)
+}
+
+output "redis_port" {
+  description = "Cache port: 6379, or 6378 with transit encryption. Set as REDIS_PORT."
+  value       = one(module.redis[*].port)
+}
+
+# Memorystore generates this rather than accepting one, so it comes out of the
+# module rather than going in.
+output "redis_auth_string" {
+  description = "Generated AUTH string. Set as REDIS_PASSWORD through a secret."
+  value       = one(module.redis[*].auth_string)
+  sensitive   = true
+}
+
+output "redis_url_scheme" {
+  description = "\"rediss\" with transit encryption, \"redis\" without. With rediss, set REDIS_SSL=true."
+  value       = one(module.redis[*].redis_url_scheme)
+}
+
+output "cloud_armor_policy_name" {
+  description = "Cloud Armor policy to name in a BackendConfig or GCPBackendPolicy, null when disabled"
+  value       = try(module.cloud_armor[0].policy_name, null)
+}

deployment/terraform/modules/gcp/onyx/tests/onyx.tftest.hcl

@@ -0,0 +1,385 @@
+# Plans the whole composition against mocked providers, so these run without a
+# GCP project or a cluster. Run with `terraform test` from the module directory.
+
+mock_provider "google" {
+  # The workload identity principal is written with the project number, and
+  # a mocked random string would make it impossible to check.
+  override_data {
+    target = module.gke.data.google_project.this
+    values = {
+      number = "123456789012"
+    }
+  }
+}
+
+mock_provider "kubernetes" {}
+
+variables {
+  name              = "onyx"
+  project_id        = "example-project"
+  region            = "us-east1"
+  postgres_password = "not-a-real-password"
+
+  # The composition refuses a public API server that no range restricts, so
+  # every case below has to say what it wants.
+  master_authorized_networks = [{ cidr_block = "203.0.113.0/24" }]
+}
+
+run "medium_is_the_default_tier" {
+  command = plan
+
+  assert {
+    condition     = local.main_node_machine_type == "n2-standard-16"
+    error_message = "Medium should land on the GCP machine closest to the AWS composition's m7i.4xlarge."
+  }
+
+  assert {
+    condition     = local.main_node_min_count == 1 && local.main_node_max_count == 5
+    error_message = "Medium runs 1-5 main nodes, like the Azure composition."
+  }
+
+  assert {
+    condition     = local.index_node_machine_type == "n2-highmem-8" && local.index_node_disk_size_gb == 512
+    error_message = "The index pool is memory-optimised because on GCP it carries the document index itself."
+  }
+
+  assert {
+    condition     = local.postgres_tier == "db-custom-2-8192" && local.postgres_disk_size_gb == 128
+    error_message = "Medium keeps the small database tier, matching the AWS composition."
+  }
+
+  assert {
+    condition     = local.redis_memory_size_gb == 10
+    error_message = "Medium gets a 10 GB cache, like the Azure composition's Balanced_B10."
+  }
+}
+
+run "small_moves_every_knob_together" {
+  command = plan
+
+  variables {
+    size = "small"
+  }
+
+  assert {
+    condition = alltrue([
+      local.main_node_machine_type == "n2-standard-8",
+      local.main_node_min_count == 1,
+      local.main_node_max_count == 3,
+      local.index_node_machine_type == "n2-highmem-4",
+      local.index_node_disk_size_gb == 256,
+      local.postgres_tier == "db-custom-2-8192",
+      local.postgres_disk_size_gb == 64,
+      local.redis_memory_size_gb == 5,
+    ])
+    error_message = "The small tier should move compute, database, cache and index sizing together."
+  }
+}
+
+run "large_moves_every_knob_together" {
+  command = plan
+
+  variables {
+    size = "large"
+  }
+
+  assert {
+    condition = alltrue([
+      local.main_node_machine_type == "n2-standard-16",
+      local.main_node_min_count == 2,
+      local.main_node_max_count == 8,
+      local.index_node_machine_type == "n2-highmem-16",
+      local.index_node_disk_size_gb == 1024,
+      local.postgres_tier == "db-custom-4-16384",
+      local.postgres_disk_size_gb == 256,
+      local.redis_memory_size_gb == 20,
+    ])
+    error_message = "The large tier should move compute, database, cache and index sizing together."
+  }
+}
+
+run "an_explicit_value_beats_its_tier_default" {
+  command = plan
+
+  variables {
+    size                  = "small"
+    postgres_disk_size_gb = 512
+    main_node_max_count   = 10
+    redis_memory_size_gb  = 8
+  }
+
+  assert {
+    condition     = local.postgres_disk_size_gb == 512 && local.main_node_max_count == 10 && local.redis_memory_size_gb == 8
+    error_message = "An explicitly set variable must win over the tier default."
+  }
+
+  assert {
+    condition     = local.main_node_machine_type == "n2-standard-8"
+    error_message = "Overriding one knob must not disturb the others in the tier."
+  }
+
+  assert {
+    condition     = local.node_pools.main.max_count == 10
+    error_message = "The override has to reach the node pool the gke module builds."
+  }
+}
+
+run "the_derived_bucket_name_is_one_gcs_accepts" {
+  command = plan
+
+  assert {
+    condition     = can(regex("^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$", local.bucket_name))
+    error_message = "Bucket names allow 3-63 lowercase letters, digits and hyphens."
+  }
+
+  assert {
+    condition     = startswith(local.bucket_name, "onyx-default-files-") && endswith(local.bucket_name, local.bucket_name_digest)
+    error_message = "The bucket name should read as its deployment and end in the digest."
+  }
+}
+
+run "a_long_name_keeps_the_whole_digest" {
+  command = plan
+
+  # The longest name the composition accepts. Only a long workspace name
+  # pushes the prefix past 54 characters, and tests cannot set one.
+  variables {
+    name = "onyx-enterprise-producti"
+  }
+
+  assert {
+    condition     = length(local.bucket_name) <= 63 && endswith(local.bucket_name, local.bucket_name_digest)
+    error_message = "Truncation must eat the readable prefix, never the digest."
+  }
+}
+
+run "an_explicit_bucket_name_is_used_as_given" {
+  command = plan
+
+  variables {
+    bucket_name = "onyx-files-prod"
+  }
+
+  assert {
+    condition     = output.gcs_bucket_name == "onyx-files-prod"
+    error_message = "An explicit bucket name must reach the bucket unchanged."
+  }
+}
+
+run "deletion_protection_is_on_everywhere_by_default" {
+  command = plan
+
+  assert {
+    condition     = alltrue(values(local.deletion_protection)) && length(local.deletion_protection) == 5
+    error_message = "The cluster, database, cache, bucket and policy must all be protected by default."
+  }
+}
+
+run "one_switch_turns_deletion_protection_off_everywhere" {
+  command = plan
+
+  variables {
+    deletion_protection = false
+  }
+
+  assert {
+    condition     = !anytrue(values(local.deletion_protection))
+    error_message = "deletion_protection = false must reach every module, or a teardown stops halfway."
+  }
+}
+
+run "data_services_wait_for_the_peering" {
+  command = plan
+
+  # Distinct ids for the two outputs make the choice visible. In the real vpc
+  # module they hold the same string and differ only in when they arrive.
+  override_module {
+    target = module.vpc
+    outputs = {
+      network_id                        = "projects/example-project/global/networks/plain"
+      private_service_access_network_id = "projects/example-project/global/networks/after-peering"
+      subnet_id                         = "projects/example-project/regions/us-east1/subnetworks/onyx-default-subnet"
+      pods_range_name                   = "onyx-default-pods"
+      services_range_name               = "onyx-default-services"
+      network_name                      = "onyx-default-vpc"
+      subnet_name                       = "onyx-default-subnet"
+    }
+  }
+
+  assert {
+    condition     = local.private_service_network_id == "projects/example-project/global/networks/after-peering"
+    error_message = "Cloud SQL and Memorystore must take the id that waits for Private Service Access, or they race the peering."
+  }
+
+  assert {
+    condition     = local.network_id == "projects/example-project/global/networks/plain"
+    error_message = "The cluster has no reason to wait for the peering."
+  }
+}
+
+run "the_bucket_grant_goes_to_the_workload_service_account" {
+  command = plan
+
+  assert {
+    condition     = local.bucket_object_admin_members == ["principal://iam.googleapis.com/projects/123456789012/locations/global/workloadIdentityPools/example-project.svc.id.goog/subject/ns/onyx/sa/onyx-workload-access"]
+    error_message = "The bucket should grant the Kubernetes service account the chart runs as, and nothing else."
+  }
+
+  # Known at plan is the point: the gcs module keys a for_each on it.
+  assert {
+    condition     = output.workload_identity_principal == local.bucket_object_admin_members[0]
+    error_message = "The published principal must be the one the bucket grants."
+  }
+}
+
+run "redis_and_cloud_armor_are_on_by_default" {
+  command = plan
+
+  assert {
+    condition     = length(module.redis) == 1 && length(module.cloud_armor) == 1
+    error_message = "Memorystore and Cloud Armor should both be created by default."
+  }
+
+  assert {
+    condition     = output.redis_url_scheme == "redis"
+    error_message = "The cache is plaintext by default, so the scheme is redis."
+  }
+}
+
+run "turning_off_redis_and_cloud_armor_drops_them" {
+  command = plan
+
+  variables {
+    enable_redis       = false
+    enable_cloud_armor = false
+  }
+
+  assert {
+    condition     = length(module.redis) == 0 && length(module.cloud_armor) == 0
+    error_message = "Disabled modules must create nothing."
+  }
+
+  assert {
+    condition = alltrue([
+      output.redis_host == null,
+      output.redis_port == null,
+      output.redis_url_scheme == null,
+      output.cloud_armor_policy_name == null,
+    ])
+    error_message = "With nothing created there is nothing to publish."
+  }
+}
+
+run "the_project_apis_are_enabled_by_default" {
+  command = plan
+
+  assert {
+    condition     = length(google_project_service.this) == 10
+    error_message = "Every API a module calls should be enabled."
+  }
+
+  assert {
+    condition     = alltrue([for s in google_project_service.this : !s.disable_on_destroy])
+    error_message = "Destroying Onyx must not turn off APIs other workloads in the project may use."
+  }
+}
+
+run "apis_managed_elsewhere_are_left_alone" {
+  command = plan
+
+  variables {
+    enable_project_apis = false
+  }
+
+  assert {
+    condition     = length(google_project_service.this) == 0
+    error_message = "enable_project_apis = false must enable nothing."
+  }
+
+  assert {
+    condition     = local.project_id == "example-project"
+    error_message = "With no API resources the modules still need the project."
+  }
+}
+
+run "an_existing_network_is_used_as_given" {
+  command = plan
+
+  variables {
+    create_network      = false
+    network_id          = "projects/example-project/global/networks/shared"
+    subnet_id           = "projects/example-project/regions/us-east1/subnetworks/gke"
+    pods_range_name     = "gke-pods"
+    services_range_name = "gke-services"
+  }
+
+  assert {
+    condition     = length(module.vpc) == 0
+    error_message = "A supplied network means no network is created."
+  }
+
+  assert {
+    condition     = local.private_service_network_id == "projects/example-project/global/networks/shared"
+    error_message = "The database and cache should join the supplied network."
+  }
+
+  assert {
+    condition     = output.network_name == "shared" && output.subnet_name == "gke"
+    error_message = "The names should be read off the supplied ids."
+  }
+}
+
+run "bringing_your_own_network_needs_every_id" {
+  command = plan
+
+  variables {
+    create_network = false
+    network_id     = "projects/example-project/global/networks/shared"
+  }
+
+  expect_failures = [var.create_network]
+}
+
+run "rejects_a_deployment_with_no_database_password" {
+  command = plan
+
+  variables {
+    postgres_password = null
+  }
+
+  expect_failures = [var.postgres_password]
+}
+
+run "rejects_an_open_control_plane" {
+  command = plan
+
+  variables {
+    master_authorized_networks = []
+  }
+
+  expect_failures = [var.allow_unrestricted_api_server_access]
+}
+
+run "an_open_control_plane_can_be_asked_for" {
+  command = plan
+
+  variables {
+    master_authorized_networks           = []
+    allow_unrestricted_api_server_access = true
+  }
+
+  assert {
+    condition     = output.cluster_name == "onyx-default"
+    error_message = "Recording that the exposure is intended should let the plan through."
+  }
+}
+
+run "rejects_a_size_that_is_not_a_tier" {
+  command = plan
+
+  variables {
+    size = "extra-large"
+  }
+
+  expect_failures = [var.size]
+}

deployment/terraform/modules/gcp/onyx/variables.tf

@@ -0,0 +1,449 @@
+variable "name" {
+  type        = string
+  description = "Name prefix for every resource. Example: \"onyx\"."
+  default     = "onyx"
+
+  # The Memorystore instance ID is the tightest limit: "<name>-redis-<workspace>"
+  # must fit in 40 characters, which leaves room for a workspace name of about 8.
+  validation {
+    condition     = can(regex("^[a-z]([-a-z0-9]{0,22}[a-z0-9])?$", var.name))
+    error_message = "name must be 1-24 characters of lowercase letters, digits and hyphens, start with a letter and not end with a hyphen."
+  }
+}
+
+variable "project_id" {
+  type        = string
+  description = "GCP project that holds every resource"
+
+  validation {
+    condition     = can(regex("^[a-z][-a-z0-9]{4,28}[a-z0-9]$", var.project_id))
+    error_message = "project_id must be a GCP project ID: 6-30 lowercase letters, digits and hyphens, starting with a letter."
+  }
+}
+
+variable "region" {
+  type        = string
+  description = "GCP region for all resources, for example \"us-east1\""
+
+  validation {
+    condition     = can(regex("^[a-z]+-[a-z]+[0-9]+$", var.region))
+    error_message = "region must be a GCP region such as \"us-east1\", not a zone."
+  }
+}
+
+variable "labels" {
+  type        = map(string)
+  description = "Base labels applied to every resource that supports them"
+  # Add an owner label here if your asset inventory expects one. GCP labels
+  # take only lowercase letters, digits, underscores and hyphens.
+  default = {
+    "project" = "onyx"
+  }
+}
+
+variable "size" {
+  type        = string
+  description = <<-EOT
+    T-shirt size that sets coherent defaults for every compute and data-plane knob:
+      small  - pilots and small teams: up to ~200 users, < ~500k documents
+      medium - typical department or company: ~200-1,000 users, ~0.5-2M documents
+      large  - org-wide deployments: 1,000+ users, multi-million documents
+    Any individual sizing variable set to a non-null value overrides its tier
+    default. See the README for the full per-tier table.
+  EOT
+  default     = "medium"
+
+  validation {
+    condition     = contains(["small", "medium", "large"], var.size)
+    error_message = "size must be one of: small, medium, large."
+  }
+}
+
+# One switch rather than one per module, so a teardown needs one apply with
+# this false and nothing is left guarded by a flag someone forgot.
+variable "deletion_protection" {
+  type        = bool
+  description = "Refuse to destroy the cluster, database, cache, bucket and Cloud Armor policy. Set false and apply before a destroy."
+  default     = true
+}
+
+# A fresh project has most of these off, and every module fails on its first
+# API call without them.
+variable "enable_project_apis" {
+  type        = bool
+  description = "Enable the Google APIs the deployment uses. Turn this off when the project is managed elsewhere and the APIs are already on."
+  default     = true
+}
+
+# --- Network -----------------------------------------------------------------
+
+variable "create_network" {
+  type        = bool
+  description = "Create the VPC network. False uses the network, subnet and secondary ranges supplied below."
+  default     = true
+
+  validation {
+    condition     = var.create_network || (var.network_id != null && var.subnet_id != null && var.pods_range_name != null && var.services_range_name != null)
+    error_message = "Bringing your own network needs network_id, subnet_id, pods_range_name and services_range_name."
+  }
+}
+
+variable "subnet_cidr" {
+  type        = string
+  description = "Primary range of the created subnet, used for node IPs"
+  default     = "10.0.0.0/20"
+}
+
+variable "pods_cidr" {
+  type        = string
+  description = "Secondary range of the created subnet for pod IPs"
+  default     = "10.4.0.0/14"
+}
+
+variable "services_cidr" {
+  type        = string
+  description = "Secondary range of the created subnet for service IPs"
+  default     = "10.8.0.0/20"
+}
+
+variable "psa_prefix_length" {
+  type        = number
+  description = "Prefix length of the range reserved for Private Service Access (Cloud SQL, Memorystore)"
+  default     = 16
+}
+
+# On by default, like the AWS modules and unlike Azure: GCP writes flow logs
+# to Cloud Logging with no extra infrastructure.
+variable "enable_flow_logs" {
+  type        = bool
+  description = "Write VPC flow logs for the subnet to Cloud Logging. Only applies to a network this module creates."
+  default     = true
+}
+
+variable "network_id" {
+  type        = string
+  description = "Existing network, as projects/<project>/global/networks/<name>. It must already have Private Service Access and a Cloud NAT. Required when create_network is false."
+  default     = null
+}
+
+variable "subnet_id" {
+  type        = string
+  description = "Existing subnet for the nodes. Required when create_network is false."
+  default     = null
+}
+
+variable "pods_range_name" {
+  type        = string
+  description = "Secondary range on the existing subnet for pods. Required when create_network is false."
+  default     = null
+}
+
+variable "services_range_name" {
+  type        = string
+  description = "Secondary range on the existing subnet for services. Required when create_network is false."
+  default     = null
+}
+
+# --- Storage -----------------------------------------------------------------
+
+variable "bucket_name" {
+  type        = string
+  description = "Name of the file store bucket. Null derives one from the name, workspace and a short digest, because bucket names are globally unique."
+  default     = null
+}
+
+variable "bucket_force_destroy" {
+  type        = bool
+  description = "Delete every object when the bucket is destroyed. Off, so a destroy fails on a bucket that still holds files."
+  default     = false
+}
+
+# --- Postgres ----------------------------------------------------------------
+
+variable "postgres_tier" {
+  type        = string
+  description = "Cloud SQL machine tier, for example \"db-custom-2-8192\". Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "postgres_disk_size_gb" {
+  type        = number
+  description = "Initial data disk in GB. It grows on its own and never shrinks. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "postgres_username" {
+  type        = string
+  description = "Login Onyx connects as. \"postgres\" is the user Cloud SQL ships and the chart's default."
+  default     = "postgres"
+  sensitive   = true
+}
+
+variable "postgres_password" {
+  type        = string
+  description = "Password for postgres_username. Supply it from a secret store."
+  default     = null
+  sensitive   = true
+
+  validation {
+    condition     = try(length(var.postgres_password) >= 8, false)
+    error_message = "postgres_password must be set and at least 8 characters."
+  }
+}
+
+# Not "postgres", the database Cloud SQL ships: Onyx gets a database of its
+# own. Onyx reads POSTGRES_DB, which defaults to "postgres", so the chart
+# values have to name this one.
+variable "postgres_db_name" {
+  type        = string
+  description = "Database Onyx connects to. Set POSTGRES_DB to this value in the chart."
+  default     = "onyx"
+}
+
+variable "postgres_database_version" {
+  type        = string
+  description = "Cloud SQL PostgreSQL version"
+  default     = "POSTGRES_16"
+}
+
+variable "postgres_availability_type" {
+  type        = string
+  description = "ZONAL runs one instance. REGIONAL adds a standby in a second zone and roughly doubles database cost."
+  default     = "ZONAL"
+}
+
+variable "postgres_max_connections" {
+  type        = number
+  description = "max_connections flag. Size it to pods x workers x (pool size + overflow). Changing it restarts the instance."
+  default     = 500
+}
+
+# --- Redis -------------------------------------------------------------------
+
+# On by default, unlike Azure. Memorystore for Redis is a single endpoint with
+# 16 databases, so Onyx runs on it unchanged.
+variable "enable_redis" {
+  type        = bool
+  description = "Create a Memorystore for Redis instance. False leaves Redis to the chart's in-cluster instance."
+  default     = true
+}
+
+variable "redis_tier" {
+  type        = string
+  description = "STANDARD_HA keeps a replica in a second zone. BASIC is one node, and a restart loses every queued Celery task."
+  default     = "STANDARD_HA"
+}
+
+variable "redis_memory_size_gb" {
+  type        = number
+  description = "Instance memory in GiB. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "redis_transit_encryption_enabled" {
+  type        = bool
+  description = "Serve TLS on 6378. Needs the server CA mounted into the pods (the chart's redisTls). Changing this replaces the instance."
+  default     = false
+}
+
+# --- Cluster -----------------------------------------------------------------
+
+variable "kubernetes_version" {
+  type        = string
+  description = "Minimum control plane version, for example \"1.33\". Null takes the release channel default."
+  default     = null
+}
+
+variable "release_channel" {
+  type        = string
+  description = "GKE release channel"
+  default     = "REGULAR"
+}
+
+variable "main_node_machine_type" {
+  type        = string
+  description = "Machine type for the main pool. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "main_node_min_count" {
+  type        = number
+  description = "Minimum nodes in the main pool, across all zones. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "main_node_max_count" {
+  type        = number
+  description = "Maximum nodes in the main pool, across all zones. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "main_node_disk_size_gb" {
+  type        = number
+  description = "Boot disk for main pool nodes"
+  default     = 100
+}
+
+# GCP has no managed OpenSearch, so this pool is where the document index
+# runs. It is on by default, unlike the AWS module's equivalent.
+variable "index_node_pool_enabled" {
+  type        = bool
+  description = "Create the document-index node pool"
+  default     = true
+}
+
+variable "index_node_machine_type" {
+  type        = string
+  description = "Machine type for the document-index pool. Memory-optimised, because the index is what needs it. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "index_node_min_count" {
+  type        = number
+  description = "Minimum nodes in the document-index pool, across all zones"
+  default     = 1
+}
+
+variable "index_node_max_count" {
+  type        = number
+  description = "Maximum nodes in the document-index pool, across all zones"
+  default     = 3
+}
+
+variable "index_node_disk_size_gb" {
+  type        = number
+  description = "Boot disk for document-index nodes. Null uses the t-shirt size default."
+  default     = null
+}
+
+variable "private_endpoint_enabled" {
+  type        = bool
+  description = "Serve the API server on its private address only"
+  default     = false
+}
+
+variable "master_authorized_networks" {
+  type = list(object({
+    cidr_block   = string
+    display_name = optional(string)
+  }))
+  description = "CIDR ranges allowed to reach the API server"
+  default     = []
+}
+
+variable "allow_unrestricted_api_server_access" {
+  type        = bool
+  description = "Accept a public API server reachable from any address. Only for throwaway deployments; set master_authorized_networks or private_endpoint_enabled instead."
+  default     = false
+
+  validation {
+    condition     = var.allow_unrestricted_api_server_access || var.private_endpoint_enabled || length(var.master_authorized_networks) > 0
+    error_message = "A public API server with no authorized networks is reachable from every address on the internet. Set master_authorized_networks, or private_endpoint_enabled, or allow_unrestricted_api_server_access to record that the exposure is intended."
+  }
+}
+
+variable "enable_gpu_node_pool" {
+  type        = bool
+  description = "Create a tainted GPU pool for the embedding model server"
+  default     = false
+}
+
+variable "enable_sandbox_node_pool" {
+  type        = bool
+  description = "Create a tainted pool for Craft sandbox workloads"
+  default     = false
+}
+
+variable "create_workload_namespace" {
+  type        = bool
+  description = "Create the namespace the workload service account lives in. Terraform has to make it: the Helm release that would otherwise is installed afterwards."
+  default     = true
+}
+
+variable "workload_namespace" {
+  type        = string
+  description = "Namespace of the workload service account. Install the Helm release into it."
+  default     = "onyx"
+}
+
+variable "workload_service_account_name" {
+  type        = string
+  description = "Kubernetes service account that gets the bucket grant. Point the chart's serviceAccount.name at it."
+  default     = "onyx-workload-access"
+}
+
+# --- Cloud Armor -------------------------------------------------------------
+
+variable "enable_cloud_armor" {
+  type        = bool
+  description = "Create a Cloud Armor policy to attach to an L7 load balancer through a BackendConfig or GCPBackendPolicy"
+  default     = true
+}
+
+variable "cloud_armor_preview" {
+  type        = bool
+  description = "Log what the WAF and rate limit rules match instead of acting on it"
+  default     = false
+}
+
+variable "cloud_armor_sensitivity" {
+  type        = number
+  description = "Sensitivity of the preconfigured WAF rules, 1 (fewest false positives) to 4"
+  default     = 1
+}
+
+variable "cloud_armor_allowed_ip_cidrs" {
+  type        = list(string)
+  description = "Ranges allowed to reach the application. Empty disables the allowlist."
+  default     = []
+}
+
+variable "cloud_armor_blocked_ip_cidrs" {
+  type        = list(string)
+  description = "Ranges refused with 403 before any other rule"
+  default     = []
+}
+
+variable "cloud_armor_rate_limit_exempt_ip_cidrs" {
+  type        = list(string)
+  description = "Ranges exempt from the rate limits"
+  default     = []
+}
+
+variable "cloud_armor_geo_restriction_countries" {
+  type        = list(string)
+  description = "Two-letter country codes to block"
+  default     = []
+}
+
+variable "cloud_armor_rate_limit_threshold" {
+  type        = number
+  description = "Requests per 5 minutes from one address before it gets 429s"
+  default     = 2000
+}
+
+variable "cloud_armor_api_rate_limit_threshold" {
+  type        = number
+  description = "Requests per 5 minutes from one address to the API path before it gets 429s"
+  default     = 1000
+}
+
+variable "cloud_armor_adaptive_protection_enabled" {
+  type        = bool
+  description = "Enable Adaptive Protection layer 7 DDoS detection"
+  default     = true
+}
+
+# --- Alerts ------------------------------------------------------------------
+
+variable "enable_alerts" {
+  type        = bool
+  description = "Create the Cloud Monitoring alert policies for the database and cache"
+  default     = true
+}
+
+variable "notification_channels" {
+  type        = list(string)
+  description = "Cloud Monitoring notification channel IDs for the database and cache alerts. Empty = alerts exist but notify nothing."
+  default     = []
+}

deployment/terraform/modules/gcp/onyx/versions.tf

@@ -0,0 +1,16 @@
+terraform {
+  required_version = ">= 1.12.0"
+
+  required_providers {
+    # The highest floor among the child modules: gcs and cloud-armor need 7.33
+    # for deletion_policy.
+    google = {
+      source  = "hashicorp/google"
+      version = "~> 7.33"
+    }
+    kubernetes = {
+      source  = "hashicorp/kubernetes"
+      version = "~> 2.37"
+    }
+  }
+}