> ## Documentation Index
> Fetch the complete documentation index at: https://docs.asymptotelabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Kubernetes Deployment

> Install self-hosted Asymptote in your Kubernetes cluster with the Helm chart.

> [!NOTE]
>
> **Planned release.** Self-hosted Asymptote is in development and offered to design partners through a Private Deployment engagement; the Helm chart and Terraform modules described in this section are not generally available yet. This section describes the planned release: chart names, values keys and paths may change before it ships. Today's Asymptote Cloud dashboard uses hosted services for sign-in and product analytics; the self-hosted release replaces them, as described in [System Architecture](https://docs.asymptotelabs.ai/self-hosting/system-architecture.md#hosted-services-in-asymptote-cloud). [Contact us](https://asymptotelabs.ai/contact) to take part.

This guide installs Asymptote from its Helm chart into a cluster you already have. To create the cluster and the services it needs first, start with [Terraform](https://docs.asymptotelabs.ai/self-hosting/terraform.md).

## How it fits together

The chart installs every Asymptote service (`web`, `api`, `ingest`, `policy`, `loader`, `workers`), the migration job, and, by default, ClickHouse through the ClickHouse operator. It does not install PostgreSQL, object storage, your ingress controller or certificates: those come from your cloud. See [System Architecture](https://docs.asymptotelabs.ai/self-hosting/system-architecture.md) for what each service does.

## Prerequisites

**Cluster**

* A Kubernetes version listed in the release notes
* An ingress controller, or the Gateway API
* A storage class for ClickHouse's persistent volumes
* `kubectl` and Helm 3.8 or later on your machine

**From Asymptote**

* Registry credentials
* A license key
* The chart version to install

**From your cloud**

* PostgreSQL 15 or later, with a database and user for Asymptote
* A dedicated bucket with new-object notifications (SQS, Pub/Sub or Event Grid)
* An identity provider application for Asymptote (see [SSO](https://docs.asymptotelabs.ai/self-hosting/sso.md))
* An LLM endpoint
* Host names and TLS certificates for the dashboard, ingest and policy endpoints

> [!WARNING]
>
> Give Asymptote its own namespace, and its own bucket, database and LLM credentials. Sharing them with other applications makes revocation and retention harder to reason about.

## Setup

**1. Create the namespace and registry secret**

```bash theme={null}
kubectl create namespace asymptote
kubectl -n asymptote create secret docker-registry asymptote-registry \
  --docker-server=<registry> --docker-username=<user> --docker-password=<token>
```

**2. Create the secrets**

```bash theme={null}
kubectl -n asymptote create secret generic asymptote-secrets \
  --from-literal=license-key=<license key> \
  --from-literal=postgres-url='postgres://asymptote:<password>@<host>:5432/asymptote?sslmode=require' \
  --from-literal=device-key-pepper="$(openssl rand -base64 32)" \
  --from-literal=enforcement-token-pepper="$(openssl rand -base64 32)" \
  --from-literal=oidc-client-secret=<client secret>
```

| Key | Required | What it is |
| - | - | - |
| `license-key` | Yes | From Asymptote |
| `postgres-url` | Yes | Connection string for Asymptote's database |
| `device-key-pepper` | Yes | Secret used to hash device keys. Changing it invalidates every enrolled device |
| `enforcement-token-pepper` | Yes | Secret used to hash enforcement tokens. Changing it invalidates every token |
| `oidc-client-secret` | With OIDC | Your identity provider application's secret |
| `llm-api-key` | With an API-key provider | For Azure OpenAI or an OpenAI-compatible API. Bedrock and Vertex AI use workload identity instead |
| `slack-bot-token` | With Slack notifications | The Slack app's bot token |

Back up both peppers with your other secrets: a deployment restored without them cannot verify any device.

Or sync them from your cloud's secret manager with the External Secrets Operator; see [Secret modes](#secret-modes).

**3. Write a values file**

```yaml title="values.yaml" theme={null}
global:
  domain: asymptote.example.com
  imagePullSecrets:
    - name: asymptote-registry
  existingSecret: asymptote-secrets

objectStorage:
  provider: s3            # s3, gcs or azure
  bucket: example-asymptote-telemetry
  notifications:
    queueUrl: https://sqs.us-east-1.amazonaws.com/123456789012/asymptote-runtime

auth:
  oidc:
    issuer: https://example.okta.com
    clientId: <client id>

llm:
  provider: bedrock       # bedrock, vertex, azure-openai or openai-compatible
  region: us-east-1

clickhouse:
  mode: bundled           # bundled or external
```

See [Configuration](#configuration) for every top-level key.

**4. Validate the chart**

```bash theme={null}
helm template asymptote oci://<registry>/asymptote/charts/asymptote \
  --version <version> --namespace asymptote --values values.yaml > /dev/null
```

**5. Install**

```bash theme={null}
helm install asymptote oci://<registry>/asymptote/charts/asymptote \
  --version <version> --namespace asymptote --values values.yaml
```

The migration job runs first; the services start once it succeeds.

**6. Verify the rollout**

```bash theme={null}
kubectl -n asymptote get pods
kubectl -n asymptote rollout status deployment --timeout=10m
curl -fsS https://ingest.asymptote.example.com/health
```

Every pod is `Running` and the health check answers `200`. (`/v1/ingest/health` checks a device's key, so it answers `401` without one.) Sign in to `https://asymptote.example.com` through your identity provider.

**7. Connect a test endpoint**

Enroll one machine (see [Connecting Endpoints](https://docs.asymptotelabs.ai/self-hosting/connect-endpoints.md)), run an agent session, and check that it appears on the dashboard within a minute.

## Configuration

| Key | Purpose |
| - | - |
| `global` | Domain and host names, image registry and pull secrets, the secret holding credentials |
| `objectStorage` | Provider, bucket, prefix and the new-object notification source |
| `auth` | SAML or OIDC settings, and the group-to-role mapping |
| `llm` | Provider, region or base URL, and model names for policy checks and reviews |
| `clickhouse` | Bundled or external, replicas, resources, volume size and retention |
| `ingest`, `policy`, `api`, `web`, `loader`, `workers` | Replicas, resources and autoscaling for each service |
| `ingress` | Ingress class or Gateway, TLS secrets, and whether `ingest` and `policy` are internal only |
| `notifications` | Slack settings for finding notifications |
| `networkPolicy` | Turn the chart's network policies on or off |
| `metrics` | Prometheus scraping and ServiceMonitor objects |

### Bundled or external ClickHouse

`clickhouse.mode: bundled` installs the ClickHouse operator and a ClickHouse cluster on persistent volumes in the namespace. Set `clickhouse.mode: external` and give its host and credentials to use a ClickHouse you already run; the migration job creates Asymptote's tables there.

### Secret modes

| Mode | How |
| - | - |
| Kubernetes secret | Create `asymptote-secrets` yourself, as in step 2, and set `global.existingSecret` |
| External Secrets Operator | Set `externalSecrets.enabled: true` and name the store; the chart creates `ExternalSecret` objects that read from Secrets Manager, Secret Manager or Key Vault |

## Operations

### Scaling

Every stateless service scales horizontally; set `replicas` or turn on `autoscaling` for it. Scale `ingest` and `policy` with fleet size, `loader` with event volume, and ClickHouse with event volume and retention.

### Updating

Read the release notes for the new version, then:

```bash theme={null}
helm upgrade asymptote oci://<registry>/asymptote/charts/asymptote \
  --version <new version> --namespace asymptote --values values.yaml
```

The migration job runs before the new services start. Asymptote supports the current release and the two before it.

### Backups

Back up PostgreSQL with your cloud's automated backups. Your bucket holds every raw object, so ClickHouse can be rebuilt from it; back up ClickHouse too if a rebuild would take longer than you can wait.

## Troubleshooting

| Symptom | Likely cause | Fix |
| - | - | - |
| `ImagePullBackOff` | Registry secret missing or wrong | Recreate `asymptote-registry` and check `global.imagePullSecrets` |
| Migration job fails | PostgreSQL unreachable, or the user lacks rights | Check `postgres-url`, network access from the cluster, and that the user owns the database |
| Sign-in loops back to the identity provider | Redirect URL mismatch | Make the application's redirect URL `https://<domain>/auth/callback` |
| Endpoints connect, nothing on the dashboard | `loader` not receiving notifications | Check the notification source in `objectStorage.notifications` and the loader's logs |
| Policy checks time out on endpoints | LLM endpoint slow or throttled | Check the `policy` latency metric and your provider's quota |

## Resources

* [System Architecture](https://docs.asymptotelabs.ai/self-hosting/system-architecture.md)
* [Terraform](https://docs.asymptotelabs.ai/self-hosting/terraform.md)
* [SSO](https://docs.asymptotelabs.ai/self-hosting/sso.md)
* [Connecting Endpoints](https://docs.asymptotelabs.ai/self-hosting/connect-endpoints.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.