> ## Documentation Index
> Fetch the complete documentation index at: https://docs.asymptotelabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Asymptote with Your Own S3 Bucket

> Keep Beacon telemetry in an S3 bucket you own and give Asymptote read-only access to it.

Use this guide when your Macs' Beacon telemetry should land in your own AWS account first, with Asymptote reading a copy for your dashboard. It covers the whole setup in order: the bucket and access, the details to send us, the rollout to your Macs, and how to confirm it works.

## What you'll set up

```mermaid theme={null}
flowchart TB
  subgraph Macs["Your Macs"]
    Beacon["AI agents and Beacon"]
  end
  subgraph Account["Your AWS account"]
    Bucket[("S3 bucket")]
    Queues["SQS queues"]
  end
  subgraph Cloud["Asymptote"]
    Dashboard["Dashboard"]
  end
  Beacon -->|"uploads with write-only key"| Bucket
  Bucket -->|"new-object notice"| Queues
  Queues -->|"which objects are new"| Dashboard
  Bucket -->|"reads with read-only key"| Dashboard
```

Beacon on each Mac records agent activity and uploads it to your bucket with a key that can only write. For each new object, S3 posts a notice to a queue. Asymptote reads the queues to learn which objects are new, fetches them with a read-only key, and loads them into your dashboard. The bucket stays yours and keeps every object.

In your AWS account:

* **A new S3 bucket just for Beacon**, with public access blocked, ACLs disabled, HTTPS required, default encryption, and versioning.
* **`beacon-endpoint-writer`**, an IAM user your Macs upload with. It can add objects below your Beacon prefix and nothing else.
* **`asymptote-s3-reader`**, an IAM user Asymptote reads with. It can list the bucket, read objects below the prefix, and consume the queues.
* **Two SQS queues**, one for agent activity (`runtime`) and one for inventory, each with a dead-letter queue. They let Asymptote load objects in any order, so an upload that arrives late is never skipped.

Objects land here:

```text theme={null}
s3://<bucket>/<prefix>/runtime/date=YYYY-MM-DD/<time>-<id>.jsonl.gz
s3://<bucket>/<prefix>/inventory/date=YYYY-MM-DD/<time>-<id>.jsonl.gz
```

<Note>
  Asymptote receives the telemetry your Macs upload in full, after Beacon's local redaction of secrets: prompts, the commands agents ran and their output, tool inputs and outputs, and file paths and diffs.
</Note>

## Before you start

| You need | Why |
| - | - |
| An AWS account where you can create S3 buckets, IAM users, and SQS queues | Step 1 |
| Terraform 1.5 or later with the AWS provider 5.0 or later, or the AWS CLI | Step 1 |
| An Asymptote organization | Asymptote connects your bucket to it in step 3 |
| An Apple Silicon Mac to try it on, and Fleet, Jamf Pro, or Rippling to roll it out | Step 4 |
| A bucket name nobody else uses | S3 bucket names are global across all AWS accounts |

## 1. Create the bucket and access

Pick a root prefix such as `beacon-prod`. Beacon adds `runtime/` and `inventory/` below it, so don't put either in the prefix. Create everything in the region you want the bucket in.

<Tabs>
  <Tab title="Terraform">
    Create a `main.tf` in a new directory. Change the region, bucket name, and prefix:

    ```hcl title="main.tf" theme={null}
    provider "aws" {
      region = "us-west-2"
    }

    # The bucket and the write-only user your Macs upload with.
    module "beacon_s3" {
      source = "github.com/asymptote-labs/agent-beacon//terraform/aws-s3?ref=v1.3.38"

      bucket_name = "acme-beacon-telemetry"
      prefix      = "beacon-prod"
    }

    # Read-only access for Asymptote, with the new-object queues.
    module "asymptote_reader" {
      source = "github.com/asymptote-labs/agent-beacon//terraform/aws-s3-asymptote-reader?ref=v1.3.38"

      bucket = module.beacon_s3.bucket
    }

    output "bucket_name" {
      value = module.beacon_s3.bucket_name
    }

    output "region" {
      value = module.beacon_s3.region
    }

    output "prefix" {
      value = module.beacon_s3.prefix
    }

    output "asymptote_source_url" {
      value = module.asymptote_reader.source_url
    }

    output "asymptote_queue_urls" {
      value = module.asymptote_reader.queue_urls
    }
    ```

    Apply it:

    ```bash theme={null}
    terraform init
    terraform apply
    ```

    Review the plan and type `yes`. It's working when the apply ends with `Apply complete! Resources: 18 added` and prints the outputs. Keep them for step 3. Encryption with your own KMS key, object expiry, and other options are in the base module's [`variables.tf`](https://github.com/asymptote-labs/agent-beacon/blob/main/terraform/aws-s3/variables.tf).
  </Tab>

  <Tab title="AWS CLI">
    **Set your values:**

    ```bash theme={null}
    export AWS_REGION="us-west-2"
    export BEACON_S3_BUCKET="<globally-unique-bucket>"
    export BEACON_S3_PREFIX="beacon-prod"
    export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text)"
    ```

    **Create the bucket** and block public access. AWS needs a different command for `us-east-1`:

    ```bash theme={null}
    if [ "$AWS_REGION" = "us-east-1" ]; then
      aws s3api create-bucket --bucket "$BEACON_S3_BUCKET" --region "$AWS_REGION"
    else
      aws s3api create-bucket --bucket "$BEACON_S3_BUCKET" --region "$AWS_REGION" \
        --create-bucket-configuration "LocationConstraint=$AWS_REGION"
    fi
    aws s3api put-public-access-block --bucket "$BEACON_S3_BUCKET" \
      --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
    ```

    **Create the writer** your Macs upload with:

    ```bash theme={null}
    aws iam create-user --user-name beacon-endpoint-writer
    aws iam put-user-policy \
      --user-name beacon-endpoint-writer \
      --policy-name BeaconEndpointWriteOnly \
      --policy-document "$(cat <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {"Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::${BEACON_S3_BUCKET}/${BEACON_S3_PREFIX}/*"},
        {"Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::${BEACON_S3_BUCKET}"}
      ]
    }
    EOF
    )"
    ```

    **Create the queues.** Each stream gets a queue that only this bucket may send to, and a dead-letter queue that keeps a message after ten failed deliveries:

    ```bash theme={null}
    for stream in runtime inventory; do
      dlq_url="$(aws sqs create-queue --region "$AWS_REGION" \
        --queue-name "asymptote-s3-reader-${stream}-dlq" \
        --attributes MessageRetentionPeriod=1209600,SqsManagedSseEnabled=true \
        --query QueueUrl --output text)"
      dlq_arn="$(aws sqs get-queue-attributes --region "$AWS_REGION" --queue-url "$dlq_url" \
        --attribute-names QueueArn --query Attributes.QueueArn --output text)"
      queue_arn="arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:asymptote-s3-reader-${stream}"
      cat > "asymptote-s3-reader-${stream}.json" <<EOF
    {
      "MessageRetentionPeriod": "1209600",
      "VisibilityTimeout": "300",
      "SqsManagedSseEnabled": "true",
      "RedrivePolicy": "{\"deadLetterTargetArn\":\"${dlq_arn}\",\"maxReceiveCount\":10}",
      "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"s3.amazonaws.com\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"${queue_arn}\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"arn:aws:s3:::${BEACON_S3_BUCKET}\"},\"StringEquals\":{\"aws:SourceAccount\":\"${AWS_ACCOUNT_ID}\"}}}]}"
    }
    EOF
      aws sqs create-queue --region "$AWS_REGION" \
        --queue-name "asymptote-s3-reader-${stream}" \
        --attributes "file://asymptote-s3-reader-${stream}.json"
    done
    ```

    The loop prints the `QueueUrl` of the `runtime` and `inventory` queues. Keep both for step 3.

    **Send new-object notifications to the queues:**

    ```bash theme={null}
    cat > beacon-notifications.json <<EOF
    {
      "QueueConfigurations": [
        {
          "Id": "beacon-runtime",
          "QueueArn": "arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:asymptote-s3-reader-runtime",
          "Events": ["s3:ObjectCreated:*"],
          "Filter": {"Key": {"FilterRules": [{"Name": "prefix", "Value": "${BEACON_S3_PREFIX}/runtime/"}]}}
        },
        {
          "Id": "beacon-inventory",
          "QueueArn": "arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:asymptote-s3-reader-inventory",
          "Events": ["s3:ObjectCreated:*"],
          "Filter": {"Key": {"FilterRules": [{"Name": "prefix", "Value": "${BEACON_S3_PREFIX}/inventory/"}]}}
        }
      ]
    }
    EOF
    aws s3api put-bucket-notification-configuration \
      --bucket "$BEACON_S3_BUCKET" \
      --notification-configuration file://beacon-notifications.json
    ```

    **Create the reader** Asymptote reads with:

    ```bash theme={null}
    aws iam create-user --user-name asymptote-s3-reader
    aws iam put-user-policy \
      --user-name asymptote-s3-reader \
      --policy-name BeaconTelemetryReadOnly \
      --policy-document "$(cat <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {"Effect": "Allow", "Action": ["s3:ListBucket", "s3:GetBucketLocation"], "Resource": "arn:aws:s3:::${BEACON_S3_BUCKET}"},
        {"Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::${BEACON_S3_BUCKET}/${BEACON_S3_PREFIX}/*"},
        {"Effect": "Allow", "Action": ["sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes"],
         "Resource": ["arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:asymptote-s3-reader-runtime",
                      "arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:asymptote-s3-reader-inventory"]},
        {"Effect": "Allow", "Action": "sqs:ListQueues", "Resource": "*"}
      ]
    }
    EOF
    )"
    ```

    If the bucket's default encryption uses a customer managed KMS key, also allow the writer `kms:GenerateDataKey` and the reader `kms:Decrypt` on that key.
  </Tab>
</Tabs>

## 2. Create the access keys

Create both keys with the AWS CLI, so their secrets never enter Terraform state:

```bash theme={null}
aws iam create-access-key --user-name beacon-endpoint-writer --output json
aws iam create-access-key --user-name asymptote-s3-reader --output json
```

AWS shows each secret only once. The keys start with `AKIA` and don't expire.

* **Writer key:** store it in your MDM's secret store. Your Macs use it in step 4.
* **Reader key:** you send it to Asymptote in step 3. Don't keep it anywhere else.

## 3. Send Asymptote the details

Send these through the secure channel your Asymptote contact gives you. Don't send the secret access key by email or chat.

| Detail | Where it comes from |
| - | - |
| Bucket name and region | `bucket_name` and `region`, or your values |
| Source URL | `asymptote_source_url`, or `https://<bucket>.s3.<region>.amazonaws.com/<prefix>` |
| The two queue URLs | `asymptote_queue_urls`, or the `QueueUrl`s from the queue step |
| The reader's access key ID and secret | Step 2 |
| KMS key ARN | Only if the bucket uses your own KMS key |

Asymptote connects the bucket to your organization and tells you when it's done.

## 4. Roll out Beacon to your Macs

Try it on one Mac first, or go straight to your MDM.

<Tabs>
  <Tab title="One Mac (no MDM)">
    **Install Beacon.** On an Apple Silicon Mac, in Terminal while logged in at the Mac:

    ```bash title="Download, verify, and install" theme={null}
    VERSION="$(curl -fsSLI -o /dev/null -w '%{url_effective}' https://github.com/asymptote-labs/agent-beacon/releases/latest)"; VERSION="${VERSION##*/v}"
    PKG="BeaconEndpointAgent-${VERSION}-arm64.pkg"
    BASE="https://github.com/asymptote-labs/agent-beacon/releases/download/v${VERSION}"
    curl -fsSLO "${BASE}/${PKG}" && curl -fsSLO "${BASE}/${PKG}.sha256"
    test "$(awk '{print $1}' "${PKG}.sha256")" = "$(shasum -a 256 "${PKG}" | awk '{print $1}')" \
      && pkgutil --check-signature "${PKG}" \
      && sudo installer -pkg "${PKG}" -target /
    ```

    The block checks the package against its published SHA-256 and its Apple signature before installing. `pkgutil` should print `Status: signed by a developer certificate issued by Apple for distribution`. The package sets up Claude Code and Codex for the logged-in user; [Try Beacon on One Mac](/mdm/try-on-one-mac) covers what it installs and how to add other agents.

    **Start the forwarder** with your values from step 1 and the writer key from step 2:

    ```bash theme={null}
    export BEACON_S3_BUCKET="<bucket>"
    export AWS_REGION="<region>"
    export BEACON_S3_PREFIX="<prefix>"
    export AWS_ACCESS_KEY_ID="<writer AccessKeyId>"
    export AWS_SECRET_ACCESS_KEY="<writer SecretAccessKey>"

    sudo env \
      BEACON_S3_BUCKET="$BEACON_S3_BUCKET" \
      AWS_REGION="$AWS_REGION" \
      BEACON_S3_PREFIX="$BEACON_S3_PREFIX" \
      BEACON_S3_STORAGE_CLASS="STANDARD" \
      AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" \
      AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" \
      /opt/beacon/jamf/claude/s3/install-forwarder.sh
    ```

    It's working when the output shows the forwarder's `state = running`. The helper keeps the key in a root-only file for the forwarder service, so you can close Terminal. When you roll out with an MDM later, this Mac gets the same setup again.
  </Tab>

  <Tab title="Fleet, Jamf, or Rippling">
    Give your MDM these values, with the writer key from step 2:

    | Value | Fleet admin helper | Jamf script parameter | `install-forwarder.sh` |
    | - | - | - | - |
    | Bucket name | `BEACON_S3_BUCKET` | 4 | `BEACON_S3_BUCKET` |
    | Region | `AWS_REGION` | 5 | `AWS_REGION` |
    | Prefix | `BEACON_S3_PREFIX` | 6 | `BEACON_S3_PREFIX` |
    | Storage class (`STANDARD`) | `BEACON_S3_STORAGE_CLASS` | 7 | `BEACON_S3_STORAGE_CLASS` |
    | Writer key | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | Your Jamf secret injection | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` |

    Then follow your MDM's guide from the step after its AWS setup, which you've already done:

    | Rollout | Continue at |
    | - | - |
    | Fleet | [Beacon with Fleet and S3, step 3](/guides/fleet-s3-mdm#3-set-up-fleet) |
    | Jamf Pro | [Claude with Jamf and S3, Jamf policy setup](/guides/jamf-s3-mdm#jamf-policy-setup) |
    | Rippling | [Beacon with Rippling and S3, step 3](/guides/rippling-s3-mdm#3-add-the-background-items-profile) |
  </Tab>
</Tabs>

## 5. Confirm it works

On a Mac with Beacon installed, write a test event:

```bash theme={null}
sudo /opt/beacon/bin/beacon endpoint s3 validate --system
```

The forwarder uploads in batches, so allow up to five minutes. Then list the objects with your own AWS identity, because the writer can't read the bucket:

```bash theme={null}
aws s3 ls "s3://<bucket>/<prefix>/runtime/" --recursive --region <region>
```

Asymptote loads each new object within a few minutes of it landing. Your Asymptote contact confirms when the first events arrive.

## Operate

### Rotate the keys

IAM allows two keys per user, so the old key keeps working while you switch.

* **Writer:** create a second key, give it to your MDM and rerun its forwarder setup on every Mac, then delete the old key.
* **Reader:** create a second key, send it to Asymptote, and delete the old key once Asymptote confirms the switch.

### Remove

1. Ask Asymptote to disconnect the bucket.
2. Remove the forwarder from your Macs. Uninstalling Beacon removes it.
3. Delete both access keys: `aws iam delete-access-key --user-name <user> --access-key-id <key id>`. IAM refuses to delete a user that still has keys.
4. Run `terraform destroy`. It stops at a bucket that still holds objects. To delete the telemetry too, add `force_destroy = true` to the `beacon_s3` module, run `terraform apply`, and then destroy.

## Troubleshooting

<AccordionGroup>
  <Accordion title="BucketAlreadyExists">
    Someone else, in any AWS account, already has a bucket with that name. Choose another bucket name.
  </Accordion>

  <Accordion title="S3 can only notify queues in the bucket's own region">
    The provider that runs the reader module is in a different region from the bucket. Use the same provider, in the bucket's region, for both modules.
  </Accordion>

  <Accordion title="Objects arrive in the bucket but not in Asymptote">
    Check the queues: `aws sqs get-queue-attributes --queue-url <queue url> --attribute-names ApproximateNumberOfMessages`. A number that keeps growing means Asymptote isn't consuming it yet; send your Asymptote contact the queue URL. Messages that failed ten times wait in the `-dlq` queue for 14 days.
  </Accordion>

  <Accordion title="DeleteConflict: Cannot delete entity, must delete access keys first">
    `terraform destroy` is trying to delete a user that still has a key you created with the AWS CLI. Delete the key first.
  </Accordion>
</AccordionGroup>

## Related

<Columns cols={2}>
  <Card title="Google Cloud Storage" icon="cloud" href="/deployment/self-hosted-gcs">
    The same setup with a Cloud Storage bucket.
  </Card>

  <Card title="Beacon with Fleet and S3" icon="bucket" href="/guides/fleet-s3-mdm">
    Roll out Beacon with Fleet.
  </Card>

  <Card title="AWS S3 with Terraform" icon="bucket" href="/log-forwarding/aws-s3-terraform">
    The bucket and writer on their own, without Asymptote reading them.
  </Card>

  <Card title="Hosting options" icon="server" href="/deployment/hosting-options">
    Compare Managed, Open Source, and Private Deployment.
  </Card>
</Columns>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.