A lightweight, minimal Prometheus exporter for S3 environments where you do not have access to the metrics.
This exporter automatically discovers all buckets your credentials have access to and exposes key metrics about their size and object count.
- 🔍 Auto-Discovery: Automatically lists and iterates through all available buckets.
- 🔒 Strict Configuration: Fails fast.
- 🏗️ Self-Hosted First: Enforces PathStyle addressing out of the box, which is required for most on-premise S3 installations.
- 🪶 Lightweight: Deploys as a statically linked binary in an empty Docker image (< 15MB), with no shell and an unprivileged user.
Metrics are exposed on the /s3-metrics endpoint. The per-bucket metrics include a bucket label with the name of the S3 bucket.
| Metric Name | Type | Description |
|---|---|---|
| s3_bucket_objects_total | Gauge | Total number of objects currently in the bucket. |
| s3_bucket_size_bytes | Gauge | Total combined size of all objects in the bucket. |
| s3_bucket_last_modified_timestamp_seconds | Gauge | Unix timestamp of the most recently modified object. |
| s3_bucket_scrape_success | Gauge | Whether the scrape fully enumerated this bucket (1) or failed (0). On failure the bucket's other metrics are omitted. |
| s3_exporter_scrape_success | Gauge | Whether the scrape successfully listed buckets (1) or failed (0). No bucket label. |
An example output looks like this:
# HELP s3_bucket_last_modified_timestamp_seconds Timestamp of the most recently modified object
# TYPE s3_bucket_last_modified_timestamp_seconds gauge
s3_bucket_last_modified_timestamp_seconds{bucket="mybucket-1"} 1.772715739e+09
s3_bucket_last_modified_timestamp_seconds{bucket="mybucket-2"} 1.772715739e+09
s3_bucket_last_modified_timestamp_seconds{bucket="mybucket-3"} 1.772715739e+09
s3_bucket_last_modified_timestamp_seconds{bucket="mybucket-4"} 1.772715739e+09
# HELP s3_bucket_objects_total Total number of objects in the bucket
# TYPE s3_bucket_objects_total gauge
s3_bucket_objects_total{bucket="mybucket-1"} 12
s3_bucket_objects_total{bucket="mybucket-2"} 3000
s3_bucket_objects_total{bucket="mybucket-3"} 3
s3_bucket_objects_total{bucket="mybucket-4"} 0
# HELP s3_bucket_size_bytes Total size of all objects in the bucket
# TYPE s3_bucket_size_bytes gauge
s3_bucket_size_bytes{bucket="mybucket-1"} 154236
s3_bucket_size_bytes{bucket="mybucket-2"} 234562143
s3_bucket_size_bytes{bucket="mybucket-3"} 65204
s3_bucket_size_bytes{bucket="mybucket-4"} 0
# HELP s3_bucket_scrape_success Whether the most recent scrape fully enumerated this bucket (1) or failed (0)
# TYPE s3_bucket_scrape_success gauge
s3_bucket_scrape_success{bucket="mybucket-1"} 1
s3_bucket_scrape_success{bucket="mybucket-2"} 1
s3_bucket_scrape_success{bucket="mybucket-3"} 1
s3_bucket_scrape_success{bucket="mybucket-4"} 1
# HELP s3_exporter_scrape_success Whether the most recent scrape successfully listed buckets (1) or failed (0)
# TYPE s3_exporter_scrape_success gauge
s3_exporter_scrape_success 1The exporter is configured via environment variables. There are no default fallbacks for AWS credentials. If a required variable is missing, the application will crash immediately on startup.
| Environment Variable | Required | Default | Description |
|---|---|---|---|
| AWS_S3_ACCESS_KEY_ID | ✅ Yes | - | Your S3 Access Key. |
| AWS_S3_SECRET_ACCESS_KEY | ✅ Yes | - | Your S3 Secret Key. |
| AWS_S3_REGION | ✅ Yes | - | The S3 region (e.g., us-east-1, or your custom region). |
| AWS_S3_ENDPOINT_URL | ✅ Yes | - | Full URL to your S3 cluster (e.g., http://s3.internal:9000). |
| METRICS_PORT | ❌ No | 9300 | The port the HTTP server binds to. |
| METRICS_PATH | ❌ No | /s3-metrics | HTTP path the metrics handler is registered on. |
| METRICS_ADDRESS | ❌ No | 127.0.0.1 | The address the HTTP server binds to. |
Build the image:
docker build -t s3-exporter:latest .Run the container:
docker run \
--name s3-exporter \
-p 9300:9300 \
-e AWS_S3_ACCESS_KEY_ID="your-access-key" \
-e AWS_S3_SECRET_ACCESS_KEY="your-secret-key" \
-e AWS_S3_REGION="us-east-1" \
-e AWS_S3_ENDPOINT_URL="http://your-s3-cluster:9000" \
s3-exporter:latestOr use a compose file with an accompanying .env file:
services:
s3-exporter:
image: ghcr.io/virtuos/s3-exporter:latest
container_name: s3-exporter
restart: unless-stopped
ports:
- "9300:9300"
env_file:
- .envEnsure you have Go 1.25+ installed, then run:
export AWS_S3_ACCESS_KEY_ID="your-access-key"
export AWS_S3_SECRET_ACCESS_KEY="your-secret-key"
export AWS_S3_REGION="us-east-1"
export AWS_S3_ENDPOINT_URL="http://localhost:9000"
go mod tidy
go run main.goVerify it works by curling the endpoint:
curl http://localhost:9300/s3-metricsAdd the following scrape job to your prometheus.yml:
scrape_configs:
- job_name: 's3-exporter'
metrics_path: '/s3-metrics'
static_configs:
- targets: ['s3-exporter:9300']Below you find two examples for alerting rules for the following scenarios:
- You might use S3 to save backups. In that case you can see from the last modified date of a bucket, when the last backup took place.
- We can not easily retrieve the quotas from the S3 environment, but if you know them, you can alert if the bucket grows too big.
groups:
- name: s3-exporter-alerts
rules:
# ==========================================
# Alert: Bucket hasn't been updated in 3 days
# useful to monitor backups into S3
# ==========================================
- alert: S3BucketStale
# time() gets current Unix timestamp.
# 3 days = 3 * 24 hours * 3600 seconds
expr: (time() - s3_bucket_last_modified_timestamp_seconds) > (3 * 24 * 3600)
for: 5m
labels:
severity: warning
annotations:
summary: "S3 Bucket '{{ $labels.bucket }}' is stale"
description: >
No objects have been uploaded or modified in the bucket '{{ $labels.bucket }}'
for over 3 days. Check if the upstream backup/upload jobs are failing.
# ==========================================
# Alert: Bucket size exceeds 100 GB
# ==========================================
- alert: S3BucketTooLarge
# 100 GB calculated in bytes (100 * 1024^3)
expr: s3_bucket_size_bytes > (100 * 1024 * 1024 * 1024)
for: 5m
labels:
severity: critical
annotations:
summary: "S3 Bucket '{{ $labels.bucket }}' exceeded 100 GB"
description: >
The S3 bucket '{{ $labels.bucket }}' has exceeded the 100 GB threshold.
Current size is {{ $value | humanize1024 }}.
# ==========================================
# Alert: The exporter could not scrape S3
# fires both when listing buckets fails and
# when an individual bucket fails to enumerate
# ==========================================
- alert: S3ScrapeFailed
expr: s3_exporter_scrape_success == 0 or s3_bucket_scrape_success == 0
for: 5m
labels:
severity: critical
annotations:
summary: "S3 exporter scrape failing"
description: >
The exporter failed to collect metrics from S3. Either listing
buckets failed (s3_exporter_scrape_success), or a specific bucket
could not be fully enumerated (s3_bucket_scrape_success), in which
case that bucket's other metrics are omitted until it recovers.This exporter performs a synchronous scrape. Every time Prometheus queries the /s3-metrics endpoint, the exporter enumerates your S3 buckets in real-time (up to 8 buckets concurrently). This works best in smaller buckets:
- Small to Medium Buckets: This happens in milliseconds and works perfectly.
- Massive Buckets: The AWS SDK will need to paginate extensively (1,000 objects per page). This takes time and causes high API load on your S3 cluster.
A few built-in safeguards bound this load:
- Scrape timeout: each scrape is capped at an internal hard limit of 30 seconds. If enumeration exceeds it, the scrape is aborted; the affected buckets report
s3_bucket_scrape_success=0and their other metrics are omitted for that scrape. This limit is a compile-time constant, so raising Prometheus'scrape_timeoutbeyond 30s has no effect — for extremely large buckets, scrape less frequently instead. - No overlapping scrapes: only one scrape runs at a time. Concurrent requests to the endpoint receive HTTP 503 rather than triggering another full enumeration, so a slow scrape cannot multiply the load on your S3 cluster.
MIT
virtUOS, Osnabrueck University