HybridS3

HybridS3

Lightweight object storage that speaks S3 (boto3/AWS SDK compatible), plain HTTP, and MCP. SQLite for metadata, flat files on disk.

---

Table of Contents

• Why
• Quick Start
• Running
• Configuration
• Keys and Access
• Key pair model
• Master key
• Bucket visibility
• HTTP API
• Authentication
• Endpoints
• Object keys
• Response format
• S3 Interface
• Authentication
• Operations
• MCP Interface
• Connecting
• Authentication
• Tools
• Agent integrations
• Presigned URLs
• MIME Type Detection
• TTL and Expiry
• Locking
• Logging
• Behind a Reverse Proxy
• Security
• Testing
• Development
• Architecture
• Limitations
• License

---

Why

Most self-hosted S3-compatible storage is designed for large-scale deployments. Distributed erasure coding, IAM policies, WORM compliance, full web consoles — useful if you're running a cloud, overkill if you just want a place to put files that various services and AI agents can read and write.

AWS Sig V4 breaks behind reverse proxy path prefixes. Most implementations verify signatures using the full original upstream path. Put them behind nginx at /storage/, nginx strips the prefix, the server sees /bucket/key instead of /storage/bucket/key, the signature check fails. HybridS3 has a path_prefix config option — set it to /storage and all routes move under that prefix. No path stripping, no special proxy headers. boto3's signed path matches what the server sees.

Three interfaces, one service. boto3 works out of the box. Plain HTTP with curl works. AI agents connect via MCP and get structured tool definitions. No separate services for different clients.

Buckets are configuration, not state. There is no API to create or delete buckets. They live in the YAML config file. You always know exactly what exists, it's version-controlled, and there are no surprise buckets accumulating garbage.

TTL expiry is built in. Set ttl: 24h on a bucket and objects expire automatically after their last write. No lifecycle policies, no cron jobs, no separate process.

Readable and modifiable. Small enough to understand in an afternoon.

---

Quick Start

# get the example config

wget -O config.yaml https://raw.githubusercontent.com/psyb0t/docker-hybrids3/master/config.example.yaml

edit it — set your keys and define your buckets

vi config.yaml

run

docker run -d --name hybrids3 \ -p 8080:8080 \ -v ./config.yaml:/config/config.yaml:ro \ -v hybrids3-data:/data \ psyb0t/hybrids3

verify

curl http://localhost:8080/health

---

Running

The container expects:

• Config file at /config/config.yaml
• Data directory at /data
• Port 8080

Runs as UID 1000.

docker run:

docker run -d --name hybrids3 \

-p 8080:8080 \ -v ./config.yaml:/config/config.yaml:ro \ -v hybrids3-data:/data \ psyb0t/hybrids3

docker-compose:

services:

hybrids3: image: psyb0t/hybrids3 ports:

• "8080:8080" volumes:
• ./config.yaml:/config/config.yaml:ro
• hybrids3-data:/data restart: unless-stopped

volumes: hybrids3-data:

---

Configuration

# Cross-bucket god key. Works on every bucket for every operation.

Only credential that can list all buckets. Keep secret.

master_key: "change-me-to-something-secret"

Non-secret identifier paired with master_key in S3 auth (aws_access_key_id).

Safe to share — it grants nothing without the master_key.

master_public_key: "master"

How often the background loop runs to delete expired objects and orphan files.

Accepts human-readable durations: 30s, 5m, 1h, or raw seconds: 60

cleanup_interval: 1m

How long a request may wait to acquire a lock on an object before giving up with 503.

Accepts seconds as a float. Applies per object key, globally across all buckets.

lock_acquire_timeout: 30

How long a request may hold a lock before it is forcibly released and the request gets 503.

Protects against slow or stalled uploads blocking writes on the same key indefinitely.

lock_hold_timeout: 300

Maximum number of requests that may queue for the same object key at once.

Once the queue is full, new requests are rejected immediately with 503.

lock_max_waiters: 100

Serve all routes under this path prefix.

Set this when running behind a reverse proxy at a subpath (e.g. /storage).

Accepts "/storage" or "/storage/" — both are normalized.

Leave empty or omit to serve at the root.

path_prefix: /storage

TTL formats: 30s 5m 1h 1h30m12s 1d 2d12h 0 (never expire)

Size formats: 500B 50KB 10MB 1GB 0 (no limit)

buckets: uploads:

true — anyone can read (GET/HEAD/LIST) without authentication

false — authentication required for all operations

public: true

Private key for this bucket.

Used as the Bearer token value and as aws_secret_access_key in S3 auth.

Never transmitted — only used to sign or verify request signatures.

Keep secret.

key: "uploads-secret"

Public identifier for this bucket in S3 auth (aws_access_key_id).

Also appears in presigned URL Credential= fields.

Safe to share — it identifies the bucket but grants no access.

Defaults to the bucket name if not set.

public_key: "uploads-id"

Objects expire this long after their last write. Overwriting resets the clock.

0 means objects never expire.

ttl: 24h

Maximum upload size. Requests over this limit are rejected with 413.

0 means no limit.

max_file_size: 50MB

permanent: public: false key: "perm-secret" public_key: "permanent-id" ttl: 0 max_file_size: 100MB

---

Keys and Access

Key pair model

Each bucket has two keys defined in config:

The split is what makes presigned URLs work safely. A presigned URL must embed an identifier in the Credential= field so the server knows which key to verify against — that identifier is the public_key. Since it is non-secret, having it in the URL is fine. The private key signs the URL on the server and never appears in it.

Master key

The master_key is a cross-bucket credential that works on every bucket for every operation, without needing individual bucket keys. Two situations call for it:

• You need to operate across multiple buckets from a single client
• You need to list all configured buckets — bucket keys only see their own bucket in list_buckets

The master_public_key is the non-secret identifier that pairs with master_key in S3 auth (used as aws_access_key_id).

Do not embed the master key in client-facing code. Use per-bucket keys for that — they limit access to exactly one bucket.

Bucket visibility

---

HTTP API

Authentication

HTTP requests authenticate using a Bearer token in the Authorization header. Pass the bucket's private key, or the master key for cross-bucket operations.

Authorization: Bearer <private_key>

• Public bucket reads (GET, HEAD, LIST) — no authentication required.
• All writes (PUT, DELETE) and reads on private buckets — require the bucket key or master key.
• Listing buckets (GET /) — master key lists all buckets; bucket key lists only its own bucket.
• Presign endpoint (POST /presign/...) — requires the bucket key or master key.

# write to a bucket

curl -X PUT http://localhost:8080/uploads/file.txt \ -H "Authorization: Bearer uploads-secret" \ -d "hello"

read from a public bucket — no auth needed

curl http://localhost:8080/uploads/file.txt

read from a private bucket

curl http://localhost:8080/permanent/doc.pdf \ -H "Authorization: Bearer perm-secret"

list all buckets — master key sees all

curl http://localhost:8080/ \ -H "Authorization: Bearer your-master-key"

list buckets — bucket key sees only its own bucket

curl http://localhost:8080/ \ -H "Authorization: Bearer uploads-secret"

---

Endpoints

| POST | /presign/{bucket}/{key} | write | Generate a presigned URL (GET or PUT, see method qu