Skip to content

Object Storage Features & Limitations

This page documents the features available on the Infomaniak Ceph Object Storage platform.

Feature Matrix

Feature Status Notes
Object Lock Supported Version-level immutability. GOVERNANCE & COMPLIANCE modes supported.
Multipart Upload Supported Create, list, abort, complete. Minimum part: 5 MB (last part exempt), maximum part: 5GB, 10 000 parts max.
SSE-C (Customer-Provided Key) Supported AES256. Key + MD5 must be provided on every request.
SSE-S3 (Server-Side Encryption) Not tested AWS CLI rejects --server-side-encryption against RGW. Untested via raw HTTP.
Versioning Supported Enable, suspend, list versions, delete specific version.
Bucket Replication Supported Enable, disable, get configuration. Cross-region supported.
Object Tagging Supported Via x-amz-tagging on upload or put-object-tagging. Up to 10 tags.
User Metadata Supported x-amz-meta-* headers returned on HEAD/GET. Max 2 KB per object.
Lifecycle Policies Supported Expiration rules, prefix filtering, multipart cleanup.
CORS Supported PUT/GET bucket CORS configuration.
Requester Pays Supported Bucket-level requester-charges configuration.
Copy Object Supported Cross-bucket, cross-tenant (with permissions). Metadata replaced or copied.
S3 Select Deprecated Deprecated by AWS and being removed in Ceph. Do not use for new workloads.
Presigned URLs Supported Temporary access via query-string signature.
Public Buckets Supported Path-style required for anonymous access with Keystone multi-tenancy.
Bucket Policies (cross-tenant) Supported Principal: arn:aws:iam::<tenant>:root works.
Static Website Hosting Not Supported Will return MethodNotAllowed.
Event Notifications Not Configured Requires RGW-side AMQP/Kafka/HTTP endpoint.
SSE-KMS Not Supported No KMS integration. Use SSE-C.
Bucket Logging Supported "Lazy" , can take several hours to write logs.

Multipart Upload

Multipart upload is the only way to upload objects larger than 5 GB. Ceph Object Storage supports the complete multipart workflow.

Step-by-step: upload a 12 MB file in 2 parts

Step 1 — Initiate the upload

aws s3api create-multipart-upload \
  --bucket my-bucket \
  --key data/large-file.dat \
  --profile my-profile --region gva2

Output:

{
  "UploadId": "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW",
  "Bucket": "my-bucket",
  "Key": "data/large-file.dat"
}

Store this UploadId. It is required for every subsequent operation.

Step 2a — Upload part 1 (5 MB minimum, last part exempt)

dd if=/dev/zero of=/tmp/part1.bin bs=1M count=5 status=none

aws s3api upload-part \
  --bucket my-bucket --key data/large-file.dat \
  --part-number 1 --body /tmp/part1.bin \
  --upload-id "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW" \
  --profile my-profile --region gva2 \
  --query ETag --output text
# "5f363e0e58a95f06cbe9bbc662c5dfb6"

Step 2b — Upload part 2 (7 MB)

dd if=/dev/zero of=/tmp/part2.bin bs=1M count=7 status=none

aws s3api upload-part \
  --bucket my-bucket --key data/large-file.dat \
  --part-number 2 --body /tmp/part2.bin \
  --upload-id "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW" \
  --profile my-profile --region gva2 \
  --query ETag --output text
# ETag of part 2

Step 3 — Complete the multipart upload

You MUST provide ALL parts with their exact ETags and part numbers:

aws s3api complete-multipart-upload \
  --bucket my-bucket --key data/large-file.dat \
  --upload-id "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW" \
  --multipart-upload 'Parts=[{ETag="5f363e0e58a95f06cbe9bbc662c5dfb6",PartNumber=1},{ETag="ETAG_PART2_HERE",PartNumber=2}]' \
  --profile my-profile --region gva2

Output:

{
  "ETag": "\"81485c0e873d222199469076b60f30e9-2\"",
  "Location": "s3.gva2.infomaniak.cloud/tenant:my-bucket/data/large-file.dat",
  "Bucket": "my-bucket",
  "Key": "data/large-file.dat"
}

The ETag ends with -2 which means it is a multipart object with 2 parts.

Inspect an active multipart upload

List uploads in progress for this bucket:

aws s3api list-multipart-uploads \
  --bucket my-bucket \
  --profile my-profile --region gva2

Output (if an upload is active):

{
  "Uploads": [
    {
      "UploadId": "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW",
      "Key": "data/large-file.dat",
      "Initiated": "2026-06-25T11:03:15+00:00"
    }
  ]
}

List the parts already uploaded for a specific upload:

aws s3api list-parts \
  --bucket my-bucket --key data/large-file.dat \
  --upload-id "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW" \
  --profile my-profile --region gva2

Output:

{
  "Parts": [
    {
      "PartNumber": 1,
      "LastModified": "2026-06-25T11:03:20+00:00",
      "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
      "Size": 5242880,
      "ChecksumCRC64NVME": "vqf3hRLTlJw="
    }
  ]
}

Abort an incomplete upload

If you do not complete an upload, it remains on the server and consumes storage. Abort it explicitly:

aws s3api abort-multipart-upload \
  --bucket my-bucket --key data/large-file.dat \
  --upload-id "2~TPPI1kyv_ZNI1VQqMSJ4orj45ycjhNW" \
  --profile my-profile --region gva2

No output on success. After abort, the part data is permanently removed.

Multipart Upload Size Limits

Hard limit: ~48.8 TB per object

S3 multipart uploads have two fixed limits:

Limit Value What it means
Maximum parts 10 000 You cannot upload more than 10 000 parts for a single object
Maximum part size 5 GB Each individual part cannot exceed 5 GB

Maximum object size: 5 GB × 10 000 = ~48.8 TB

Objects larger than ~48 TB cannot be stored as a single S3 object. You must split them into multiple objects (e.g., dataset/part-001, dataset/part-002).

Practical examples

Object size Part size Number of parts Status
10 TB 5 GB ~2 050 Works
40 TB 5 GB ~8 200 Works
48 TB 5 GB ~9 830 Works (near limit)
100 TB 5 GB 20 480 Fails — exceeds 10 000 part limit

For files approaching the limit, rclone automatically optimizes multipart parameters:

rclone copy large-file.dat s3aas1:my-bucket/

rclone will use the largest possible parts (up to 5 GB) and warn if the file exceeds what can be uploaded as a single object.

Manual multipart upload

If using aws s3api directly for large files, ensure you stay under both limits. The minimum part size (except the last) is 5 MB.

# Example: Upload a 10 TB file with 5 GB parts (~2 048 parts)
# 1. Initiate
aws s3api create-multipart-upload --bucket my-bucket --key large-file.dat

# 2. Upload parts (5 GB each, last part can be smaller)
for i in $(seq 1 2048); do
  aws s3api upload-part --bucket my-bucket --key large-file.dat \
    --part-number $i --body part-$i.bin --upload-id "YOUR_UPLOAD_ID"
done

# 3. Complete
aws s3api complete-multipart-upload --bucket my-bucket --key large-file.dat \
  --upload-id "YOUR_UPLOAD_ID" --multipart-upload file://parts.json

Do NOT exceed 10 000 parts

Attempting to complete a multipart upload with more than 10 000 parts will fail with an error. Always verify: total_parts <= 10000 and each_part <= 5 GB.


Automated multipart cleanup with lifecycle

See the lifecycle section below for automatically removing incomplete multipart uploads older than N days.


Server-Side Encryption (SSE-C)

SSE-C lets you provide your own AES-256 encryption key. Ceph encrypts the object but does not store the key. You must provide the same key on every GET/HEAD.

Generate a key

# Generate 32 bytes (256 bits) as Base64
export SSE_KEY=$(openssl rand -base64 32)
# Compute MD5 of the raw key bytes (not the Base64 string)
export SSE_KEY_MD5=$(echo -n "$SSE_KEY" | openssl md5 -binary | base64)

echo "Key (keep secret): $SSE_KEY"
echo "MD5 (sent in headers): $SSE_KEY_MD5"

Example output:

Key (keep secret): 2R6ks18Jg+ZAWDHqtviNX5ZGwVhcr7evIS2gU9Iesp8=
MD5 (sent in headers): 5OA/MjuqXDtMP04vL9rq/Q==

Important: The MD5 is sent to the server via x-amz-server-side-encryption-customer-key-MD5 so the server can validate it received the correct key.

Upload with SSE-C

aws s3api put-object \
  --bucket my-bucket \
  --key sensitive-report.pdf \
  --body ./sensitive-report.pdf \
  --sse-customer-algorithm AES256 \
  --sse-customer-key "$SSE_KEY" \
  --sse-customer-key-md5 "$SSE_KEY_MD5" \
  --profile my-profile --region gva2

Output:

{
  "ETag": "\"6f5902ac237024bdd0c176cb93063dc4\"",
  "SSECustomerAlgorithm": "AES256",
  "SSECustomerKeyMD5": "t8xKGB5x0owGs4iEFHZ+qg=="
}

Notice SSECustomerAlgorithm and SSECustomerKeyMD5 in the response.

Download with SSE-C

You MUST provide the same key, or the GET will fail:

aws s3api get-object \
  --bucket my-bucket \
  --key sensitive-report.pdf \
  --sse-customer-algorithm AES256 \
  --sse-customer-key "$SSE_KEY" \
  --sse-customer-key-md5 "$SSE_KEY_MD5" \
  ./downloaded-report.pdf \
  --profile my-profile --region gva2

Output:

{
  "AcceptRanges": "bytes",
  "ContentLength": 1337,
  "ContentType": "application/octet-stream",
  "ETag": "\"6f5902ac237024bdd0c176cb93063dc4\"",
  "LastModified": "2026-06-25T11:03:15+00:00",
  "SSECustomerAlgorithm": "AES256",
  "SSECustomerKeyMD5": "t8xKGB5x0owGs4iEFHZ+qg=="
}

HEAD without key fails

aws s3api head-object \
  --bucket my-bucket \
  --key sensitive-report.pdf \
  --profile my-profile --region gva2

Error:

An error occurred (400) when calling the HeadObject operation: Bad Request

SSE-C objects cannot be read without the key. The server returns 400 Bad Request because it needs the key to decrypt and verify the object.

Note: SSE-S3 (Ceph-managed encryption) could not be validated because the AWS CLI rejects --server-side-encryption against RGW endpoints. Use SSE-C as a reliable alternative.


Versioning

When enabled, every PUT on an existing key creates a new version. Old versions are preserved until explicitly deleted.

Enable versioning on a bucket

aws s3api put-bucket-versioning \
  --bucket my-bucket \
  --versioning-configuration Status=Enabled \
  --profile my-profile --region gva2

Get the current status:

aws s3api get-bucket-versioning \
  --bucket my-bucket \
  --profile my-profile --region gva2

Output:

{
  "Status": "Enabled",
  "MFADelete": "Disabled"
}

Upload two versions of the same file

# Version 1 — upload original content
echo "original content v1" > /tmp/file.txt
aws s3 cp /tmp/file.txt s3://my-bucket/versioned.txt \
  --profile my-profile --region gva2
# upload: ./file.txt to s3://my-bucket/versioned.txt

# Version 2 — overwrite with new content
echo "updated content v2" > /tmp/file.txt
aws s3 cp /tmp/file.txt s3://my-bucket/versioned.txt \
  --profile my-profile --region gva2
# upload: ./file.txt to s3://my-bucket/versioned.txt

List versions

aws s3api list-object-versions \
  --bucket my-bucket \
  --prefix versioned.txt \
  --profile my-profile --region gva2

Output:

{
  "Versions": [
    {
      "ETag": "\"9d4d14c9f2d54d1249e6ac856dbdf9ad\"",
      "Size": 19,
      "StorageClass": "STANDARD",
      "Key": "versioned.txt",
      "VersionId": "iD2IkNeb0v5UA4swLlf-ZBBld1Pad-e",
      "IsLatest": true,
      "LastModified": "2026-06-25T11:03:28+00:00",
      "Owner": {
        "DisplayName": "PCP-V2ZD4G3",
        "ID": "ecff9c4d00d5497f8cd7ee9482c652ad$ecff9c4d00d5497f8cd7ee9482c652ad"
      }
    },
    {
      "ETag": "\"7f67cac5a4edd578dea1c95387bad47c\"",
      "Size": 18,
      "Key": "versioned.txt",
      "VersionId": "meCZ5HWVF2fb0pPIIQ2ChxlPQYqDK2p",
      "IsLatest": false,
      "LastModified": "2026-06-25T11:03:27+00:00"
    }
  ]
}

Key observations:

  • VersionId is unique per upload
  • IsLatest: true on the most recent
  • IsLatest: false on older versions

Read an old version explicitly

aws s3api get-object \
  --bucket my-bucket \
  --key versioned.txt \
  --version-id meCZ5HWVF2fb0pPIIQ2ChxlPQYqDK2p \
  /tmp/old-version.txt \
  --profile my-profile --region gva2

cat /tmp/old-version.txt
# original content v1

Delete a specific version (permanent)

aws s3api delete-object \
  --bucket my-bucket \
  --key versioned.txt \
  --version-id meCZ5HWVF2fb0pPIIQ2ChxlPQYqDK2p \
  --profile my-profile --region gva2

Output:

{
  "VersionId": "meCZ5HWVF2fb0pPIIQ2ChxlPQYqDK2p"
}

The version is permanently removed. No delete marker is created.

Warning: aws s3 rm s3://my-bucket/versioned.txt (without --version-id) creates a DeleteMarker, not a real deletion. The file remains in old versions.

Suspend versioning

aws s3api put-bucket-versioning \
  --bucket my-bucket \
  --versioning-configuration Status=Suspended \
  --profile my-profile --region gva2

After suspension, new uploads overwrite the current version without creating new versions. Existing versions are preserved.


Bucket Replication

Bucket replication automatically copies bucket content to another datacenter. In case of Disaster Recovery scenario such as primary Datacenter down, Infomaniak can switch the S3 endpoints to the secondary zone, no action needed for customners, replicated objects will be available as nothing happened.

Prerequisites

  • The source bucket must have versioning enabled.
  • The target bucket must exist and have versioning enabled.
  • A bucket replication policy JSON file is required.

Enable replication

Create replication.json:

{
  "Role": "arn:aws:iam::account-id:role/replication-role",
  "Rules": [
    {
      "Status": "Enabled",
      "Priority": 1,
      "Filter": {},
      "Destination": {
        "Bucket": "arn:aws:s3:::my-bucket",
        "StorageClass": "STANDARD"
      },
      "DeleteMarkerReplication": {
        "Status": "Disabled"
      }
    }
  ]
}

Note

DeleteMarkerReplication is not implemented in Ceph. Delete markers are always replicated regardless of this setting.

Apply it:

aws s3api put-bucket-replication \
  --bucket my-bucket \
  --replication-configuration file://replication.json \
  --profile my-profile --region gva2

Verify replication status

aws s3api get-bucket-replication \
  --bucket my-bucket \
  --profile my-profile --region gva2

Output:

{
    "ReplicationConfiguration": {
        "Role": "",
        "Rules": [
            {
                "ID": "",
                "Priority": 1,
                "Status": "Enabled",
                "Destination": {
                    "Bucket": "arn:aws:s3::<tenant-id>:my-bucket",
                    "StorageClass": "STANDARD"
                }
            }
        ]
    }
}

Disable replication

aws s3api delete-bucket-replication \
  --bucket my-bucket \
  --profile my-profile --region gva2

No output on success. Existing replicated objects remain in the destination bucket.


Object Tagging

Tags are key-value strings attached to objects. Useful for classification, cost allocation, and filtering. Maximum 10 tags per object.

Add tags during upload

aws s3api put-object \
  --bucket my-bucket \
  --key report.pdf \
  --body ./annual-report.pdf \
  --tagging "env=prod&team=finance&year=2026" \
  --profile my-profile --region gva2

Add or overwrite tags on an existing object

aws s3api put-object-tagging \
  --bucket my-bucket \
  --key report.pdf \
  --tagging '{
    "TagSet": [
      {"Key": "env", "Value": "prod"},
      {"Key": "team", "Value": "finance"},
      {"Key": "retention", "Value": "7-years"}
    ]
  }' \
  --profile my-profile --region gva2

Read tags

aws s3api get-object-tagging \
  --bucket my-bucket \
  --key report.pdf \
  --profile my-profile --region gva2

Output:

{
  "TagSet": [
    {
      "Key": "env",
      "Value": "prod"
    },
    {
      "Key": "team",
      "Value": "finance"
    },
    {
      "Key": "retention",
      "Value": "7-years"
    }
  ]
}

Delete all tags

aws s3api delete-object-tagging \
  --bucket my-bucket \
  --key report.pdf \
  --profile my-profile --region gva2

No output. Subsequent get-object-tagging returns an empty TagSet.


User Metadata

User metadata is a set of custom key-value headers stored alongside the object and returned on every HEAD/GET.

Upload with metadata

aws s3api put-object \
  --bucket my-bucket \
  --key dataset-2026.csv \
  --body ./sales.csv \
  --metadata "project=alpha,owner=taylor,department=engineering,classification=internal" \
  --profile my-profile --region gva2

Read metadata via HEAD

aws s3api head-object \
  --bucket my-bucket \
  --key dataset-2026.csv \
  --profile my-profile --region gva2

Output:

{
  "ContentLength": 1337,
  "ContentType": "application/octet-stream",
  "ETag": "\"d6eb32081c822ed572b70567826d9d9d\"",
  "LastModified": "2026-06-25T11:03:15+00:00",
  "Metadata": {
    "x-amz-meta-classification": "internal",
    "x-amz-meta-department": "engineering",
    "x-amz-meta-owner": "taylor",
    "x-amz-meta-project": "alpha"
  }
}

AWS prefixes all user metadata keys with x-amz-meta- automatically.

Immutable metadata

Metadata is immutable for a given version. To change metadata on an existing object without uploading new data, copy the object to itself and replace metadata:

aws s3api copy-object \
  --copy-source "my-bucket/dataset-2026.csv" \
  --bucket my-bucket \
  --key dataset-2026.csv \
  --metadata-directive REPLACE \
  --metadata "project=beta,owner=taylor,updated=2026-06-25" \
  --profile my-profile --region gva2

This creates a new version (if versioning is enabled) or overwrites the object.


Lifecycle Policies

Automate expiration, transition, and cleanup based on object age or prefix.

Rule: delete objects in logs/ after 30 days

Create a file lifecycle.json:

{
  "Rules": [
    {
      "ID": "expire-logs-after-30-days",
      "Filter": {
        "Prefix": "logs/"
      },
      "Status": "Enabled",
      "Expiration": {
        "Days": 30
      }
    }
  ]
}

Apply it:

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json \
  --profile my-profile --region gva2

Rule: clean up incomplete multipart uploads after 7 days

Create cleanup.json:

{
  "Rules": [
    {
      "ID": "cleanup-incomplete-uploads",
      "Filter": {
        "Prefix": ""
      },
      "Status": "Enabled",
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    }
  ]
}

Apply it:

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration file://cleanup.json \
  --profile my-profile --region gva2

Read current rules

aws s3api get-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --profile my-profile --region gva2

Output:

{
  "Rules": [
    {
      "Expiration": {
        "Days": 30
      },
      "ID": "expire-logs-after-30-days",
      "Filter": {
        "Prefix": "logs/"
      },
      "Status": "Enabled"
    }
  ]
}

Delete all lifecycle rules

Send an empty configuration:

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration '{"Rules": []}' \
  --profile my-profile --region gva2

CORS (Cross-Origin Resource Sharing)

Allow browsers to read/write your bucket from other origins. Required for direct browser-to-S3 uploads and JavaScript apps.

Configuration file cors.json

{
  "CORSRules": [
    {
      "AllowedOrigins": [
        "https://example.com",
        "https://app.example.com"
      ],
      "AllowedMethods": [
        "GET",
        "HEAD",
        "PUT",
        "POST"
      ],
      "AllowedHeaders": [
        "*"
      ],
      "MaxAgeSeconds": 3000
    }
  ]
}

If you want to allow any origin (for development only, dangerous in production):

{
  "CORSRules": [
    {
      "AllowedOrigins": ["*"],
      "AllowedMethods": ["GET", "HEAD"],
      "AllowedHeaders": ["*"],
      "MaxAgeSeconds": 3000
    }
  ]
}

Apply and verify

aws s3api put-bucket-cors \
  --bucket my-bucket \
  --cors-configuration file://cors.json \
  --profile my-profile --region gva2

aws s3api get-bucket-cors \
  --bucket my-bucket \
  --profile my-profile --region gva2

Output:

{
  "CORSRules": [
    {
      "AllowedHeaders": ["*"],
      "AllowedMethods": ["GET", "HEAD"],
      "AllowedOrigins": ["https://example.com"],
      "MaxAgeSeconds": 3000
    }
  ]
}

Test with curl (preflight)

curl -I -X OPTIONS \
  -H "Origin: https://example.com" \
  -H "Access-Control-Request-Method: GET" \
  "https://s3.gva2.infomaniak.cloud/my-bucket/test.txt"

Expected headers in response:

Access-Control-Allow-Origin: https://example.com
Access-Control-Allow-Methods: GET, HEAD
Access-Control-Max-Age: 3000

Delete CORS

aws s3api delete-bucket-cors \
  --bucket my-bucket \
  --profile my-profile --region gva2

Requester Pays

By default, the bucket owner pays for bandwidth. Enable Requester Pays to make the caller tenant pay instead. This is the AWS S3 mechanism for shifting egress costs from the bucket owner to the requester.

Enable Requester Pays

aws s3api put-bucket-request-payment \
  --bucket my-bucket \
  --request-payment-configuration '{"Payer": "Requester"}' \
  --profile my-profile --region gva2

Verify

aws s3api get-bucket-request-payment \
  --bucket my-bucket \
  --profile my-profile --region gva2

Output:

{
  "Payer": "Requester"
}

How access changes with Requester Pays

Bucket owner: No change. Continues to access normally without any additional header.

Requester (another tenant or anonymous): Must include x-amz-request-payer: requester in every request. Without this header, requests are rejected.

# Requester must add --request-payer requester
curl -I -H "x-amz-request-payer: requester" \
  "https://s3.gva2.infomaniak.cloud/<tenant>:<bucket>/<object>"

Important: Requester Pays does not make a private bucket public. The requester still needs read permission via a bucket policy or ACL. The x-amz-request-payer header only affects who is billed for the transfer.

Disable Requester Pays

aws s3api put-bucket-request-payment \
  --bucket my-bucket \
  --request-payment-configuration '{"Payer": "BucketOwner"}' \
  --profile my-profile --region gva2

Infomaniak billing model

Storage costs vs traffic costs

On Infomaniak Ceph Object Storage:

  • Storage is billed to the bucket owner (the tenant that owns the bucket)
  • There are currently no traffic/egress fees on Infomaniak Ceph Object Storage
  • Requester Pays is implemented and functional at the API level

Because Infomaniak does not charge for traffic, Requester Pays has no practical billing effect in the current pricing model. The feature is available for API compatibility and future use, but enabling it will not shift any actual costs to requesters.

If traffic-based billing is introduced in the future, Requester Pays will become relevant for cross-tenant shared datasets.


Copy Object

Copy objects within a bucket, across buckets, or across tenants.

Simple same-bucket copy

aws s3api copy-object \
  --copy-source "my-bucket/source.txt" \
  --bucket my-bucket \
  --key backup/source.txt \
  --profile my-profile --region gva2

Output:

{
  "CopyObjectResult": {
    "ETag": "\"d6eb32081c822ed572b70567826d9d9d\"",
    "LastModified": "2026-06-25T11:03:45+00:00"
  }
}

Cross-bucket copy

Requires read access on source and write access on destination.

aws s3api copy-object \
  --copy-source "source-bucket/backup.zip" \
  --bucket destination-bucket \
  --key imported/backup.zip \
  --profile my-profile --region gva2

Copy a specific version

aws s3api copy-object \
  --copy-source "my-bucket/document.pdf?version-id=iD2IkNeb0v5UA4swLlf-ZBBld1Pad-e" \
  --bucket my-bucket \
  --key document-restore.pdf \
  --profile my-profile --region gva2

Copy and replace metadata

aws s3api copy-object \
  --copy-source "my-bucket/old-report.pdf" \
  --bucket my-bucket \
  --key new-report.pdf \
  --metadata-directive REPLACE \
  --metadata "version=2,status=final,owner=alice" \
  --profile my-profile --region gva2

Copy with preserved metadata (default)

aws s3api copy-object \
  --copy-source "my-bucket/source.txt" \
  --bucket my-bucket \
  --key dest.txt \
  --metadata-directive COPY \
  --profile my-profile --region gva2

If --metadata-directive is omitted, metadata is copied by default.

Bucket Logging

Bucket logging provides logging for the requests that are made to a bucket. The log data can be used to monitor bucket activity, detect unauthorized access, get insights into bucket usage, and use the logs as a journal for bucket changes.

Note : The log bucket cannot have logging enabled on it. The log bucket cannot have any encryption set on it. The log bucket must not have RequestPayer enabled. The log bucket may have object lock enabled with default retention period.

Step-by-step: enable bucket logging on 2 different buckets

** Step 1 - Create two buckets

aws s3 mb s3://my-bucket1 
aws s3 mb s3://my-bucket2

** Step 2 - Create the log bucket

aws s3 mb s3://my-logging-bucket

** Step 3 - Allow the sources buckets ( my-bucket[1-2] ) to put logs on the log bucket

  • create and apply the bucket policy document:
cat > allow-logging.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowLoggingFromSourceBucket",
      "Effect": "Allow",
      "Principal": {
        "Service": "logging.s3.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": “arn:aws:s3::<tenant-id>:my-logging-bucket/*”,
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": “<tenant-id>$<tenant-id>“,
          "aws:SourceArn": [
            “arn:aws:s3::<tenant-id>:my-bucket1”,
            “arn:aws:s3::<tenant-id>:my-bucket2”
        ]
      }
     }
    }
  ]
}
EOF

aws s3api put-bucket-policy --bucket=my-logging-bucket --policy file://allow-logging.json

  • Enable bucket logging on the source buckets :
cat > my-bucket1-logging.json <<EOF
{
  "LoggingEnabled": {
    "TargetBucket": "my-logging-bucket",
    "TargetPrefix": "my-bucket1/"
  }
}

aws s3api put-bucket-logging --bucket my-bucket1 --bucket-logging-status file://my-bucket1-logging.json

Repeat the step 2-b for the my-bucket2

** Inspect the applied policies :

Check the bucket policy on the my-logging-bucket: aws s3api get-bucket-policy --bucket=my-logging-bucket

Example Output:

{
    "Policy": "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"AllowLoggingFromSourceBucket\",\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"Service\": \"logging.s3.amazonaws.com\"\n      },\n      \"Action\": \"s3:PutObject\",\n      \"Resource\": \"arn:aws:s3::2c718fb4ecb440e7917bc9312bb99988:my-logging-bucket/*\",\n      \"Condition\": {\n        \"StringEquals\": {\n          \"aws:SourceAccount\": \"2c718fb4ecb440e7917bc9312bb99988$2c718fb4ecb440e7917bc9312bb99988\",\n          \"aws:SourceArn\": [\n            \"arn:aws:s3::2c718fb4ecb440e7917bc9312bb99988:my-bucket-1\",\n            \"arn:aws:s3::2c718fb4ecb440e7917bc9312bb99988:my-bucket-2\"\n        ]\n      }\n     }\n    }\n  ]\n}\n"
}

Check the applied bucket logging policy : aws s3api get-bucket-logging —bucket=my-bucket-1

Example Output:

{
    "LoggingEnabled": {
        "TargetBucket": "my-logging-bucket",
        "TargetPrefix": "my-bucket-1/",
        "TargetObjectKeyFormat": {
            "SimplePrefix": {}
        }
    }
}

** Deactivate bucket logging on an enabled logging bucket:

aws s3api put-bucket-logging --bucket my-bucket-1 --bucket-logging-status ‘{}'

  • Log Format:

The S3 access logs are stored with the following format :

<Targetprefix>/<year-month-day-hour-minute-second>-<16 bytes unique-id>

Note : Logging to the bucket may take several hours after the operation to the log is been written to the target bucket. To accelerate that, you can flush all logging objects on a given source bucket to log them . Flushing will happen automatically when logging is deactivated on a bucket, or its logging configuration is changed, or the bucket is deleted.


Known Limitations

AWS CLI Compatibility Issues

Some AWS CLI commands fail against RGW before the request is even sent:

Operation CLI Error Workaround
aws s3 sync XML parsing errors Use s3cmd sync or rclone
aws s3 cp --server-side-encryption ParamValidation: Unknown options Use SSE-C (--sse-customer-*) instead
aws s3api put-bucket-website NoneType is not iterable Not supported on cluster
aws s3 ls on large buckets Slow / timeout Use s3cmd or rclone

Features Not Supported on Tested Cluster

Feature Status Alternative
Static Website Hosting Returns MethodNotAllowed Use a CDN or reverse proxy (e.g., Nginx with proxy_pass)
Event Notifications No SNS/AMQP/Kafka configured Poll with ListObjectsV2 or use a Lambda-like external worker
SSE-KMS No KMS integration Use SSE-C with your own keys