Skip to content

S3 compatible

S3 protocole is now available.

Note

It is S3 Compatible and not S3 amazon. Some features might not be available yet or simply not available.

OpenStack Docs: S3/Swift REST API Comparison Matrix

For example, multipart storage behavior isn't the same when overwrite a file. Previous chunks in +segments bucket will be preserved and will require manual action to delete them.

Region : Some software ask for the region to be put in a setting field. Use the default us-east-1 even though all data is located in our datacenters in Switzerland, it is simply a compatibility parameter.

SDK : You should set option forcePathStyle=true if you use a S3 SDK, otherwise it should fail with some bucket operations.

Create S3 Credentials

You first need to enable the credentials for the S3 protocol.

Run the following command and keep the access and secret keys in a safe place.

$ openstack ec2 credentials create
+------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                                                |
+------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| access     | 1161b3e891f24xxxxxxxxxxxxxxxxxxx                                                                                                                     |
| links      | {'self': 'https://api.pub1.infomaniak.cloud/identity/v3/users/153855c4fa1b4415bdd7681221259f72/credentials/OS-EC2/1161b3e89xxx1f24932xxxxxxxxxxxxx'} |
| project_id | d1440aa24a65411fb9bac2b842c8defa                                                                                                                     |
| secret     | 2e39f75488954xxxxxxxxxxxxxxxxxxx                                                                                                                     |
| trust_id   | None                                                                                                                                                 |
| user_id    | 153855c4fa1b4415bdd7681221259f72                                                                                                                     |
+------------+------------------------------------------------------------------------------------------------------------------------------------------------------+

List S3 credentials

    openstack ec2 credentials list

The server to connect to is : https://s3.pub1.infomaniak.cloud/

Setting up aws client

$ apt install awscli 
$ aws configure
AWS Access Key ID [None]: <your access key> 
AWS Secret Access Key [None]: <your secret key>
Default region name [None]:
Default output format [None]:

Listing buckets using aws-cli

    aws --endpoint-url=https://s3.pub1.infomaniak.cloud s3api list-buckets
    {
        "Buckets": [
            {
                "Name": "volumebackups",
                "CreationDate": "2009-02-03T16:45:09.000Z"
            }
        ],
        "Owner": {
            "DisplayName": "taylor:taylor",
            "ID": "taylor:taylor"
        }
    }

Note

The creationDate while listing directories will always be 2009-02-03T16:45:09.000Z'' which correpsonds to the date of the first commit of the s3api. it is a hardcoded value.

Bug #1889386

Creating a bucket

Creating the bucket "plesk"

    aws --endpoint-url=https://s3.pub1.infomaniak.cloud s3api create-bucket --bucket plesk
{ "Location": "/plesk" }
### Uploading a directory

```bash
    aws s3 cp --endpoint-url=https://s3.pub1.infomaniak.cloud <your-directory> s3://<your-bucket>/ --recursive

Warning

The header 100-continue isn't supported. In case your uploads show no progress with the information Waiting for 100 Continue response then you have to modify your s3api tool:

sudo vim /usr/lib/python3/dist-packages/botocore/awsrequest.py
def _send_request(self, method, url, body, headers, *args, **kwargs):
self._response_received = False
if headers.get('Expect', b'') == b'100-continue':
    self._expect_header_set = False

Set the line self._expect_header_set to False instead of True

Setting up s3cmd client

s3cmd --configure

Settings:
  Access Key: 9fcc2212146f4bb49ca2f78XXXXXXXXX
  Secret Key: 2850d658e2144f0c94bb6aeXXXXXXXXX
  Default Region: us-east-1
  S3 Endpoint: s3.pub1.infomaniak.cloud
  DNS-style bucket+hostname:port template for accessing a bucket: s3.pub1.infomaniak.cloud
  Encryption password: ENCR_KEY_OF_YOUR_CHOICE
  Path to GPG program: /bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n]
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/home/taylor/.s3cfg'

Object versioning

Creating a bucket with versioning enabled

Using the swift-client

taylor@laptop (pub1|taylor):~$ swift post -H "X-Versions-Enabled: true" mybucket

Using curl

Variables OS_AUTH_TOKEN and OS_STORAGE_URL used below can be retrieved with the command swift auth.

taylor@laptop (pub1|taylor):~$ curl -i -XPUT -H "X-Auth-Token: ${OS_AUTH_TOKEN}" -H "X-Versions-Enabled: true" ${OS_STORAGE_URL}/mybucket
HTTP/1.1 202 Accepted
content-type: text/html; charset=UTF-8
content-length: 76
x-trans-id: tx1165a6d1471645f2b866c-00642c81be
x-openstack-request-id: tx1165a6d1471645f2b866c-00642c81be
date: Tue, 04 Apr 2023 19:59:58 GMT
strict-transport-security: max-age=63072000

<html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>

Validating object versioning is active :

Let's assume you have a local file called myfile1

We upload the first version :

taylor@laptop (pub1|taylor):~$ aws --profile pub1-taylor --endpoint-url=https://s3.pub1.infomaniak.cloud s3api put-object --bucket mybucket --key object1 --body myfile1
{
    "ETag": "\"394e2c3ec5452b1875f45e2116325687\"",
    "VersionId": "1680638455.31246"
}

We modify the file and upload the second version

taylor@laptop (pub1|taylor):~$ touch myfile1
taylor@laptop (pub1|taylor):~$ aws --profile pub1-taylor --endpoint-url=https://s3.pub1.infomaniak.cloud s3api put-object --bucket mybucket --key myfile1 --body myfile1
{
    "ETag": "\"394e2c3ec5452b1875f45e2116325687\"",
    "VersionId": "1680638464.77031"
}

Listing object versions in a bucket

taylor@laptop (pub1|taylor):~$ aws --profile pub1-taylor --endpoint-url=https://s3.pub1.infomaniak.cloud s3api list-object-versions --bucket mybucket
{
    "Versions": [
        {
            "ETag": "\"394e2c3ec5452b1875f45e2116325687\"",
            "Size": 183,
            "StorageClass": "STANDARD",
            "Key": "myfile1",
            "VersionId": "1680638464.77031",
            "IsLatest": true,
            "LastModified": "2023-04-04T20:01:04.770Z",
            "Owner": {
                "DisplayName": "taylor:taylor",
                "ID": "taylor:taylor"
            }
        },
        {
            "ETag": "\"394e2c3ec5452b1875f45e2116325687\"",
            "Size": 183,
            "StorageClass": "STANDARD",
            "Key": "myfile1",
            "VersionId": "1680638455.31246",
            "IsLatest": false,
            "LastModified": "2023-04-04T20:00:55.312Z",
            "Owner": {
                "DisplayName": "taylor:taylor",
                "ID": "taylor:taylor"
            }
        },
    ]
}

Retrieving a specific version of an object

    aws --profile pub1-taylor --endpoint-url=https://s3.pub1.infomaniak.cloud s3api get-object --version-id "1680638455.31246" --bucket mybucket --key myfile1 myfile1

Deleting a specific version of an object

    aws --profile pub1-taylor --endpoint-url=https://s3.pub1.infomaniak.cloud s3api delete-object --version-id "1680638455.31246" --bucket mybucket

Note

Deleting an object without specifying a version will delete the latest version of the object only.

SDK considerations

With some S3 SDKs, for example AWSJavaScriptSDK or AWS SDK for Go, you need to setup option forcePathStyle=true in your code. Otherwise, the SDK will not be able to access available buckets.

Performances

Performances using s3-benchmark and 100 samples for each size of objects from one standard Debian 11 VM, no tuning and flavor a32-ram64-disk20-perf1

Download performance with 1 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       0.1 MB/s |   70    51    58    64    78    88   115   130 |   72    53    60    66    80    90   117   132 |
|       9 |       0.1 MB/s |   54    37    45    51    57    71    74    80 |   55    38    46    53    58    72    75    82 |
|      10 |       0.1 MB/s |   62    46    53    58    69    77    93    96 |   64    47    55    59    71    78    95    98 |
|      11 |       0.2 MB/s |   53    42    44    50    57    68    80    84 |   54    43    45    52    59    69    81    85 |
|      12 |       0.2 MB/s |   55    40    48    53    58    69    75    80 |   56    41    49    54    59    71    76    81 |
|      13 |       0.2 MB/s |   66    36    56    61    75    90    97   102 |   67    37    57    64    76    93   105   105 |
|      14 |       0.2 MB/s |   74    38    65    72    82    96   106   127 |   77    39    68    74    84    98   108   129 |
|      15 |       0.2 MB/s |   67    42    56    63    76    91    99   103 |   70    43    57    66    80    95   104   105 |
|      16 |       0.2 MB/s |   78    39    70    78    88    96   102   103 |   81    39    72    78    92   102   107   109 |
|      17 |       0.3 MB/s |   59    39    50    59    65    74    77    78 |   60    40    51    60    66    75    79    80 |
|      18 |       0.3 MB/s |   61    40    48    56    69    80   104   110 |   62    42    50    57    70    82   105   112 |
|      19 |       0.2 MB/s |   60    39    49    58    70    78    83    86 |   61    40    51    59    71    80    84    89 |
|      20 |       0.3 MB/s |   61    38    48    60    71    86    90    92 |   63    39    49    61    72    87    92    94 |
|      21 |       0.2 MB/s |   69    48    56    70    77    85    92    95 |   70    49    57    71    79    87    95    96 |
|      22 |       0.2 MB/s |   89    38    72    89   108   120   136   152 |   93    39    73    93   115   128   139   154 |
|      23 |       0.3 MB/s |   68    43    54    64    78    92    98   116 |   69    44    55    65    80    94    99   117 |
|      24 |       0.2 MB/s |   90    40    64    90   111   128   154   162 |   93    40    67    91   115   130   156   166 |
|      25 |       0.3 MB/s |   78    39    56    73    88   128   131   137 |   79    39    57    74    90   128   132   138 |
|      26 |       0.3 MB/s |   71    41    58    69    79    97   111   119 |   72    42    59    70    81    98   112   121 |
|      27 |       0.2 MB/s |   86    43    71    79    96   119   129   159 |   87    44    73    80    97   120   131   160 |
|      28 |       0.3 MB/s |   66    38    46    66    81    88    98    98 |   67    39    47    67    83    90   100   101 |
|      29 |       0.3 MB/s |   73    44    57    73    84    93   106   107 |   74    46    58    75    86    94   108   109 |
|      30 |       0.2 MB/s |  129    44   103   128   141   183   219   220 |  133    45   108   132   144   187   222   224 |
|      31 |       0.3 MB/s |   76    36    54    69    83   118   143   145 |   77    37    55    71    84   119   145   146 |
|      32 |       0.3 MB/s |   80    44    63    72    92   123   129   134 |   81    45    64    73    93   124   131   136 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 2 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       0.2 MB/s |   62    41    52    60    67    79    93   111 |   64    42    54    62    69    83    94   113 |
|       9 |       0.3 MB/s |   61    39    53    61    63    75    89    91 |   63    40    56    63    66    77    91    93 |
|      10 |       0.3 MB/s |   53    39    45    50    56    63    86    89 |   54    40    46    51    57    65    87    89 |
|      11 |       0.3 MB/s |   61    37    51    59    68    77    87   102 |   63    38    52    61    71    81    91   104 |
|      12 |       0.4 MB/s |   54    38    47    51    59    68    79    90 |   55    39    48    52    61    69    80    93 |
|      13 |       0.4 MB/s |   53    40    46    51    59    67    70    73 |   55    41    47    52    60    68    72    74 |
|      14 |       0.4 MB/s |   55    37    45    50    58    68    94   100 |   56    38    47    52    60    69    95   101 |
|      15 |       0.4 MB/s |   56    39    47    52    59    73    81    86 |   57    40    49    54    60    74    82    88 |
|      16 |       0.4 MB/s |   59    40    50    57    67    73    85    86 |   60    41    51    58    69    74    86    87 |
|      17 |       0.4 MB/s |   75    36    59    69    85   106   125   136 |   77    38    60    71    88   113   127   137 |
|      18 |       0.5 MB/s |   61    40    50    55    69    79    92   114 |   63    41    52    57    71    81    94   115 |
|      19 |       0.3 MB/s |  100    42    83   100   111   132   151   183 |  105    43    87   105   120   136   153   185 |
|      20 |       0.4 MB/s |   88    37    59    89   109   127   138   163 |   91    38    60    92   113   136   145   165 |
|      21 |       0.5 MB/s |   65    41    53    63    75    80    89   106 |   67    42    54    65    77    82    90   107 |
|      22 |       0.5 MB/s |   74    40    54    72    88   106   117   139 |   77    41    55    73    92   112   122   141 |
|      23 |       0.6 MB/s |   62    38    48    58    70    84    98   104 |   63    39    49    59    71    85    99   106 |
|      24 |       0.6 MB/s |   67    40    52    64    76    92   102   107 |   68    40    53    65    77    94   103   108 |
|      25 |       0.5 MB/s |   75    39    57    65    77   111   192   196 |   76    40    58    66    79   112   193   197 |
|      26 |       0.5 MB/s |   78    39    66    72    82    98   132   150 |   79    40    67    74    83    99   134   151 |
|      27 |       0.7 MB/s |   62    44    52    60    70    78    82   110 |   64    45    53    61    72    81    85   111 |
|      28 |       0.5 MB/s |   90    45    71    83   100   128   147   152 |   92    46    73    84   101   129   148   153 |
|      29 |       0.5 MB/s |   87    57    72    80    97   118   126   131 |   88    58    73    81    98   120   128   132 |
|      30 |       0.5 MB/s |   91    54    70    82   105   128   135   139 |   92    55    72    85   106   129   136   141 |
|      31 |       0.4 MB/s |   80    39    63    73    83   115   127   235 |   81    40    65    74    85   117   128   236 |
|      32 |       0.4 MB/s |  113    38    89   107   134   178   196   197 |  115    39    91   109   138   183   198   201 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 4 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       0.5 MB/s |   55    37    44    51    65    70    82    82 |   56    38    45    52    66    71    83    83 |
|       9 |       0.5 MB/s |   62    43    52    58    67    79   101   101 |   64    44    53    59    69    81   102   104 |
|      10 |       0.7 MB/s |   51    39    45    48    53    59    79    89 |   52    40    46    49    54    61    80    90 |
|      11 |       0.8 MB/s |   51    38    44    49    54    69    78    80 |   52    39    45    50    55    70    80    81 |
|      12 |       0.7 MB/s |   60    36    51    59    66    77    92    95 |   62    37    53    60    67    80    94    95 |
|      13 |       0.8 MB/s |   53    39    48    53    57    61    65    67 |   54    41    49    54    58    62    66    68 |
|      14 |       0.7 MB/s |   70    37    63    72    79    87    92   104 |   72    38    65    74    82    90   100   109 |
|      15 |       0.8 MB/s |   61    40    49    53    66    91   107   113 |   62    41    50    54    67    93   109   113 |
|      16 |       0.9 MB/s |   58    40    47    57    65    71    79    81 |   59    41    48    58    66    72    80    82 |
|      17 |       1.0 MB/s |   57    41    48    53    62    70    91    93 |   58    42    49    54    63    72    93    94 |
|      18 |       1.0 MB/s |   61    43    49    55    72    78    96    99 |   62    45    50    56    73    80    97   100 |
|      19 |       1.0 MB/s |   58    46    52    57    64    71    74    74 |   60    47    53    59    65    73    76    76 |
|      20 |       1.0 MB/s |   60    38    48    56    66    76   104   107 |   61    39    50    58    67    77   104   108 |
|      21 |       1.2 MB/s |   60    39    46    59    70    80    93    96 |   62    40    48    60    71    82    94    97 |
|      22 |       1.1 MB/s |   62    42    49    55    69    79   109   127 |   64    43    50    57    70    81   110   129 |
|      23 |       1.0 MB/s |   72    38    57    66    81    93   130   134 |   73    39    58    67    82    95   132   136 |
|      24 |       0.9 MB/s |   78    37    54    79    95   111   131   132 |   81    38    55    81   102   116   135   136 |
|      25 |       1.3 MB/s |   63    41    52    63    69    78    87    88 |   64    42    54    64    71    80    89    91 |
|      26 |       1.0 MB/s |   75    39    63    74    81    91   118   131 |   76    40    64    76    83    92   119   133 |
|      27 |       0.9 MB/s |   88    51    68    79   102   123   136   140 |   89    52    70    80   103   124   138   142 |
|      28 |       0.7 MB/s |  113    39    91   114   132   163   189   193 |  116    39    92   119   135   168   194   195 |
|      29 |       1.1 MB/s |   80    39    65    74    84   120   126   136 |   81    40    66    75    85   121   128   138 |
|      30 |       1.1 MB/s |   79    41    61    69    93   120   132   176 |   80    42    62    71    95   122   133   177 |
|      31 |       1.2 MB/s |   74    38    45    72    92   105   120   122 |   77    39    46    73    96   112   122   126 |
|      32 |       1.2 MB/s |   83    40    70    79    88   118   138   144 |   85    41    71    80    90   119   139   145 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 8 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       1.0 MB/s |   59    42    47    55    65    82    91    93 |   60    43    48    56    66    83    92    94 |
|       9 |       1.2 MB/s |   52    38    46    49    57    63    65    74 |   53    39    47    50    58    64    66    75 |
|      10 |       1.2 MB/s |   60    39    49    58    68    79    82    84 |   62    40    50    60    70    81    85    86 |
|      11 |       1.5 MB/s |   51    37    44    48    54    67    74    79 |   52    37    45    49    55    68    76    81 |
|      12 |       1.3 MB/s |   63    40    56    64    68    77    84    91 |   65    41    58    67    71    81    87    94 |
|      13 |       1.8 MB/s |   51    39    45    50    55    62    70    71 |   53    40    46    51    56    63    71    72 |
|      14 |       1.8 MB/s |   54    40    46    52    59    69    75    76 |   55    41    47    53    60    71    76    77 |
|      15 |       1.4 MB/s |   67    41    57    65    74    86    99   112 |   70    42    59    67    76    92   107   113 |
|      16 |       1.9 MB/s |   56    39    48    51    60    75    83    86 |   57    40    49    52    61    77    84    87 |
|      17 |       1.8 MB/s |   61    45    52    59    65    75    81   102 |   62    46    53    60    66    76    82   103 |
|      18 |       2.0 MB/s |   60    41    51    60    66    71    75    76 |   61    42    52    61    67    73    78    80 |
|      19 |       2.1 MB/s |   62    42    49    61    70    82    91    95 |   63    43    51    63    72    83    92    96 |
|      20 |       1.8 MB/s |   61    39    49    55    68    79   117   125 |   63    40    50    56    69    80   118   126 |
|      21 |       2.2 MB/s |   63    39    48    55    79    87   103   103 |   64    40    49    56    80    89   104   104 |
|      22 |       2.1 MB/s |   65    42    53    65    73    85    89    91 |   67    43    54    66    75    86    90    92 |
|      23 |       2.1 MB/s |   71    52    58    65    78    88   119   120 |   72    53    59    66    79    90   121   121 |
|      24 |       1.5 MB/s |   97    40    82    96   105   128   175   202 |  101    41    89   100   112   133   177   211 |
|      25 |       2.3 MB/s |   71    41    59    65    74    98   117   124 |   72    42    60    66    76    99   118   125 |
|      26 |       1.6 MB/s |  100    42    80   100   114   137   156   171 |  103    43    84   100   117   146   165   175 |
|      27 |       2.2 MB/s |   73    42    57    69    77   107   125   131 |   74    44    58    71    78   109   127   133 |
|      28 |       2.0 MB/s |   81    41    63    75    90   123   142   143 |   82    41    64    76    91   123   144   144 |
|      29 |       2.0 MB/s |   85    45    71    81    91   118   140   151 |   86    46    72    83    92   118   141   153 |
|      30 |       0.5 MB/s |  121    57    77    88   102   136   753   811 |  122    58    79    89   104   137   754   812 |
|      31 |       2.6 MB/s |   73    48    60    69    80    91   127   128 |   74    49    62    71    82    93   128   129 |
|      32 |       2.0 MB/s |   92    54    72    82   107   129   137   140 |   93    54    74    83   108   130   138   142 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 16 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       2.1 MB/s |   51    39    43    48    56    65    74    76 |   52    40    44    49    58    66    75    77 |
|       9 |       2.6 MB/s |   49    37    43    48    52    58    70    71 |   50    37    44    49    52    59    71    72 |
|      10 |       2.9 MB/s |   49    36    43    47    53    58    64    68 |   50    37    44    48    53    59    65    68 |
|      11 |       3.1 MB/s |   49    39    43    48    53    60    63    65 |   50    40    44    50    54    61    64    66 |
|      12 |       3.2 MB/s |   53    39    45    51    58    65    69    72 |   54    40    46    52    60    66    70    73 |
|      13 |       2.7 MB/s |   64    39    54    61    68    89    99   102 |   65    40    55    62    69    90   101   104 |
|      14 |       3.4 MB/s |   55    41    47    52    58    72    79    80 |   56    42    48    53    59    72    81    81 |
|      15 |       2.9 MB/s |   71    37    63    73    81    94    99   100 |   73    38    66    75    83    96   103   116 |
|      16 |       3.8 MB/s |   57    40    51    56    63    67    76    84 |   58    41    51    58    64    68    76    85 |
|      17 |       3.8 MB/s |   60    41    48    53    63    88   110   112 |   61    42    49    54    64    89   111   113 |
|      18 |       4.1 MB/s |   57    40    46    50    65    84    95   100 |   58    41    48    52    65    84    96   100 |
|      19 |       4.6 MB/s |   53    40    44    50    57    72    80    85 |   54    41    45    51    58    73    81    86 |
|      20 |       3.1 MB/s |   83    41    64    81    93   121   133   135 |   85    42    65    84    97   123   135   142 |
|      21 |       4.1 MB/s |   61    44    51    56    69    77    88    90 |   62    45    52    57    70    79    89    91 |
|      22 |       4.5 MB/s |   63    41    48    59    73    86    98   101 |   64    42    50    60    74    88    99   102 |
|      23 |       4.4 MB/s |   67    45    55    63    73    81   108   134 |   68    45    57    64    74    83   109   134 |
|      24 |       3.6 MB/s |   78    56    64    74    84   107   122   127 |   79    56    66    74    85   108   123   128 |
|      25 |       4.5 MB/s |   71    37    61    65    80    87   110   113 |   72    38    62    66    81    89   111   114 |
|      26 |       3.5 MB/s |   93    37    72    87   106   138   189   200 |   95    38    76    89   107   144   194   200 |
|      27 |       4.5 MB/s |   76    40    62    70    83   112   124   127 |   77    41    63    71    84   113   125   127 |
|      28 |       5.1 MB/s |   64    41    48    64    76    87    94    98 |   65    42    49    65    77    88    95    99 |
|      29 |       4.1 MB/s |   88    48    71    80    94   127   141   142 |   89    48    72    82    95   128   142   143 |
|      30 |       4.7 MB/s |   72    42    56    68    74   107   129   143 |   73    43    57    69    75   108   130   144 |
|      31 |       4.2 MB/s |   86    42    66    78    96   130   143   161 |   87    43    67    79    97   131   145   162 |
|      32 |       5.4 MB/s |   71    40    55    69    85    91    99   100 |   72    41    55    70    86    92   100   101 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 32 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       4.1 MB/s |   56    40    46    51    65    73    76    81 |   57    41    47    52    65    74    78    82 |
|       9 |       4.8 MB/s |   53    40    46    49    56    67    88    89 |   54    41    47    50    58    69    89    90 |
|      10 |       4.9 MB/s |   58    40    49    56    62    77    83    85 |   59    41    50    58    64    78    84    87 |
|      11 |       6.2 MB/s |   50    40    45    48    54    59    66    73 |   51    40    46    49    55    60    67    73 |
|      12 |       6.2 MB/s |   53    38    46    52    58    67    73    77 |   54    39    47    52    59    68    74    77 |
|      13 |       6.5 MB/s |   54    39    48    51    58    66    75    77 |   55    39    48    52    59    67    76    77 |
|      14 |       6.5 MB/s |   57    41    45    55    62    72    89    90 |   58    42    46    56    63    74    90    90 |
|      15 |       7.6 MB/s |   53    39    44    50    57    68    82    82 |   54    40    45    50    58    69    82    82 |
|      16 |       7.7 MB/s |   54    37    45    50    60    73    88    90 |   55    38    47    51    61    73    90    91 |
|      17 |       5.9 MB/s |   73    37    57    70    85   100   111   115 |   75    38    58    72    90   107   117   118 |
|      18 |       8.2 MB/s |   57    38    45    53    63    74    82    91 |   57    38    45    54    65    75    83    92 |
|      19 |       8.0 MB/s |   61    41    53    63    68    73    86    94 |   63    42    54    64    69    76    87    95 |
|      20 |       8.2 MB/s |   62    39    50    60    73    81    90    99 |   63    40    51    62    75    83    91   101 |
|      21 |       6.3 MB/s |   82    42    68    82    95   107   110   115 |   86    43    70    86   100   114   120   125 |
|      22 |       5.6 MB/s |   98    39    81    95   115   123   170   175 |  102    39    83    99   119   131   172   177 |
|      23 |       7.5 MB/s |   78    54    65    73    85   102   127   129 |   79    55    65    74    86   102   128   130 |
|      24 |       8.5 MB/s |   71    52    61    65    79    90   106   113 |   72    53    62    66    80    91   107   114 |
|      25 |       5.8 MB/s |  104    44    89   102   112   140   183   214 |  109    44    94   106   119   146   186   218 |
|      26 |       6.4 MB/s |   89    39    55    92   105   120   191   202 |   92    40    55    95   109   123   194   203 |
|      27 |       7.5 MB/s |   84    36    68    78    99   121   127   144 |   85    37    69    80   100   122   128   145 |
|      28 |      10.8 MB/s |   67    43    51    66    76    92   109   112 |   68    43    52    67    77    93   110   114 |
|      29 |       8.1 MB/s |   89    47    74    86    93   127   149   154 |   90    48    75    87    94   128   150   154 |
|      30 |       7.8 MB/s |   81    44    65    71    92   122   132   132 |   82    45    66    72    93   123   133   134 |
|      31 |       8.3 MB/s |   85    41    66    75    97   128   148   149 |   86    41    67    76    97   129   149   149 |
|      32 |       5.9 MB/s |  122    40    97   122   137   186   210   220 |  125    41   101   124   141   187   212   222 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 64 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |       7.9 MB/s |   58    41    47    53    65    79    91   100 |   59    42    49    54    66    79    93   101 |
|       9 |       7.6 MB/s |   65    45    58    64    70    80    87    88 |   67    45    60    66    72    84    90    90 |
|      10 |      10.0 MB/s |   54    39    47    53    60    66    74    79 |   55    40    48    54    61    68    75    80 |
|      11 |      11.7 MB/s |   53    42    46    50    57    63    76    78 |   54    42    47    51    58    64    77    79 |
|      12 |      10.5 MB/s |   64    38    57    60    69    84    93    94 |   66    38    58    62    72    86    96    98 |
|      13 |      13.3 MB/s |   54    41    46    53    61    65    68    76 |   55    42    47    53    62    67    69    78 |
|      14 |      14.1 MB/s |   54    40    49    52    57    64    66    73 |   55    41    50    53    58    66    67    74 |
|      15 |      13.5 MB/s |   60    41    49    57    67    75    93    99 |   61    42    50    58    68    76    94   101 |
|      16 |      12.1 MB/s |   70    40    53    65    79    98   113   130 |   72    41    53    66    81   100   119   131 |
|      17 |      16.5 MB/s |   58    42    49    55    65    76    87    87 |   59    43    51    55    66    77    88    89 |
|      18 |      16.8 MB/s |   58    38    49    54    62    71    98    99 |   59    39    50    55    63    72    99   100 |
|      19 |      16.4 MB/s |   64    41    54    61    74    83    90    97 |   65    42    55    63    75    84    91    98 |
|      20 |      10.7 MB/s |   94    47    77    89   116   125   144   154 |   97    47    78    93   122   132   148   160 |
|      21 |      16.8 MB/s |   64    42    50    62    72    82    95    98 |   65    43    51    65    73    82    96    99 |
|      22 |      17.0 MB/s |   64    38    53    66    72    82    88    89 |   65    39    54    66    74    83    90    91 |
|      23 |      11.6 MB/s |   94    42    69   100   108   127   146   160 |   97    43    70   101   113   138   155   160 |
|      24 |      16.4 MB/s |   75    37    63    75    82    92   124   125 |   76    38    64    77    83    94   125   127 |
|      25 |      16.6 MB/s |   69    34    57    65    81   103   110   114 |   70    35    58    65    82   104   111   115 |
|      26 |      12.1 MB/s |  105    47    84   106   119   156   186   190 |  110    48    89   108   125   158   191   193 |
|      27 |      11.0 MB/s |  114    45    92   110   129   171   210   215 |  119    46    99   117   133   173   217   218 |
|      28 |       6.5 MB/s |   90    49    70    80    89   118   136   401 |   91    51    71    81    89   119   137   401 |
|      29 |      15.5 MB/s |   89    54    74    82    98   126   136   150 |   91    55    75    83    99   127   137   151 |
|      30 |      16.6 MB/s |   87    53    71    79    95   128   138   139 |   88    54    72    80    95   129   139   140 |
|      31 |      16.4 MB/s |   90    45    72    85    96   134   150   162 |   91    46    73    86    97   134   151   162 |
|      32 |      21.1 MB/s |   72    40    51    69    84   102   111   111 |   73    42    52    70    85   104   112   113 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 128 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |      15.0 MB/s |   61    45    49    56    70    80    83    97 |   62    46    51    57    72    81    85    98 |
|       9 |      19.8 MB/s |   52    40    43    47    58    72    78    88 |   54    41    45    49    60    74    79    89 |
|      10 |      21.9 MB/s |   51    40    46    49    53    62    68    69 |   52    41    47    51    54    64    70    70 |
|      11 |      18.5 MB/s |   65    41    55    65    70    80   103   112 |   68    42    56    68    73    82   105   114 |
|      12 |      21.5 MB/s |   58    39    48    54    63    74    84    93 |   59    40    50    56    64    75    85    94 |
|      13 |      25.8 MB/s |   56    40    48    54    62    69    72    74 |   57    41    49    56    63    70    73    75 |
|      14 |      20.6 MB/s |   73    38    61    72    82    91   108   115 |   76    40    63    75    86   100   115   120 |
|      15 |      28.5 MB/s |   56    41    50    53    60    69    75    78 |   57    43    50    55    62    71    78    80 |
|      16 |      29.0 MB/s |   61    40    48    59    72    78    97   108 |   63    42    49    60    73    80    99   109 |
|      17 |      29.6 MB/s |   61    40    51    56    70    77   105   106 |   63    41    52    58    72    79   106   107 |
|      18 |      24.7 MB/s |   75    42    58    72    84   106   112   115 |   79    43    59    75    87   114   120   121 |
|      19 |      32.0 MB/s |   63    40    51    56    72    86    96    97 |   64    41    52    59    73    87    97    99 |
|      20 |      24.5 MB/s |   85    40    70    82    97   111   139   145 |   88    41    72    86   100   124   148   149 |
|      21 |      29.4 MB/s |   67    46    53    68    75    84    97   100 |   69    48    55    69    76    86    98   101 |
|      22 |      34.1 MB/s |   65    40    55    61    73    87    97    98 |   66    41    56    62    75    90    99   100 |
|      23 |      34.4 MB/s |   68    46    55    66    77    85   108   129 |   70    47    56    67    79    88   109   132 |
|      24 |      32.4 MB/s |   74    48    59    69    78   103   121   135 |   76    50    61    71    80   104   123   137 |
|      25 |      25.0 MB/s |  102    44    79   100   118   146   168   175 |  106    46    82   104   130   151   174   179 |
|      26 |      35.1 MB/s |   75    41    61    73    87    94   121   133 |   77    42    63    75    88    96   123   135 |
|      27 |      32.9 MB/s |   79    57    67    73    82   102   134   135 |   81    58    68    74    84   103   135   138 |
|      28 |      32.5 MB/s |   86    40    65    77   100   128   136   143 |   87    41    67    78   102   129   137   145 |
|      29 |      24.5 MB/s |  110    42    93   109   128   145   196   201 |  115    43    95   112   133   155   209   214 |
|      30 |      29.2 MB/s |   90    44    72    82   100   134   139   160 |   92    46    74    84   103   136   141   161 |
|      31 |      32.8 MB/s |   89    39    71    80   100   129   140   148 |   91    41    73    83   101   130   141   150 |
|      32 |      41.8 MB/s |   74    45    57    76    85    88   136   139 |   76    46    59    78    86    91   138   141 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 256 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |      31.3 MB/s |   57    41    50    53    62    70    82    92 |   60    44    53    57    65    74    85    95 |
|       9 |      36.9 MB/s |   52    44    48    52    55    59    67    67 |   55    46    51    54    58    62    69    69 |
|      10 |      38.8 MB/s |   55    44    50    52    56    66    81    83 |   58    48    53    56    60    69    84    87 |
|      11 |      44.8 MB/s |   54    40    48    52    58    64    70    74 |   57    43    51    55    62    67    74    77 |
|      12 |      47.3 MB/s |   55    43    49    54    60    66    71    78 |   58    45    52    57    63    69    74    82 |
|      13 |      50.6 MB/s |   56    42    50    54    58    63    83    93 |   59    44    53    58    62    66    86    96 |
|      14 |      43.1 MB/s |   68    41    51    63    77    90   127   146 |   72    43    54    66    81    93   130   151 |
|      15 |      38.2 MB/s |   83    58    71    77    88   111   122   123 |   87    61    75    83    92   116   124   127 |
|      16 |      56.7 MB/s |   61    43    52    56    70    78    81    83 |   64    45    54    60    74    81    83    86 |
|      17 |      46.3 MB/s |   79    46    64    74    90   111   124   127 |   83    48    67    77    94   114   129   130 |
|      18 |      58.1 MB/s |   64    43    53    60    70    79   111   116 |   67    45    56    62    74    81   115   121 |
|      19 |      62.1 MB/s |   62    44    52    60    66    81    88    98 |   65    45    55    63    70    84    93   101 |
|      20 |      63.0 MB/s |   64    42    54    61    72    81    90    92 |   67    46    57    65    78    84    94    94 |
|      21 |      53.4 MB/s |   73    50    61    68    79    89   121   133 |   76    52    65    72    82    92   125   136 |
|      22 |      50.1 MB/s |   88    39    60    87   105   124   158   178 |   93    41    63    93   114   129   161   182 |
|      23 |      58.5 MB/s |   71    49    59    68    80    87    95   130 |   75    52    61    72    83    93    98   133 |
|      24 |      46.2 MB/s |   97    38    77    92   110   131   191   198 |  104    41    82   103   122   134   193   202 |
|      25 |      64.6 MB/s |   74    42    61    72    81    92   122   128 |   77    44    63    74    84    95   126   131 |
|      26 |      72.1 MB/s |   69    43    54    66    79    96   106   109 |   72    45    57    69    82    99   112   113 |
|      27 |      62.2 MB/s |   83    50    64    77    87   132   139   142 |   86    53    67    80    92   135   144   145 |
|      28 |      57.6 MB/s |   90    51    74    83   102   126   150   156 |   93    53    77    88   104   129   153   158 |
|      29 |      65.9 MB/s |   82    39    70    75    89   124   135   139 |   86    42    73    80    92   128   137   143 |
|      30 |      61.2 MB/s |   89    46    73    81    99   133   141   143 |   92    49    75    84   102   137   144   146 |
|      31 |      61.0 MB/s |   93    47    72    84   105   134   153   157 |   96    50    77    87   107   138   159   161 |
|      32 |      62.9 MB/s |   92    46    74    80   105   134   139   141 |   96    48    78    84   108   138   145   146 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 512 KB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |      51.6 MB/s |   67    45    52    62    75    93   105   108 |   73    51    58    69    82    98   113   114 |
|       9 |      64.8 MB/s |   59    43    51    53    64    74    99    99 |   65    50    56    60    70    83   104   105 |
|      10 |      76.4 MB/s |   55    40    47    51    55    69    98   101 |   60    46    53    57    61    74   104   107 |
|      11 |      82.1 MB/s |   56    45    49    54    58    67    78    80 |   61    51    55    60    63    73    83    84 |
|      12 |      82.4 MB/s |   60    44    52    58    65    67    77   120 |   66    51    58    64    71    74    82   132 |
|      13 |      94.3 MB/s |   58    45    52    58    61    67    71    72 |   64    51    59    64    67    73    77    77 |
|      14 |      97.9 MB/s |   58    41    49    55    63    71    91    94 |   64    47    55    60    69    78    95   100 |
|      15 |      97.8 MB/s |   58    42    48    52    64    77    84    86 |   64    48    54    59    71    84    88    90 |
|      16 |     102.7 MB/s |   61    44    51    55    68    78    88    90 |   67    48    57    60    76    85    94    97 |
|      17 |     112.0 MB/s |   63    41    52    58    72    84    89    91 |   69    45    59    65    77    88    95    98 |
|      18 |     116.6 MB/s |   62    41    53    62    69    74    78    86 |   67    44    60    68    75    79    81    92 |
|      19 |     117.3 MB/s |   61    43    53    59    68    76    87    87 |   68    48    62    66    75    83    91    91 |
|      20 |     103.8 MB/s |   71    47    62    69    77    82   100   116 |   77    52    68    75    83    88   104   120 |
|      21 |     118.0 MB/s |   66    44    53    59    78    86    99   104 |   73    52    60    68    87    95   101   109 |
|      22 |      91.0 MB/s |   91    43    77    96   106   117   137   177 |  102    52    81   108   121   129   143   189 |
|      23 |     132.6 MB/s |   62    40    52    56    72    82    89    90 |   68    46    57    62    81    88    94    98 |
|      24 |     109.2 MB/s |   80    53    68    76    89   101   126   131 |   87    58    74    85    94   105   135   139 |
|      25 |     116.6 MB/s |   78    54    65    73    78   104   139   140 |   85    59    71    80    90   108   148   154 |
|      26 |     108.4 MB/s |   83    42    67    80    95   113   135   141 |   89    47    72    85   103   119   139   153 |
|      27 |     117.9 MB/s |   85    44    64    76    90   138   155   156 |   91    49    69    82    95   148   160   161 |
|      28 |      88.7 MB/s |  112    45    92   109   127   182   203   214 |  123    50    98   122   137   189   217   222 |
|      29 |     101.0 MB/s |   85    46    66    74    99   136   147   149 |   93    51    73    85   104   147   153   155 |
|      30 |     148.7 MB/s |   70    44    57    71    81    88   100   101 |   78    49    63    78    91    97   106   107 |
|      31 |     118.6 MB/s |   91    53    69    83   104   139   153   155 |   98    61    76    88   109   148   159   161 |
|      32 |     115.8 MB/s |   91    43    70    82   100   134   143   166 |   98    50    77    89   107   143   149   172 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 1 MB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |      98.2 MB/s |   65    46    54    60    73    84    96   115 |   77    57    64    70    85   103   121   127 |
|       9 |     106.1 MB/s |   69    45    62    68    75    80    87    96 |   80    52    71    79    85    94   106   111 |
|      10 |     129.4 MB/s |   57    40    50    55    62    69    80    99 |   68    51    61    66    74    78    93   108 |
|      11 |     118.2 MB/s |   71    45    64    68    73    84   103   105 |   83    57    76    80    85    98   111   122 |
|      12 |     151.1 MB/s |   60    45    53    56    67    70    87    96 |   70    51    63    66    77    82    93   108 |
|      13 |     162.7 MB/s |   58    44    51    57    64    68    81    82 |   69    53    61    69    75    79    88    90 |
|      14 |     178.1 MB/s |   60    44    53    58    65    70    82    82 |   72    54    65    70    78    81    93    93 |
|      15 |     176.0 MB/s |   62    45    52    57    71    76    92   107 |   73    50    64    70    82    88    99   118 |
|      16 |     191.0 MB/s |   63    47    55    61    68    75    81    84 |   74    59    66    74    80    87    92    92 |
|      17 |     174.4 MB/s |   74    45    64    73    81    92   108   110 |   89    52    78    87    98   112   132   135 |
|      18 |     208.9 MB/s |   65    43    53    59    76    83    93    95 |   76    52    65    70    88    96   106   107 |
|      19 |     159.7 MB/s |   88    45    74    90   102   122   137   138 |  103    53    83   107   120   141   150   153 |
|      20 |     212.8 MB/s |   66    47    56    64    74    84    92   110 |   79    54    67    73    89   102   109   116 |
|      21 |     220.0 MB/s |   66    45    55    63    71    83    97   100 |   78    54    67    78    85    93   107   111 |
|      22 |     151.1 MB/s |   99    40    84    96   114   132   157   192 |  117    50   103   119   137   159   189   203 |
|      23 |     228.8 MB/s |   70    44    51    60    78    90   134   144 |   82    54    64    69    94   106   149   158 |
|      24 |     232.5 MB/s |   68    40    57    65    73    91   102   124 |   81    49    68    81    92   102   110   131 |
|      25 |     154.2 MB/s |  109    43    89   105   128   157   212   218 |  128    52   106   129   148   172   228   239 |
|      26 |     222.4 MB/s |   80    51    60    75    91   104   145   150 |   91    57    73    87   104   115   156   167 |
|      27 |     238.4 MB/s |   77    38    62    72    85   108   135   136 |   89    44    74    84    99   126   148   153 |
|      28 |     169.5 MB/s |  105    42    63   100   129   144   203   233 |  122    49    73   122   151   182   215   252 |
|      29 |     221.9 MB/s |   90    52    70    79   102   145   161   161 |  104    65    82    95   113   161   176   181 |
|      30 |     150.9 MB/s |  132    47   107   119   150   224   242   244 |  155    56   132   148   164   251   284   293 |
|      31 |     228.5 MB/s |   92    46    71    84   100   143   165   176 |  104    53    84    96   109   154   177   190 |
|      32 |     200.4 MB/s |  102    52    81    90   111   150   158   168 |  113    61    92   106   120   162   174   175 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 2 MB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |     179.0 MB/s |   64    47    53    62    71    77    88    88 |   83    64    71    83    92   102   112   124 |
|       9 |     222.6 MB/s |   56    41    49    52    61    69    79    84 |   76    61    68    73    78    89    98   152 |
|      10 |     217.4 MB/s |   68    42    59    67    75    81   103   110 |   88    52    77    86    97   106   120   135 |
|      11 |     217.8 MB/s |   70    42    62    69    76    87    99   109 |   91    51    80    90   102   117   125   131 |
|      12 |     213.2 MB/s |   81    64    70    76    86   103   113   115 |  105    81    91    98   116   126   132   152 |
|      13 |     279.2 MB/s |   65    39    53    64    72    84    90   103 |   84    55    72    82    92   103   123   129 |
|      14 |     319.3 MB/s |   59    45    48    58    63    74    81    81 |   80    56    71    76    85   102   112   129 |
|      15 |     319.0 MB/s |   60    44    53    58    62    73    85    88 |   81    61    72    75    87    97   106   115 |
|      16 |     342.8 MB/s |   61    43    52    59    68    78    82    94 |   81    55    73    79    89    95   107   110 |
|      17 |     364.1 MB/s |   63    48    55    61    69    78    88    90 |   85    62    75    85    91   101   106   106 |
|      18 |     241.2 MB/s |   83    41    58    83    97   125   138   149 |  108    52    76   103   129   157   171   172 |
|      19 |     397.0 MB/s |   64    46    54    61    69    80    95   107 |   83    59    70    79    91   105   123   123 |
|      20 |     414.8 MB/s |   62    44    53    60    69    80    91   104 |   82    57    70    77    92   107   118   123 |
|      21 |     270.2 MB/s |  104    46    83    99   128   153   174   185 |  138    64   107   134   171   200   219   228 |
|      22 |     378.2 MB/s |   74    48    60    72    79    92   104   143 |   94    63    78    94   106   115   119   168 |
|      23 |     434.1 MB/s |   64    44    52    60    70    81   120   125 |   85    56    71    80    92   107   139   149 |
|      24 |     379.2 MB/s |   75    41    58    66    82   119   139   139 |   98    57    80    92   109   134   158   177 |
|      25 |     266.0 MB/s |  108    38    82   106   123   146   233   235 |  146    51   111   134   178   206   281   285 |
|      26 |     370.5 MB/s |   85    45    67    77    92   122   154   159 |  109    62    93   105   119   142   181   195 |
|      27 |     406.8 MB/s |   84    43    66    74    96   119   144   157 |  105    54    83    97   113   136   177   184 |
|      28 |     489.2 MB/s |   71    47    58    68    82    93    97    99 |   95    60    75    93   111   123   135   144 |
|      29 |     336.3 MB/s |   92    44    69    77   105   143   167   170 |  115    73    90   104   127   165   183   188 |
|      30 |     384.9 MB/s |   99    49    72    88   112   149   184   189 |  120    66    94   113   134   168   202   214 |
|      31 |     402.2 MB/s |   96    52    69    83   107   163   174   178 |  120    74    95   106   123   190   205   207 |
|      32 |     412.3 MB/s |   94    38    68    84   109   150   161   164 |  117    53    95   103   127   184   194   200 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 4 MB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |     281.0 MB/s |   62    45    50    56    71    79    91    92 |  101    70    77    86   108   131   268   281 |
|       9 |     384.8 MB/s |   55    45    50    54    56    67    77    80 |   86    68    78    82    93   100   110   146 |
|      10 |     399.9 MB/s |   54    44    49    54    56    63    70    84 |   90    67    80    85    97   102   115   207 |
|      11 |     453.6 MB/s |   57    44    50    54    59    68    78    91 |   89    69    79    84    91   108   125   128 |
|      12 |     425.4 MB/s |   60    46    50    56    65    80    95    98 |   96    62    82    91   111   120   123   166 |
|      13 |     502.0 MB/s |   61    42    53    57    65    70    94    96 |   95    59    82    90    97   120   148   165 |
|      14 |     520.7 MB/s |   62    44    53    60    67    74    88    91 |   97    70    85    95   102   119   125   136 |
|      15 |     439.9 MB/s |   74    45    64    72    83    93    97   115 |  122    72    96   109   141   184   200   210 |
|      16 |     546.6 MB/s |   63    46    53    60    68    78    94    97 |  102    75    88    98   107   124   148   149 |
|      17 |     451.8 MB/s |   84    46    68    79    93   121   140   163 |  130    69   101   122   149   180   233   241 |
|      18 |     608.5 MB/s |   64    47    54    62    67    82    99   100 |  103    74    87    95   110   134   164   166 |
|      19 |     629.7 MB/s |   64    45    55    61    67    77   102   122 |  105    68    90    98   118   142   162   178 |
|      20 |     675.3 MB/s |   65    42    56    64    72    83    95   102 |  102    71    86    99   111   131   142   145 |
|      21 |     651.6 MB/s |   70    45    57    67    78    88   104   158 |  110    75    89   107   127   140   149   186 |
|      22 |     621.0 MB/s |   80    46    59    68    81   111   180   278 |  119    80    93   104   124   152   210   322 |
|      23 |     604.0 MB/s |   84    52    62    73    93   110   173   184 |  123    83    97   114   139   149   222   232 |
|      24 |     668.8 MB/s |   80    41    63    72    87    96   167   177 |  121    70   100   117   135   150   197   240 |
|      25 |     463.9 MB/s |  105    42    76    95   110   174   259   270 |  185    66   123   192   226   266   383   386 |
|      26 |     651.4 MB/s |   96    60    71    88   106   154   168   175 |  133    86   112   126   140   180   211   222 |
|      27 |     679.6 MB/s |   82    39    61    72    90   140   163   180 |  128    65   104   123   141   163   211   233 |
|      28 |     430.7 MB/s |  131    43    58   121   156   216   315   325 |  206    69    92   212   260   317   415   423 |
|      29 |     720.5 MB/s |   87    44    59    74    90   148   172   195 |  128    75    96   115   150   191   221   247 |
|      30 |     657.1 MB/s |   90    42    63    77   101   163   173   185 |  135    72   101   127   148   210   239   249 |
|      31 |     438.9 MB/s |  134    46    80   104   149   269   325   326 |  228    68   138   217   306   368   453   456 |
|      32 |     717.2 MB/s |   92    41    62    76   106   161   173   185 |  137    70   105   127   154   212   221   252 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 8 MB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |     419.3 MB/s |   73    43    55    61    71    80   154   481 |  146    89   105   122   142   169   437   786 |
|       9 |     514.9 MB/s |   59    46    53    58    65    70    76    77 |  131   100   113   127   148   163   178   180 |
|      10 |     518.1 MB/s |   64    44    54    61    71    80    92   101 |  142   105   121   133   145   183   243   248 |
|      11 |     635.0 MB/s |   57    40    50    55    60    70    85    85 |  125    92   108   122   137   152   166   174 |
|      12 |     596.4 MB/s |   62    45    53    58    61    71   118   132 |  138    97   115   139   152   170   179   199 |
|      13 |     326.1 MB/s |   59    45    52    57    65    72    76    77 |  144    96   110   124   144   157   257   812 |
|      14 |     760.3 MB/s |   60    41    51    57    63    70    86   158 |  130    85   105   123   144   172   189   217 |
|      15 |     733.8 MB/s |   64    50    57    61    70    80    87    88 |  145   101   119   128   150   187   226   392 |
|      16 |     807.1 MB/s |   59    44    51    61    64    70    78    79 |  137    91   128   134   150   159   168   171 |
|      17 |     620.6 MB/s |   64    45    56    61    67    78    89   168 |  148    93   114   131   161   203   342   348 |
|      18 |     696.2 MB/s |   75    46    52    76    89   107   111   135 |  188    95   121   178   244   281   313   320 |
|      19 |     488.4 MB/s |   93    41    75    91   110   128   148   175 |  250    79   173   216   284   341   753   819 |
|      20 |     840.3 MB/s |   65    52    57    63    72    78    86    87 |  169   106   139   164   195   221   233   238 |
|      21 |     758.1 MB/s |   71    47    55    64    82    93   124   145 |  169    93   144   175   190   206   214   304 |
|      22 |     618.3 MB/s |   97    42    62    96   108   147   214   224 |  247    88   151   242   302   405   458   517 |
|      23 |     823.2 MB/s |   74    52    61    70    77    91   122   188 |  171   109   149   163   180   194   267   486 |
|      24 |     546.3 MB/s |  122    46    80   111   143   174   316   323 |  303    90   189   275   362   519   616   665 |
|      25 |     858.0 MB/s |   85    47    62    72    83   143   197   228 |  178    92   133   161   203   248   359   367 |
|      26 |     557.9 MB/s |  139    45    77   127   153   298   363   393 |  324    89   230   299   377   607   639   665 |
|      27 |     933.0 MB/s |   82    47    59    69    83   126   197   200 |  179   113   141   157   194   262   342   351 |
|      28 |     987.3 MB/s |   99    46    67    77    91   200   215   223 |  197   113   154   184   209   298   359   368 |
|      29 |     832.9 MB/s |  110    46    72    87   116   208   232   243 |  220   143   175   194   232   326   396   476 |
|      30 |     939.3 MB/s |   92    49    67    76    93   181   190   202 |  193   118   150   166   222   282   310   318 |
|      31 |     381.3 MB/s |  154    42    87   119   147   374   396   406 |  356    85   253   364   449   596   673   676 |
|      32 |     757.2 MB/s |  111    49    68    81   138   203   227   278 |  218   131   170   194   231   326   372   382 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 16 MB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |     487.2 MB/s |   91    48    54    65    75    85   601   712 |  251   132   147   168   208   460  1014  1037 |
|       9 |     708.5 MB/s |   62    46    54    58    65    71   116   116 |  189   137   149   168   205   232   385   406 |
|      10 |     688.5 MB/s |   62    43    52    59    65    76    95   156 |  197   139   160   185   203   262   306   549 |
|      11 |     430.0 MB/s |   74    46    58    71    84   100   130   138 |  275   146   186   235   330   366   694   839 |
|      12 |     572.2 MB/s |   89    46    73    84   100   115   134   161 |  308   158   234   295   366   438   483   504 |
|      13 |     853.7 MB/s |   66    44    56    61    67    78    84   258 |  224   146   186   209   251   275   369   379 |
|      14 |     795.3 MB/s |   70    49    57    61    76    95   126   141 |  241   149   191   226   277   314   346   350 |
|      15 |     963.4 MB/s |   67    50    60    64    68    79   104   135 |  218   142   166   208   250   279   377   397 |
|      16 |     715.5 MB/s |   76    45    58    68    77    90   115   370 |  268   144   199   233   272   389   582   634 |
|      17 |     892.0 MB/s |   68    46    58    63    76    92   107   115 |  252   162   190   249   273   317   416   423 |
|      18 |     729.6 MB/s |   98    46    66    88   118   140   193   254 |  354   140   245   372   466   490   551   640 |
|      19 |    1052.5 MB/s |   69    46    56    64    74    93   113   175 |  252   132   201   251   298   327   335   397 |
|      20 |     959.8 MB/s |   72    47    63    69    78    91   100   112 |  295   159   220   296   344   401   434   463 |
|      21 |    1064.1 MB/s |   73    48    59    67    85    91   102   111 |  280   128   218   294   322   361   374   398 |
|      22 |    1033.1 MB/s |   77    49    65    71    80    93   107   250 |  283   146   214   273   321   376   473   482 |
|      23 |     910.2 MB/s |   86    48    69    76    89   102   277   287 |  314   172   222   294   375   408   611   720 |
|      24 |     893.0 MB/s |   81    47    59    73    83    94   250   273 |  272   152   203   251   306   336   522   579 |
|      25 |     898.2 MB/s |  103    54    71    81   110   149   263   271 |  322   169   266   313   359   413   535   644 |
|      26 |     909.3 MB/s |  101    47    67    78    92   180   284   309 |  333   205   271   322   349   384   609   685 |
|      27 |     715.9 MB/s |  140    49    73   103   138   219   512   524 |  484   155   219   522   630   774  1108  1109 |
|      28 |    1138.8 MB/s |   97    52    66    73    87   235   242   283 |  346   197   235   294   425   518   602   641 |
|      29 |     675.0 MB/s |  166    43    61   115   160   449   554   573 |  586   137   243   566   793  1015  1133  1140 |
|      30 |     988.4 MB/s |  124    50    69    94   109   277   299   434 |  343   162   268   328   417   442   600   681 |
|      31 |    1182.7 MB/s |  111    44    66    78   100   250   267   276 |  357   188   255   311   426   576   674   676 |
|      32 |    1113.8 MB/s |  125    48    66    77   132   283   308   317 |  386   196   285   359   461   557   642   649 |
+---------+----------------+------------------------------------------------+------------------------------------------------+

Download performance with 32 MB objects (a32-ram64-disk20-perf1)
                           +-------------------------------------------------------------------------------------------------+
                           |            Time to First Byte (ms)             |            Time to Last Byte (ms)              |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads |     Throughput |  avg   min   p25   p50   p75   p90   p99   max |  avg   min   p25   p50   p75   p90   p99   max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
|       8 |     711.0 MB/s |   67    44    59    66    73    77    87   159 |  345   205   263   291   333   566   816   986 |
|       9 |     812.0 MB/s |   67    45    53    57    62    82   238   260 |  325   219   240   265   315   380  1239  1428 |
|      10 |     914.8 MB/s |   59    45    50    57    63    73    84    88 |  321   229   266   295   340   409   587   612 |
|      11 |     785.9 MB/s |   77    43    60    77    87    97   113   125 |  398   197   268   400   462   554   629   708 |
|      12 |     747.7 MB/s |   63    45    56    61    66    73   102   108 |  371   202   300   338   384   436   943  1032 |
|      13 |     912.2 MB/s |   67    43    54    60    65    77    95   302 |  398   234   308   346   418   481  1019  1164 |
|      14 |     909.4 MB/s |   69    45    54    63    70    82   159   180 |  394   207   274   350   429   590   913   960 |
|      15 |     772.3 MB/s |  100    50    73    91   115   151   181   192 |  559   242   383   507   691   803  1055  1192 |
|      16 |     901.3 MB/s |   65    46    54    61    67    80   101   190 |  417   241   329   385   473   544   736   762 |
|      17 |     986.4 MB/s |   70    46    58    65    75    93   108   128 |  454   236   349   444   480   640   834  1167 |
|      18 |     854.1 MB/s |   75    51    58    67    75    97   159   220 |  466   228   335   416   505   660  1227  1291 |
|      19 |     620.0 MB/s |   73    48    60    68    75    93   131   132 |  560   241   328   425   591   944  1436  1956 |
|      20 |     733.9 MB/s |   82    46    60    70    83   103   174   396 |  510   243   322   426   583   704  1541  1562 |
|      21 |    1122.9 MB/s |   80    47    63    73    84   107   164   252 |  450   217   327   421   492   640   853  1121 |
|      22 |    1224.0 MB/s |   90    51    67    74    91   105   341   376 |  500   314   369   486   582   693   854   992 |
|      23 |     968.3 MB/s |   94    52    68    77    86   128   312   365 |  554   280   431   542   628   698   919  1092 |
|      24 |     709.6 MB/s |   95    55    68    74    90   104   462   465 |  604   270   392   490   579   895  1912  2255 |
|      25 |     742.4 MB/s |  129    48    74    88   126   265   409   524 |  589   282   375   492   634   870  1651  1790 |
|      26 |    1294.9 MB/s |  106    52    67    77    90   125   408   412 |  534   318   409   498   570   843   953  1046 |
|      27 |     719.1 MB/s |  200    44    63   115   156   613  1002  1059 |  921   222   376   919  1287  1447  2008  2163 |
|      28 |    1018.9 MB/s |  150    61    89   104   120   400   518   523 |  669   323   502   618   780   995  1111  1310 |
|      29 |    1360.5 MB/s |  147    51    74    87   136   411   459   461 |  602   334   451   541   682   915  1112  1163 |
|      30 |    1269.7 MB/s |  158    46    78   103   149   419   462   496 |  611   271   433   557   702   908  1042  1250 |
|      31 |     665.8 MB/s |  299    46   101   131   234  1016  1167  1202 | 1152   223   715  1181  1437  1934  2401  2403 |
|      32 |    1051.3 MB/s |  219    55    78   180   314   473   543   546 |  754   445   537   688   925  1085  1197  1201 |
+---------+----------------+------------------------------------------------+------------------------------------------------+