Getting started
Once you get your public cloud credentials, you can create a container (bucket) and store objects. An object stores data content, such as documents, images...
Main features
- Guarantees data availability, durability and consistency
- Ideal for storing "unstructured" data that can grow without limit
- Swift allows files to be sent via the dashboard or via an API
- Files are contained in containers
- The upload is limited to :
- 5Gb per file via the dashboard
- 50Tb via API (multi-threading DLO/SLO)
- The name of each container must be unique in each project
- Unlimited storage capacity
- Swift cannot be used to install an operating system or a database
- If a file upload is successful it returns an HTTP 200 code
- By default containers are private
- Possibility to make a container public
Object Storage using Swift Client
Some commands are not available through the OpenStack CLI, so you will need to install the Swift Client.
Ubuntu / Debian
apt install python3-swiftclient
Centos / Red Hat
yum install python3-swiftclient
Create a container
I'll create a container named public:
taylor@laptop:~ (pub1|taylor)$ openstack container create public
+---------------------------------------+-----------+------------------------------------+
| account | container | x-trans-id |
+---------------------------------------+-----------+------------------------------------+
| AUTH_d7a1d09c0323488aa00d9e163f3655bf | public | tx59c2735e9bfd4a5181079-006048ddc8 |
+---------------------------------------+-----------+------------------------------------+
List your containers
taylor@laptop:~ (pub1|taylor)$ openstack container list
+--------+
| Name |
+--------+
| public |
+--------+
Store your first object
taylor@laptop:~ (pub1|taylor)$ openstack object create --name 'eBooks/Cuisine/Les Crêpes du Monde.pdf' public 'Documents/Ebooks/Cuisine/Les Crêpes du Monde.pdf'
+----------------------------------------+-----------+----------------------------------+
| object | container | etag |
+----------------------------------------+-----------+----------------------------------+
| eBooks/Cuisine/Les Crêpes du Monde.pdf | public | a87f8e432d5c0c163d575a7fde411be8 |
+----------------------------------------+-----------+----------------------------------+
Here with --name
option I override the final object name, but it's not mandatory. By default it uses your local object name (Documents/Ebooks/Cuisine/Les Crêpes du Monde.pdf)
List your objects
You need to specify your container name...
taylor@laptop:~ (pub1|taylor)$ openstack object list public
+----------------------------------------+
| Name |
+----------------------------------------+
| eBooks/Cuisine/Les Crêpes du Monde.pdf |
+----------------------------------------+
Object Storage using CURL
Get an authentication token
You can get a token using the swift cli : swift auth
Example:
taylor@laptop:~$ swift auth
export OS_STORAGE_URL=https://s3.pub1.infomaniak.cloud/object/v1/AUTH_d1440aaxx1qwd610b9bac2b842c8defa
export OS_AUTH_TOKEN=gAAAAABlXIoel6rrwF733jxw3Ex7Uc5djiVoVk8JEwTAmlNTtGaIKExRX3aSQJRB3OSVTqwCyDxaZMxesv_jfhGDGYWefCnvyiaiQpj7ErtuRMUlj3Ld6TDXj7zXzwFYgCn41kA1sZvTK-nZwdlxIGzCwQJr4Y15JWHIYjciinsdf8765w3pny6o
or using curl, create a script called get_token.sh
with the following content:
#!/bin/sh
TMPFILE=`mktemp`
JSONFILE=`mktemp`
cat >${JSONFILE} <<EOF
{
"auth": {
"identity": {
"methods": ["password"],
"password": {
"user": {
"domain": {"name": "${OS_USER_DOMAIN_NAME}"},
"name": "${OS_USERNAME}",
"password": "${OS_PASSWORD}"
}
}
},
"scope": {
"project": {
"domain": {"name": "${OS_PROJECT_DOMAIN_NAME}"},
"name": "${OS_PROJECT_NAME}"
}
}
}
}
EOF
curl -si \
-H "Content-Type: application/json" \
-o ${TMPFILE} \
-d @${JSONFILE} \
${OS_AUTH_URL}/auth/tokens 2>/dev/null
tail -1 ${TMPFILE} | jq
grep -i "x-subject-token" ${TMPFILE}
rm -f ${TMPFILE} ${JSONFILE}
Once you have a Token you can operate with the Object Storage:
Create a container
taylor@laptop:~$ curl -i -X PUT -H "X-Auth-Token: $OS_AUTH_TOKEN" $OS_STORAGE_URL/mycontainer
HTTP/1.1 202 Accepted
content-type: text/html; charset=UTF-8
content-length: 76
x-trans-id: tx1d77854fae534495a2a94-00655c8b2d
x-openstack-request-id: tx1d77854fae534495a2a94-00655c8b2d
date: Tue, 21 Nov 2023 10:49:17 GMT
strict-transport-security: max-age=63072000
<html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>
Upload an object
curl -i -T my_object -X PUT -H "X-Auth-Token: $OS_AUTH_TOKEN" ${OS_STORAGE_URL}/mycontainer/my_object
Download an object
curl -S -X GET -H "X-Auth-Token: $OS_AUTH_TOKEN" ${OS_STORAGE_URL}/mycontainer/my_object -O
List containers
curl -S -X GET -H "X-Auth-Token: $OS_AUTH_TOKEN" ${OS_STORAGE_URL}/
List objects in a container
curl -S -X GET -H "X-Auth-Token: $OS_AUTH_TOKEN" ${OS_STORAGE_URL}/mycontainer/
Delete an object
curl -S -X DELETE -H "X-Auth-Token: $OS_AUTH_TOKEN" ${OS_STORAGE_URL}/mycontainer/myobject
Delete a container (must be empty)
curl -S -X DELETE -H "X-Auth-Token: $OS_AUTH_TOKEN" ${OS_STORAGE_URL}/mycontainer
Object versioning
Creating a bucket with versioning enabled
taylor@laptop:~ (pub1|taylor):~$ swift post -H "X-Versions-Enabled: true" mybucket
from now on, each uploaded object will be versioned with an ID. this ID is the timestamp of the upload.
taylor@laptop:~ (pub1|taylor):~$ swift upload mybucket object1
object1
taylor@laptop:~ (pub1|taylor):~$ swift upload mybucket object1
object1
Listing the objects and available versions
taylor@laptop:~ (pub1|taylor):~$ swift list mybucket --versions
183 2023-05-26 07:19:27 1685085567.70333 application/octet-stream object1
183 2023-05-26 07:19:22 1685085562.07717 application/octet-stream object1
Downloading a specific version of an object
taylor@laptop:~ (pub1|taylor):~$ swift download mybucket object1 --version-id 1685085562.07717
object1 [auth 0.471s, headers 0.691s, total 0.692s, 0.001 MB/s]
Deleting a specific version of an object
taylor@laptop:~ (pub1|taylor):~$ swift delete mybucket object1 --version-id 1685085567.70333
object1
Note
Deleting a versioned object will delete the current version but not the previous ones. Older versions are therefore still available.
taylor@laptop:~ (pub1|taylor):~$ swift delete mybucket object1
object1
taylor@laptop:~ (pub1|taylor):~$ swift list mybucket --versions
0 2023-05-26 07:21:00 1685085660.97534 application/x-deleted;swift_versions_deleted=1 object1
183 2023-05-26 07:20:54 1685085654.97972 application/octet-stream object1
183 2023-05-26 07:19:22 1685085562.07717 application/octet-stream object1
View your account statistics
taylor@laptop:~ (pub1|taylor)$ openstack object store account show
+------------+---------------------------------------+
| Field | Value |
+------------+---------------------------------------+
| Account | AUTH_d7a1d09c0323488aa00d9e163f3655bf |
| Bytes | 9149046 |
| Containers | 1 |
| Objects | 1 |
+------------+---------------------------------------+
Display capabilities
To know enabled plugins (middlewares) on our cluster, you can use:
$ swift capabilities