Object Stores Management with OpenStack Swift CLI¶

OpenStack Swift Client¶

swift client is the native command line utility for interacting with an OpenStack object storage environment. Although it lacks a graphical interface, it is the more popular and powerful tool for managing the data in object store. 

Setup Environment¶

Install the Swift python CLI and the keystone client (needed to authenticate on ScienceCloud) by issuing below cmds on your terminal:

pip install python-swiftclient
pip install python-keystoneclient

In order to setup your environment, please download OpenStack RC file from ScienceCloud GUI by: Access & Security > API Access > Download OpenStack RC file.

Then import this variables into your shell environment by:

source /path/to/downloaded/rc_file.sh

Manage containers¶

To manage the containers some operations needs SwiftOperator while others have no need for the operator role enough to be a member of the project.

SwiftOperator role operations¶

List containers/objects¶

Basically gives you the containers list and, if called against a container, the list of objects within:

swift list
swift list container_name

Create a container with the default storage policy¶

To create a new container with the default storage policy (i.e. replica-2) you can simply do:

swift post <container_name>
swift post test_container

Create a container with the ec104 (Erasure Coding) storage policy¶

If you are aware of the downsides (see storage policies) but would like to leverage the smaller storage overhead the ec104 storage policy has, you can create a container by:

swift post <container_name> -H 'X-Storage-Policy: <policy-name>'
swift post test_container -H 'X-Storage-Policy: ec104'

It is also advisable to allocate the _segments container that will be used by Swift when object larger than 5 GB are uploaded (failing to do so will result in a container to be automatically created with the default replica-2 policy)

swift post <container_name>_segments -H 'X-Storage-Policy: <policy-name>
swift post test_container_segments -H 'X-Storage-Policy: ec104'

Set access list¶

Swift post is the right command for setting up the container ACL. '-r'/'-w' option is used for read/write ACL respectively. Then we need to specify the project name and  the target user to be granted  access. Otherwise using '*' in any place is mend for 'any' user or project.

swift post -r 'project_name:shortname' <container_name>

Give the write access only to hrajab, and read right to everyone in project s3it.playground to container test.¶

swift post -r 's3it.playground:*' test
swift post -w 's3it.playground:hrajab' test

To delete the ACL just use the empty ''¶

swift post -r '' <container_name>
swift post -w '' <container_name>
swift stat test

Show the metadata for the container test¶

Create a public container¶

Public container is a public repository available to be read by everyone. The '.r,.rlistings'  are the right option for this purpose.

swift post -r '.r:*,.rlistings' <container_name>

The ".r:*" will remove all restrictions on reading. The ".rlistings" will allow also listing.

If you want to allow web listing of a container (so that you can easily access it with a browser) set the 'web-listings' meta key:

swift post -m "web-listings: true" <container_name>

Member role operations¶

Swift copy/download/upload¶

To download

# Example download
swift download <container_name> <object_name>
swift download test_container myobject.txt

# Example upload
swift upload <container_name> <path/to/object> --object-name <desired_object_name>
swift upload test_container ~/myobject.txt --object-name myobject.txt

# Example copy
swift copy <container_name> <object_name> -d </path/to/target_container>
swift copy test_container myobject.txt -d /target_container

Swift delete¶

Delete a container or objects within a container.

swift delete <container_name> <object_name>
swift delete test text.txt

Swift stat¶

It gives you more info such as number of objects, the size, Read/Write ACL and, policy storage for given container.

swift stat <container>
swift stat test_container
         Account: AUTH_6169dd64a7ed498581cee5abff0c6cd7
       Container: test
         Objects: 0
           Bytes: 0
        Read ACL:
       Write ACL:
         Sync To:
        Sync Key:
   Accept-Ranges: bytes
X-Storage-Policy: replica-2
     X-Timestamp: 1479479622.50440
      X-Trans-Id: txb88a0f4c9128426e9a11f-00584fe7b7
    Content-Type: text/plain; charset=utf-8

Where Account point your project path.

Swift tempurl¶

If you need to give a temporary access to your Swift object to whom is not part of your project, generating a temporary URL would be the right solution.

swift tempurl <method> <seconds> <path> <key>
swift tempurl GET 86400 /v1/AUTH_6169dd64a7ed498581cee5abff0c6cd7/test mykey
/v1/AUTH_6169dd64a7ed498581cee5abff0c6cd7/test?temp_url_sig=4a741a28ec93ee75d86d233a9e50b87282a9e15c&temp_url_expires=1481717953

This link is valid for 24 hours download.