Bucket Management#
This section describes how to create new buckets and how to manage them in Carbonio. This functionality is available on the Carbonio Admin Panel as well, under (see Section Bucket list).
To prevent possible misunderstanding, it is necessary to make clear the difference between a Remote Bucket (also called ObjectStorage) and a Carbonio Bucket.
- Remote Bucket
-
The Remote Bucket, called also (External) Storage or ObjectStorage, is created on a remote side, i.e, on the provider’s infrastructure or on a local storage device, possibly external to Carbonio's infrastructure . The same Bucket can be used by Carbonio Storage as External ObjectStorage and for Carbonio Backup at the same time.
- Carbonio Bucket
-
The Carbonio Bucket is created on the local Carbonio infrastructure and instantiates a Remote Bucket. Carbonio Buckets are mutually exclusive, that is, you can not use the same bucket for both Carbonio Storage and Carbonio Backup.
It is therefore suggested to pay attention and give the Buckets a meaningful name and suitable labels or prefixes to be able to distinguish them quickly.
Bucket Creation#
In this section, we will show the creation and management of an S3 bucket; to create buckets on other providers replace S3 with the name of your preferred provider.
Other supported providers are S3, Ceph, Cloudian, EMC, Scality S3, Custom S3, Alibaba, and MinIO.
Hint
These values are used as the type of bucket in the command presented in the remainder of the section.
The option for these providers may vary, simply run carbonio core DoCreateBucket provider help, replacing provider with any of the supported providers to obtain the list of the required parameters.
The command used to create a bucket is carbonio core doCreateBucket, whose syntax is the following
zextras$ carbonio core doCreateBucket provider bucket_name access_key \
secret [param VALUE[,VALUE]]
Parameter List
NAME |
TYPE |
EXPECTED VALUES |
DEFAULT |
|---|---|---|---|
provider (M) |
String |
Supported storage provider |
|
bucket_name (M) |
String |
Amazon AWS bucket |
|
access_key (M) |
String |
Service username of bucket |
|
secret (M) |
String |
Service password |
|
label (M) |
String |
Bucket configuration description |
|
region (O) |
String |
Amazon AWS Region |
|
url (O) |
String |
S3 API compatible service URL (e.g., s3api.example.com) |
|
prefix (O) |
String |
Prefix added to blobs keys |
|
notes (O) |
String |
Bucket configuration details |
(M) == mandatory parameter, (O) == optional parameter
Example
zextras$ carbonio core doCreateBucket S3 MyBucketName \
X58Y54E5687R543 abCderT577eDfjhf My_New_Bucket url \
https://example_bucket_provider.com
The following values are used
S3 as the type of bucket
BucketName as the name of the bucket, which must coincide with the name on the remote provider, otherwise the command will fail
X58Y54E5687R543 as the remote username (access_key)
abCderT577eDfjhf as the remote password (bucket_name)
My_New_Bucket is a label given to the bucket
https://example_bucket_provider.com is the endpoint to which Carbonio Storage connects to the bucket
The above example creates an S3 bucket on Amazon AWS, using the provided credentials. The most important output of this command is the Bucket ID, which is an UUID string similar to 60b8139c-d56f-4012-a928-4b6182756301 or 58fa4ca2-31dd-4209-aa23-48b33b116090. This is needed to verify connection or to retrieve information about the bucket. Please refer to next section Checking a Bucket for details.
Once created, a bucket can be used as an External ObjectStorage in the Carbonio Storage module or for keeping a remote (external) Carbonio Backup. In the former case, you need to create a volume on it (see directions in Section Activate Backup on External Storage), in the latter you can use it straight away.
Hint
If used in Carbonio Storage, you can find useful configuration tips in Section Carbonio Storage and S3 buckets.
Finally, before starting to use the Bucket, good practices suggest that you test that the connection is properly working, by using the command shown in the next section.
Checking a Bucket#
There are two commands that you can use to verify that a bucket is correctly accessible to Carbonio.
Test the connection using the bucket ID received in the previous step (for example 60b8139c-d56f-4012-a928-4b6182756301):
zextras$ carbonio core testS3Connection 60b8139c-d56f-4012-a928-4b6182756301
If the command is successful you will see the message connection
ok.
There is a dedicated command used to report the bucket usage and a number of information.
zextras$ carbonio core listBuckets all
The output will look similar to:
bucketName hsm
protocol HTTPS
storeType S3
accessKey xxxxx
region EU_WEST_1
uuid 58fa4ca2-31dd-4209-aa23-48b33b116090
usage in powerstore volumes
server: srv1 volume: centralized-s3
server: srv2 volume: centralized-s3
usage in external backup unused
bucketName backup
protocol HTTPS
storeType S3
accessKey xxxxxxx
region EU_WEST_1
destinationPath server2
uuid 5d32b50d-79fc-4591-86da-35bedca95de7
usage in powerstore volumes unused
usage in external backup
server: srv2
Instead of the all keyword, you can use the type of the
bucket to filter the results.
This command can be used also to retrieve the bucket ID, which is
the uuid line.