OpenIO : How to Flush a Bucket

Background

When trying out OpenIO or when some unexpected event occurs on the platform, it may happen that you would like to flush a specific bucket from its content. I had the opportunity to very quickly fill-up my test platform with some outdated backup sets archived out of my Rubrik platform. It was not critical data and it was safe to flush them. But ok, how can we do this ?

Flushing a Bucket

This is not feasible out of the box and OpenIO support guided me through this process.

1) Get the exact name of the bucket you would like to flush

2) List all object in that bucket

3) Issue a delete command

2) and 3) can be done in a "piped" command so, you can automate the process which makes life easier (less safer?).

Getting the bucket name

# openio --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 container list 

+--------------------------+-------------+-------+

| Name                     |       Bytes | Count |

+--------------------------+-------------+-------+

| rubrik-rubrik-0          |     1428516 |   311 |

| rubrik-rubrik-0+segments | 45051929040 |  2751 |

+--------------------------+-------------+-------+

Let assume that I would like to flush the bucket called "rubrik-rubrik-0". This is where Rubrik is storing the meta data of the snapshot it actually archive on the OpenIO platform. Again, blinking red light, I assume that you know what you do by flushing data from the bucket. The purpose here is to flush and not to tell you why to flush and I'm not guaranteeing any miss-functioning features of the software that fills the object store platform.

Listing objects in a single bucket

# openio object list rubrik-rubrik-0 --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 -f value -c Name             

96ea5630-0d1f-4c35-8946-bd4b9f4dd68f

blobstore/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/d4151af1-c0f7-482b-9651-bd3ab1cfa78b/.metadata_backup/16153_d922b40a-be62-4db5-b4ce-6ccf6b3b1a27.bak.gz

blobstore/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/d4151af1-c0f7-482b-9651-bd3ab1cfa78b/.metadata_backup/16192_9e5be9bd-7d1e-4e96-8055-b179896c4682.bak.gz

blobstore/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/d4151af1-c0f7-482b-9651-bd3ab1cfa78b/.metadata_backup/16195_16341e0f-48c4-4c75-938e-53bf96782962.bak.gz

blobstore/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/d4151af1-c0f7-482b-9651-bd3ab1cfa78b/.metadata_backup/16198_0951f57d-c4b9-47e3-84bd-b7bb66429184.bak.gz

blobstore/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/d4151af1-c0f7-482b-9651-bd3ab1cfa78b/.metadata_backup/16201_0a12b8d0-d13f-4000-b292-ed7051f8f4c2.bak.gz

[...]

Deleting a single object

# openio --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 object delete rubrik-rubrik-0  snappables/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/08a470ef-a9fe-4313-9706-eded9e703913/08a470ef-a9fe-4313-9706-eded9e703913_1_snapshot.json

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+

| Name                                                                                                                                                                                      | Deleted |

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+

| snappables/009/291/101319ea-62c0-4ace-bf9a-31a9322b121b-vm-3cf38311-8627-4bfd-bf21-af10bba5a164/08a470ef-a9fe-4313-9706-eded9e703913/08a470ef-a9fe-4313-9706-eded9e703913_1_snapshot.json | True    |

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+

Now, it is easy to combine the listing and deleting command in a chained command with pipes like this : 

# openio object list rubrik-rubrik-0 --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 -f value -c Name -c Size | while read line; do echo "object delete rubrik-rubrik-0 $line"; done | openio --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842

This will delete 1000 objects at a time, to delete all objects, no matter the amount, use the --full switch immediately after "-c Size".

Again, pay attention to what you are doing, deleting objects is not a trivial task and you absolutely need to know what you are doing.

Once the bucket is empty, you can remove it with this command :

# openio --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 container list
+--------------------------+-------------+-------+
| Name                     |       Bytes | Count |
+--------------------------+-------------+-------+
| rubrik-rubrik-0          |           0 |     0 |
| rubrik-rubrik-0+segments | 45051929040 |  2751 |
+--------------------------+-------------+-------+

# openio --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 container delete rubrik-rubrik-0        

# openio --oio-ns OPENIO --oio-account AUTH_34698f84fb3541e1b35b8663f70a0842 container list 
+--------------------------+-------------+-------+
| Name                     |       Bytes | Count |
+--------------------------+-------------+-------+
| rubrik-rubrik-0+segments | 45051929040 |  2751 |
+--------------------------+-------------+-------+

Enjoy !