Configuring Replication on Bucket

Anirudha | Tue, 12/22/2020 - 06:42

You can configure replication between two buckets created on different Objects cluster within same Prism Central Or even across different Prism Central. In this post, we will take a look at both the replication configuration.

Configuring Replication within same Prism Central :

    

I am going to use *SJ-Objects-01* as a source cluster and SF-Objects-03 as a destination cluster.

Create a bucket in “SJ-Objects-01”. Select Bucket ->  Click on Update -> Create Replication Rule

      

 

Select Prism Central as “Local AZ” :

      

And SF-Objects-03 in Object Store drop down :

      

Create a bucket on the destination cluster (i.e SF-Objects-03) and input name in the Bucket field.      

 

Now if you click on the bucket (replication-demo) and navigate to “Replication” page on left panel, you should see all the details about your replication relation and ongoing replication status :

      

There isn't anything listed as of now, since I don't have anything. So let's try to put some data and see how this page looks.

I am going to use Objects Browser to all my experiments :

      

In above screenshot, on left I am connected to source cluster - SJ-Objects-01, and you can see "replication-demo" bucket which we created in earlier screens. And on right side, I am connected to destination cluster - SF-Objects-03, bucket "replication-dest" is listed. You will need IAM credentials and make sure you share these buckets with IAM user so its listed in the Objects Browser.

We have configured replication between, replication-demo -> replication-dest  . And both these buckets are empty:

      

 

Upload few objects to - "replication-demo" bucket , click on Upload Objects and select few files from your local machine. 

      

 

And now if you click on "Refresh" button on the objects browser UI, you should see all the files you just uploaded to "replication-demo" bucket are instantly visible in "replication-dest" bucket as well.

      

In this case I had no other workload running on this cluster. So all the storage bandwidth and network bandwidth is available for replication. The moment you write the data to source bucket, Objects will immediately read the data from the bucket and replicate it to destination bucket. And this happens quite fast so it almost appears to be sync replication. If you upload larger objects, then you may see small delay since network replication may take a while. But replication gets trigger as soon as data is written to source bucket, to achieve near-sync capability.

Quick look at the replication stats on Objects UI :

      

So here you can see, UI shows “Last replication point” , Pending objects and performance chart.  This also shows the replication relationship, i.e source and destination bucket & cluster.

You can also configure the reverse replication i.e any data which gets uploaded to replication-dest buckst should get replicated to replication-demo bucket.

Once you configure reverse replication, both the clusters (i.e source and destination should detect this configuration) should give option in UI to monitor inbound or outbound replication stats :

      

If your incoming rate is high enough, then you may see "Object size pending replication" stats showing how much data is to be replicated. Objects will backoff a bit and will fall to background replication. Background replication is triggered at every 6 hours, and depending on the load and network bandwidth, it may finish sooner or may take bit longer. 

 

Next Read - IAM Replication