There's no reason you can't create another set of zones that have a master in us-west, call it us-west-2. Then users that need low latency to us-west write to http://us-west-2.cluster/, and users that need low latency to us-east write to http://us-east-1.cluster/. In general, you want your replication zones geographically separated. In my setup, I have a single region, us. I have the primary zone in California and the secondary zone in Texas. When I create objects in the primary, they appear in the secondary zone in a couple minutes. Then if I need low latency read access in Texas, I read from the secondary. I don't currently need to write from Texas. When I do, I'll create a new set of zones that are primary in Texas and secondary in California. Then Texas can write to that new zone. On Mon, Jul 14, 2014 at 7:50 AM, Bachelder, Kurt < Kurt.Bachelder at sierra-cedar.com> wrote: > Hi Craig ? Thanks for the info? My understanding is the same as what > you?ve laid out below ? I?ve scraped together enough information to > understand that Ceph should be working the same as Amazon S3, which is a > relief. > > > > Here?s my issue ? I have 2 regions, geographically separated. Each region > contains 2 zones, which replicate metadata + data ? and that works > perfectly ? no issues whatsoever. However, when I add regional replication > for a unified namespace, the location constraint SEEMS to be completely > ignored. Let?s say ?us-east? is my master region and ?us-west? is my slave > ? for latency reasons, I want the actual data to be stored in us-west. So > I submit a command to us-east to create a bucket with > location_constraint=us-west. However, the radosgw seems to completely > ignore the constraint and it creates the bucket in us-east. When I start > writing data to the bucket, it physically resides in us-east, not us-west, > which doesn?t work for us from a latency standpoint. > > > > Creating users and buckets in us-east works fine, and the metadata filters > down to all other regions/zones ? so I know the replication working to some > extent. > > > > Oh, and you?re right about the redirect ? when I try to create a bucket > against the ?slave? region us-west, it DOES redirect the request to the > master region? but still fails to create the bucket in us-west. It is > created in pools in us-east instead. > > > > Frustrating ? the documentation is so sparse around this whole > multi-region setup ? especially in regards to location constraints. I?m > still not sure whether my configuration is off or whether the RGW just > isn?t working as expected? > > > > Thank you for your reply ? keep in touch if you end up doing some > multi-region replication, would love to hear your experience. > > > > Kurt > > > > *From:* Craig Lewis [mailto:clewis at centraldesktop.com] > *Sent:* Saturday, July 12, 2014 7:51 PM > *To:* Bachelder, Kurt > *Subject:* Re: [ceph-users] Creating a bucket on a non-master region in a > multi-region configuration with unified namespace/replication > > > > Disclaimer: I haven't used Amazon S3 yet, so I'm only familiar with > regions, zones, and placements as they apply to Ceph. I'm pretty sure Ceph > is trying hard to emulate Amazon, but it's obviously missing some features. > I was hoping somebody else would chime in, but I'll give it a shot. > > > > > > > > Bucket creation (really, all writes) should only occur in a master zone. > > > > Zones (within a region) are intended to have data+metadata replication, so > the primary zone will get the writes, then replicate everything to the > secondary zone(s). You can read from the primary or secondary, as long as > you remember that replication is asynchronous, and generally takes a few > minutes even when it's keeping up. You can have multiple zones in a region > that partition instead of replicate as well. For example, us-west-1 > replicates to us-east-1, and us-east-2 replicates to us-west-2. > > > > Regions can have metadata replication (without data replication), if you > want a globally unique list of users and pools. It's up to you if you want > that. If you plan to move data between regions, then you probably do. > It's not required if you want us-west-1/bucket/ and us-east-2/bucket/ to > be different objects (but it's confusing). > > > > If you want geo-replication, each zone in a region should have a secondary > zone with data replication. zones & data replication are the method of > accomplishing geo-replication. Regions are a way to isolate data for > latency/legal reasons. > > > > > > Placement targets are setup inside a region. They let the end user select > different storage mechanisms that you've configured. For example, you > might have default (3x replication on HDD), fast (2x replication on SSD), > and durable (4x replication on HDD). The user could pick the most > appropriate one when creating a bucket. I haven't tested non-default > placement targets and replication yet, so I don't know exactly what > happens, or how to set it up. > > > > > > I haven't setup multiple regions yet, just multiple zones in one region. > I'm really not sure what the behavior should be if you have metadata > replication enabled, and create an object in region1, then try to read it > in region2. I would hope it returns a 302 redirection to the primary > region and zone. > > > > > > So when you say > > The issue we?re having is with creating a bucket in the slave region. > As we understand it, a bucket is considered ?metadata?, and should be > created via the master region, but using a ?location constraint? to ensure > the objects within the bucket are physically stored in the secondary > region. > > In Ceph, you should create the bucket in the master zone of your region, > and it will be physically stored in the master and slave zones. > > > > > > Does that answer your question? > > > > > > > > On Thu, Jul 10, 2014 at 7:40 AM, Bachelder, Kurt < > Kurt.Bachelder at sierra-cedar.com> wrote: > > OK ? I found this information ( > http://comments.gmane.org/gmane.comp.file-systems.ceph.user/4992): > > > > When creating a bucket users can specify which placement target they > > want to use with that specific bucket (by using the S3 create bucket > > location constraints field, format is > > location_constraints='[region][:placement-target]', e.g., > > location_constraints=':fast-placement'). > > > > This looks like exactly what I want to do ? but HOW is the > location_constraints field used to specify a region for bucket creation? > Is there any documentation around this? > > > > *From:* ceph-users [mailto:ceph-users-bounces at lists.ceph.com] *On Behalf > Of *Bachelder, Kurt > *Sent:* Tuesday, July 08, 2014 3:22 PM > *To:* ceph-users at ceph.com > *Subject:* [ceph-users] Creating a bucket on a non-master region in a > multi-region configuration with unified namespace/replication > > > > Hi ? > > > > We have a multi-region configuration with metadata replication between the > regions for a unified namespace. Each region has pools in a different > cluster. Users created in the master region are replicated to the slave > region without any issues ? we can get user info, and everything is > consistent. Buckets created in the master region are also indexed in the > slave region without any issues. > > > > The issue we?re having is with creating a bucket in the slave region. As > we understand it, a bucket is considered ?metadata?, and should be created > via the master region, but using a ?location constraint? to ensure the > objects within the bucket are physically stored in the secondary region. > However, there is a lack of documentation on exactly **how** to > accomplish this ? we?ve attempted with s3cmd using the ?bucket-location > parameter to point to the slave region, but the bucket is still created in > the master region. > > > > We noticed that the ceph S3 API documentation ( > http://ceph.com/docs/master/radosgw/s3/#features-support) shows that the > Amazon S3 functionality for ?bucket location? is not supported? but I can?t > imagine that restriction renders the entire slave region setup useless? > does it? > > > > So how are buckets created in a secondary region? > > > > Thanks in advance! > > > > Kurt > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140714/b5f82c22/attachment.htm>
 |