Quantcast

Zoo Keep widely distributed?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Zoo Keep widely distributed?

wayne.rasmuss
I think zoo keeper looks very handy, but I would like to have a pretty good idea if it can work well/at all across different LANs. I would expect to have to establish a way for the members of the ensemble to talk to each other, but having to open a bunch of ports to multiple hosts would be a show stopper. Also, I'm wondering how the latency will effect overall performance.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zoo Keep widely distributed?

Martin Kou
The latency will make your writes really slow - since each write operation
would need to be confirmed by more than half of your total number of
servers in the cluster to succeed. Writes to ZooKeeper are also serialized,
so you can't parallelize the writes in a single cluster either - so your
throughput will also be low.

You can still get high throughput despite the high latency if you can shard
your writes into multiple clusters though.

Best Regards,
Martin Kou


On Mon, Aug 26, 2013 at 2:02 PM, wayne.rasmuss <
[hidden email]> wrote:

> I think zoo keeper looks very handy, but I would like to have a pretty good
> idea if it can work well/at all across different LANs. I would expect to
> have to establish a way for the members of the ensemble to talk to each
> other, but having to open a bunch of ports to multiple hosts would be a
> show
> stopper. Also, I'm wondering how the latency will effect overall
> performance.
>
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Zoo-Keep-widely-distributed-tp7579027.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Zoo Keep widely distributed?

Flavio Junqueira-2
I assume that when you say across LANs you mean different colos, although I
suppose what I'm about to say holds even if the assumption is incorrect. The
overall performance depends on your read:write ratio. Since reads are local
to a server, they don't cross the boundaries of a colo.  You could also
consider using observers to avoid the penalty of coordinating across colos
for writes to the zookeeper state.

Consider reading this blog post by Camille Fournier:

http://whilefalse.blogspot.co.uk/2012/12/building-global-highly-available.ht
ml

-Flavio  

> -----Original Message-----
> From: Martin Kou [mailto:[hidden email]]
> Sent: 27 August 2013 07:19
> To: [hidden email]
> Subject: Re: Zoo Keep widely distributed?
>
> The latency will make your writes really slow - since each write operation
> would need to be confirmed by more than half of your total number of
> servers in the cluster to succeed. Writes to ZooKeeper are also
serialized, so
> you can't parallelize the writes in a single cluster either - so your
throughput
> will also be low.
>
> You can still get high throughput despite the high latency if you can
shard

> your writes into multiple clusters though.
>
> Best Regards,
> Martin Kou
>
>
> On Mon, Aug 26, 2013 at 2:02 PM, wayne.rasmuss <
> [hidden email]> wrote:
>
> > I think zoo keeper looks very handy, but I would like to have a pretty
> > good idea if it can work well/at all across different LANs. I would
> > expect to have to establish a way for the members of the ensemble to
> > talk to each other, but having to open a bunch of ports to multiple
> > hosts would be a show stopper. Also, I'm wondering how the latency
> > will effect overall performance.
> >
> >
> >
> > --
> > View this message in context:
> > http://zookeeper-user.578899.n2.nabble.com/Zoo-Keep-widely-
> distributed
> > -tp7579027.html Sent from the zookeeper-user mailing list archive at
> > Nabble.com.
> >

Loading...