outstanding requests

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

outstanding requests

rammohan ganapavarapu
Hi,

In zookeeper what will happened if outstanding requests are high? we have
default global limit to 1000, what will happened if the outstanding
requests go beyond global limit with is 1000? if we see huge outstanding
requests are there any actions to bring the count down?

Thanks,
Ram
Reply | Threaded
Open this post in threaded view
|

Re: outstanding requests

Abraham Fine
Hi Ram-

Outstanding requests can lead to high memory usage (possibly bringing
down members of the quorum)  and slower responses to requests.

I would take a look at the section from the "ZooKeeper: Distributed
Process Coordination" book for a good description of what happens when
the globalOutstandingLimit is reached:
https://books.google.com/books?id=OWQdAgAAQBAJ&pg=PT255&lpg=PT255&dq=%22globalOutstandingLimit%22+zookeeper&source=bl&ots=14zMtuu14N&sig=nbIIbjryge-p9UMQAWcu06mnUEQ&hl=en&sa=X&ved=0ahUKEwjsxb6grsbXAhVjyFQKHdztCOwQ6AEISDAD#v=onepage&q=%22globalOutstandingLimit%22%20zookeeper&f=false

I don't think there is any general answer to how to bring the count
down, as the answer is totally related to your usage of ZooKeeper.

Thanks,
Abe

On Fri, Nov 17, 2017, at 09:39, rammohan ganapavarapu wrote:
> Hi,
>
> In zookeeper what will happened if outstanding requests are high? we have
> default global limit to 1000, what will happened if the outstanding
> requests go beyond global limit with is 1000? if we see huge outstanding
> requests are there any actions to bring the count down?
>
> Thanks,
> Ram
Reply | Threaded
Open this post in threaded view
|

Re: outstanding requests

rammohan ganapavarapu
Abe,

Thanks for the  explanation and book reference, do you know how much memory
each outstanding request in a queue can take? We have enough memory so
trying to see if i can increase the global limit to do that i need to know
how much memory a outstanding request can take.

Thanks,
Ram

On Fri, Nov 17, 2017 at 11:43 AM, Abraham Fine <[hidden email]> wrote:

> Hi Ram-
>
> Outstanding requests can lead to high memory usage (possibly bringing
> down members of the quorum)  and slower responses to requests.
>
> I would take a look at the section from the "ZooKeeper: Distributed
> Process Coordination" book for a good description of what happens when
> the globalOutstandingLimit is reached:
> <a href="https://books.google.com/books?id=OWQdAgAAQBAJ&pg=PT255&lpg=PT255&dq=%">https://books.google.com/books?id=OWQdAgAAQBAJ&pg=PT255&lpg=PT255&dq=%
> 22globalOutstandingLimit%22+zookeeper&source=bl&ots=
> 14zMtuu14N&sig=nbIIbjryge-p9UMQAWcu06mnUEQ&hl=en&sa=X&ved=
> 0ahUKEwjsxb6grsbXAhVjyFQKHdztCOwQ6AEISDAD#v=onepage&q=%
> 22globalOutstandingLimit%22%20zookeeper&f=false
>
> I don't think there is any general answer to how to bring the count
> down, as the answer is totally related to your usage of ZooKeeper.
>
> Thanks,
> Abe
>
> On Fri, Nov 17, 2017, at 09:39, rammohan ganapavarapu wrote:
> > Hi,
> >
> > In zookeeper what will happened if outstanding requests are high? we have
> > default global limit to 1000, what will happened if the outstanding
> > requests go beyond global limit with is 1000? if we see huge outstanding
> > requests are there any actions to bring the count down?
> >
> > Thanks,
> > Ram
>
Reply | Threaded
Open this post in threaded view
|

Re: outstanding requests

Abraham Fine
Apologies for the late reply.

That's a tough question. The size of each request is dependent on many
factors (primarily use case and maxBufferSize). It would likely to be
best to experiment with the types of requests you are sending to
zookeeper and getting an idea of the approximate size of each request
that way.

Abe

On Fri, Nov 17, 2017, at 15:41, rammohan ganapavarapu wrote:

> Abe,
>
> Thanks for the  explanation and book reference, do you know how much
> memory
> each outstanding request in a queue can take? We have enough memory so
> trying to see if i can increase the global limit to do that i need to
> know
> how much memory a outstanding request can take.
>
> Thanks,
> Ram
>
> On Fri, Nov 17, 2017 at 11:43 AM, Abraham Fine <[hidden email]> wrote:
>
> > Hi Ram-
> >
> > Outstanding requests can lead to high memory usage (possibly bringing
> > down members of the quorum)  and slower responses to requests.
> >
> > I would take a look at the section from the "ZooKeeper: Distributed
> > Process Coordination" book for a good description of what happens when
> > the globalOutstandingLimit is reached:
> > <a href="https://books.google.com/books?id=OWQdAgAAQBAJ&pg=PT255&lpg=PT255&dq=%">https://books.google.com/books?id=OWQdAgAAQBAJ&pg=PT255&lpg=PT255&dq=%
> > 22globalOutstandingLimit%22+zookeeper&source=bl&ots=
> > 14zMtuu14N&sig=nbIIbjryge-p9UMQAWcu06mnUEQ&hl=en&sa=X&ved=
> > 0ahUKEwjsxb6grsbXAhVjyFQKHdztCOwQ6AEISDAD#v=onepage&q=%
> > 22globalOutstandingLimit%22%20zookeeper&f=false
> >
> > I don't think there is any general answer to how to bring the count
> > down, as the answer is totally related to your usage of ZooKeeper.
> >
> > Thanks,
> > Abe
> >
> > On Fri, Nov 17, 2017, at 09:39, rammohan ganapavarapu wrote:
> > > Hi,
> > >
> > > In zookeeper what will happened if outstanding requests are high? we have
> > > default global limit to 1000, what will happened if the outstanding
> > > requests go beyond global limit with is 1000? if we see huge outstanding
> > > requests are there any actions to bring the count down?
> > >
> > > Thanks,
> > > Ram
> >