Quantcast

What is the role of Zookeeper and its external Integration dependencies

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

What is the role of Zookeeper and its external Integration dependencies

upendar devu
We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
Apache Storm and Zookeeper.

I would like to understand the following things. Please help me to clarify.

1. What are Integration dependencies of Zookeeper ?
   Since we are using Kafka, Storm , so we need Zookeeper ? which external
integration has dependencies

2. What is the actual role of zookeeper with the integration of Apache
Kafka,Apache Storm.   Could you please provide a better use case to
understand.

3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to upgrade
depedency integration components like Apache Storm(*using 0.10.0*) and
Apache Kafka (Using *0.8.1.1*) ? which version those should be upgraded to
 ?


Thanks
Upendar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What is the role of Zookeeper and its external Integration dependencies

upendar devu
Including Dev Team to address my queries

On Thu, Apr 13, 2017 at 2:39 PM, upendar devu <[hidden email]>
wrote:

> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
> Apache Storm and Zookeeper.
>
> I would like to understand the following things. Please help me to clarify.
>
> 1. What are Integration dependencies of Zookeeper ?
>    Since we are using Kafka, Storm , so we need Zookeeper ? which external
> integration has dependencies
>
> 2. What is the actual role of zookeeper with the integration of Apache
> Kafka,Apache Storm.   Could you please provide a better use case to
> understand.
>
> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to upgrade
> depedency integration components like Apache Storm(*using 0.10.0*) and
> Apache Kafka (Using *0.8.1.1*) ? which version those should be upgraded
> to  ?
>
>
> Thanks
> Upendar
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What is the role of Zookeeper and its external Integration dependencies

upendar devu
Sorry looks like I missed to share my queries to dev team

We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
Apache Storm and Zookeeper.

I would like to understand the following things. Please help me to clarify.

1. What are Integration dependencies of Zookeeper ?
   Since we are using Kafka, Storm , so we need Zookeeper ? which external
integration has dependencies

2. What is the actual role of zookeeper with the integration of Apache
Kafka,Apache Storm.   Could you please provide a better use case to
understand. sorry asking this question, I need to understand and am aware
of this at least now from you.

3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to upgrade
depedency integration components like Apache Storm(*using 0.10.0*) and
Apache Kafka (Using *0.8.1.1*) ? which version those should be upgraded to
 ?

On Thu, Apr 13, 2017 at 2:43 PM, upendar devu <[hidden email]>
wrote:

> Including Dev Team to address my queries
>
> On Thu, Apr 13, 2017 at 2:39 PM, upendar devu <[hidden email]>
> wrote:
>
>> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
>> Apache Storm and Zookeeper.
>>
>> I would like to understand the following things. Please help me to
>> clarify.
>>
>> 1. What are Integration dependencies of Zookeeper ?
>>    Since we are using Kafka, Storm , so we need Zookeeper ? which
>> external integration has dependencies
>>
>> 2. What is the actual role of zookeeper with the integration of Apache
>> Kafka,Apache Storm.   Could you please provide a better use case to
>> understand.
>>
>> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
>> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to
>> upgrade depedency integration components like Apache Storm(*using 0.10.0*)
>> and Apache Kafka (Using *0.8.1.1*) ? which version those should be
>> upgraded to  ?
>>
>>
>> Thanks
>> Upendar
>>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What is the role of Zookeeper and its external Integration dependencies

upendar devu
Could you please respond to my query. Thanks

On Thu, Apr 13, 2017 at 2:46 PM, upendar devu <[hidden email]>
wrote:

> Sorry looks like I missed to share my queries to dev team
>
> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
> Apache Storm and Zookeeper.
>
> I would like to understand the following things. Please help me to clarify.
>
> 1. What are Integration dependencies of Zookeeper ?
>    Since we are using Kafka, Storm , so we need Zookeeper ? which external
> integration has dependencies
>
> 2. What is the actual role of zookeeper with the integration of Apache
> Kafka,Apache Storm.   Could you please provide a better use case to
> understand. sorry asking this question, I need to understand and am aware
> of this at least now from you.
>
> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to upgrade
> depedency integration components like Apache Storm(*using 0.10.0*) and
> Apache Kafka (Using *0.8.1.1*) ? which version those should be upgraded
> to  ?
>
> On Thu, Apr 13, 2017 at 2:43 PM, upendar devu <[hidden email]>
> wrote:
>
>> Including Dev Team to address my queries
>>
>> On Thu, Apr 13, 2017 at 2:39 PM, upendar devu <[hidden email]>
>> wrote:
>>
>>> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
>>> Apache Storm and Zookeeper.
>>>
>>> I would like to understand the following things. Please help me to
>>> clarify.
>>>
>>> 1. What are Integration dependencies of Zookeeper ?
>>>    Since we are using Kafka, Storm , so we need Zookeeper ? which
>>> external integration has dependencies
>>>
>>> 2. What is the actual role of zookeeper with the integration of Apache
>>> Kafka,Apache Storm.   Could you please provide a better use case to
>>> understand.
>>>
>>> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
>>> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to
>>> upgrade depedency integration components like Apache Storm(*using
>>> 0.10.0*) and Apache Kafka (Using *0.8.1.1*) ? which version those
>>> should be upgraded to  ?
>>>
>>>
>>> Thanks
>>> Upendar
>>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What is the role of Zookeeper and its external Integration dependencies

Bobby Evans
Upendar,
You are asking questions about multiple projects on a mailing list only for one of them.  Your questions are also a bit confusing which is probably why no one has answered them yet.  I understand a bit about ZK, and kafka, but I am a storm committer so hopefully I can answer some of your questions.
1)  The dependencies for ZK are called out in the pom.xml for the version you are using.

https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper
For 3.4.10, the version you are asking about https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.4.10 lists several dependencies.   The client and the server are together in the same package and there are some configuration options here too that control the usage of some dependencies.  As such I am not sure exactly which are required just for the client and what are just for the server.  I do know that jline is just for the client and is not even a 100% requirement there.

2) The role of ZK in storm is to store the current state of the cluster.  In the 1.x release and above it also does leader election.  For Kafka the role is similar.  It holds the state of the system, although I am not as familiar with the internals here.
As for providing a better use case to understand I am not sure at all what you mean.  ZK is a fairly general purpose state store that is used by a lot of different projects in different ways.  If you want to see some of what is possible look at http://curator.apache.org/ which provides higher level APIs on top of ZK to do a lot of different useful things.
3) This is where I am not an expert.  The CVE you mentioned appears to be for a buffer overflow in the C API.  I know storm only uses the java API so it should not be an issue for you.  I am not sure about kafka, but I suspect that it too does not use the C API.  You might want to check on the kafka mailing list though.
If you just want to upgrade to 3.4.10, you probably can on the server side.  I believe that all of the 3.4.x clients should be compatible with all of the 3.4.x servers, but you probably want to test it out first to be sure it all appears to be working.  As for upgrading the clients that is something you need to work with both storm and kafka to do.   For storm I am not totally sure on 0.10 if zookeeper is shaded or not.  I don't think it is so you can probably just replace the zookeeper jar in the lib directory on all of the nodes with the new one.  But I don't know for sure.

- Bobby

On Thursday, April 20, 2017, 9:02:51 PM CDT, upendar devu <[hidden email]> wrote:Could you please respond to my query. Thanks

On Thu, Apr 13, 2017 at 2:46 PM, upendar devu <[hidden email]>
wrote:

> Sorry looks like I missed to share my queries to dev team
>
> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
> Apache Storm and Zookeeper.
>
> I would like to understand the following things. Please help me to clarify.
>
> 1. What are Integration dependencies of Zookeeper ?
>    Since we are using Kafka, Storm , so we need Zookeeper ? which external
> integration has dependencies
>
> 2. What is the actual role of zookeeper with the integration of Apache
> Kafka,Apache Storm.  Could you please provide a better use case to
> understand. sorry asking this question, I need to understand and am aware
> of this at least now from you.
>
> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to upgrade
> depedency integration components like Apache Storm(*using 0.10.0*) and
> Apache Kafka (Using *0.8.1.1*) ? which version those should be upgraded
> to  ?
>
> On Thu, Apr 13, 2017 at 2:43 PM, upendar devu <[hidden email]>
> wrote:
>
>> Including Dev Team to address my queries
>>
>> On Thu, Apr 13, 2017 at 2:39 PM, upendar devu <[hidden email]>
>> wrote:
>>
>>> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
>>> Apache Storm and Zookeeper.
>>>
>>> I would like to understand the following things. Please help me to
>>> clarify.
>>>
>>> 1. What are Integration dependencies of Zookeeper ?
>>>    Since we are using Kafka, Storm , so we need Zookeeper ? which
>>> external integration has dependencies
>>>
>>> 2. What is the actual role of zookeeper with the integration of Apache
>>> Kafka,Apache Storm.  Could you please provide a better use case to
>>> understand.
>>>
>>> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
>>> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to
>>> upgrade depedency integration components like Apache Storm(*using
>>> 0.10.0*) and Apache Kafka (Using *0.8.1.1*) ? which version those
>>> should be upgraded to  ?
>>>
>>>
>>> Thanks
>>> Upendar
>>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What is the role of Zookeeper and its external Integration dependencies

hanm
Some notes on the CVE - it's only affecting the C client shell, which is
not part of the C client API. Even if some of the projects mentioned here
use C client API (which afaik does not), they should not be impacted by
this specific CVE from a functional point of view.

On Fri, Apr 21, 2017 at 6:48 AM, Bobby Evans <[hidden email]>
wrote:

> Upendar,
> You are asking questions about multiple projects on a mailing list only
> for one of them.  Your questions are also a bit confusing which is probably
> why no one has answered them yet.  I understand a bit about ZK, and kafka,
> but I am a storm committer so hopefully I can answer some of your questions.
> 1)  The dependencies for ZK are called out in the pom.xml for the version
> you are using.
>
> https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper
> For 3.4.10, the version you are asking about https://mvnrepository.com/
> artifact/org.apache.zookeeper/zookeeper/3.4.10 lists several
> dependencies.   The client and the server are together in the same package
> and there are some configuration options here too that control the usage of
> some dependencies.  As such I am not sure exactly which are required just
> for the client and what are just for the server.  I do know that jline is
> just for the client and is not even a 100% requirement there.
>
> 2) The role of ZK in storm is to store the current state of the cluster.
> In the 1.x release and above it also does leader election.  For Kafka the
> role is similar.  It holds the state of the system, although I am not as
> familiar with the internals here.
> As for providing a better use case to understand I am not sure at all what
> you mean.  ZK is a fairly general purpose state store that is used by a lot
> of different projects in different ways.  If you want to see some of what
> is possible look at http://curator.apache.org/ which provides higher
> level APIs on top of ZK to do a lot of different useful things.
> 3) This is where I am not an expert.  The CVE you mentioned appears to be
> for a buffer overflow in the C API.  I know storm only uses the java API so
> it should not be an issue for you.  I am not sure about kafka, but I
> suspect that it too does not use the C API.  You might want to check on the
> kafka mailing list though.
> If you just want to upgrade to 3.4.10, you probably can on the server
> side.  I believe that all of the 3.4.x clients should be compatible with
> all of the 3.4.x servers, but you probably want to test it out first to be
> sure it all appears to be working.  As for upgrading the clients that is
> something you need to work with both storm and kafka to do.   For storm I
> am not totally sure on 0.10 if zookeeper is shaded or not.  I don't think
> it is so you can probably just replace the zookeeper jar in the lib
> directory on all of the nodes with the new one.  But I don't know for sure.
>
> - Bobby
>
> On Thursday, April 20, 2017, 9:02:51 PM CDT, upendar devu <
> [hidden email]> wrote:Could you please respond to my query. Thanks
>
> On Thu, Apr 13, 2017 at 2:46 PM, upendar devu <[hidden email]>
> wrote:
>
> > Sorry looks like I missed to share my queries to dev team
> >
> > We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
> > Apache Storm and Zookeeper.
> >
> > I would like to understand the following things. Please help me to
> clarify.
> >
> > 1. What are Integration dependencies of Zookeeper ?
> >    Since we are using Kafka, Storm , so we need Zookeeper ? which
> external
> > integration has dependencies
> >
> > 2. What is the actual role of zookeeper with the integration of Apache
> > Kafka,Apache Storm.  Could you please provide a better use case to
> > understand. sorry asking this question, I need to understand and am aware
> > of this at least now from you.
> >
> > 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
> > CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to upgrade
> > depedency integration components like Apache Storm(*using 0.10.0*) and
> > Apache Kafka (Using *0.8.1.1*) ? which version those should be upgraded
> > to  ?
> >
> > On Thu, Apr 13, 2017 at 2:43 PM, upendar devu <[hidden email]>
> > wrote:
> >
> >> Including Dev Team to address my queries
> >>
> >> On Thu, Apr 13, 2017 at 2:39 PM, upendar devu <[hidden email]>
> >> wrote:
> >>
> >>> We are using zookeeper 3.4.6 version with integration of Apache Kafka ,
> >>> Apache Storm and Zookeeper.
> >>>
> >>> I would like to understand the following things. Please help me to
> >>> clarify.
> >>>
> >>> 1. What are Integration dependencies of Zookeeper ?
> >>>    Since we are using Kafka, Storm , so we need Zookeeper ? which
> >>> external integration has dependencies
> >>>
> >>> 2. What is the actual role of zookeeper with the integration of Apache
> >>> Kafka,Apache Storm.  Could you please provide a better use case to
> >>> understand.
> >>>
> >>> 3. We are planning to upgrade Zookeeper version to 3.4.10 due to CVE (
> >>> CVE-2016-5017) mentioned with version 3.4.6 ;  Do we also need to
> >>> upgrade depedency integration components like Apache Storm(*using
> >>> 0.10.0*) and Apache Kafka (Using *0.8.1.1*) ? which version those
> >>> should be upgraded to  ?
> >>>
> >>>
> >>> Thanks
> >>> Upendar
> >>>
> >>
> >>
> >
>



--
Cheers
Michael.
Loading...