Reply
New Contributor
Posts: 4
Registered: ‎01-06-2016

Adding a Kafka service using Parcel CDH 5.4.8

Kafka Parcel distributed and activated succesfully.  While adding sevice (kafka), process fails with error:

 

"Error found before invoking supervisord: User [kafka] does not exist:"     

 

Already verified this option is checked  on parcel configuration screen,

 

CM > Administration > Parcels > Create Users and Groups, and Apply File Permissions for Parcels

 

 

However I can see in cloudera-scm-agent.log,   create user and groups failing (below lines). 

 

 

 

[29/Sep/2016 23:23:13 +0000] 78149 Thread-13 parcel_cache INFO Unpack of parcel /opt/cloudera/parcel-cache/KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel successful
[29/Sep/2016 23:23:13 +0000] 78149 Thread-13 downloader INFO Finished download [ url: http://is-hadoop1nb.gwl.com:7180/cmf/parcel/download/KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel, state: complete, total_bytes: 32378038, downloaded_bytes: 32378038, start_time: 2016-09-29 23:23:13, download_end_time: 2016-09-29 23:23:13, end_time: 2016-09-29 23:23:13, code: 200, exception: None, path: /opt/cloudera/parcel-cache/KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel ]
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Loading parcel manifest for: CDH-5.4.8-1.cdh5.4.8.p0.4
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Loading parcel manifest for: KAFKA-2.0.2-1.2.0.2.p0.5
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Loading parcel manifest for: KEYTRUSTEE_SERVER-5.4.3-1.keytrustee5.4.3.p0.230
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Ensuring users/groups exist for new parcel KAFKA-2.0.2-1.2.0.2.p0.5.
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Executing command ['/usr/sbin/groupadd', '-r', 'kafka']
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel ERROR Failed to create group kafka for parcel KAFKA-2.0.2-1.2.0.2.p0.5: 4
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Executing command ['/usr/sbin/groupadd', '-r', 'kafka']
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel ERROR Failed to create group kafka for parcel KAFKA-2.0.2-1.2.0.2.p0.5: 4
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Executing command ['/usr/sbin/useradd', '-r', '-m', '-g', 'kafka', '-K', 'UMASK=022', '--home', '/var/lib/kafka', '--comment', 'Kafka', '--shell', '/sbin/nologin', 'kafka']
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel ERROR Failed to create user kafka for parcel KAFKA-2.0.2-1.2.0.2.p0.5: 6
[29/Sep/2016 23:23:28 +0000] 78149 MainThread parcel INFO Ensuring correct file permissions for new parcel KAFKA-2.0.2-1.2.0.2.p0.5.
[29/Sep/2016 23:23:50 +0000] 78149 CP Server Thread-7 _cplogging INFO 143.199.102.180 - - [29/Sep/2016:23:23:50] "GET /heartbeat HTTP/1.1" 200 2 "" "NING/1.0"

 

Any suggetions on this error?

Cloudera Employee
Posts: 277
Registered: ‎01-09-2014

Re: Adding a Kafka service using Parcel CDH 5.4.8

Can you please confirm what version of Cloudera Manager you are running?

 

Kafka 2.0.2 requires CM 5.5.3 and above: https://www.cloudera.com/documentation/kafka/latest/topics/kafka_requirements.html

 

-pd

Posts: 1,128
Topics: 1
Kudos: 290
Solutions: 137
Registered: ‎04-22-2014

Re: Adding a Kafka service using Parcel CDH 5.4.8

The inability to create users indicates a problem on that host.  I recommend using the following to manually run the commands on the host where the kafka role is not starting:

 

# /usr/sbin/groupadd -r kafka

# /usr/sbin/useradd -r -m -g kafka -K UMASK=022 --home /var/lib/kafka --comment Kafka --shell /sbin/nologin kafka

 

Running them manually should highlight why they failed.

 

Regards,

 

Ben