Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas failed to start embedded kafka

Atlas failed to start embedded kafka

Explorer

15415-atlaserror.png

11 REPLIES 11

Re: Atlas failed to start embedded kafka

Expert Contributor

@Shafi Ahmad There is a dependency on HBase (Storage), Kafka (Tag Sync), and Ambari Infra (Solr - Graph Index search, depends on the version of HDP) already running before starting Atlas.

Atlas does not start Kafka, HBase, Ambari Infra (when done in Ambari).

Re: Atlas failed to start embedded kafka

Explorer

yes hbase,kafka and ambari infra is running..and work fine..but i m getting error in atlas only.

Highlighted

Re: Atlas failed to start embedded kafka

@Shafi Ahmad

The issue is due to wrong value provided for this "atlas.kafka.zookeeper.connect" property. The exception you are showing in your question is clearly saying that, MalformedURLException: For input string: "2181, localhost:2181,localhost:2181".

So, fixing this property value should get your atlas up and running. Let me know if it works.

Re: Atlas failed to start embedded kafka

Explorer

thank you for response.

I have mention there with hostname:2181,hostname:2181,hostname:2181 ..i think which is right value.and the error is still same.let me know if i did mistake and help me to fix this issue.or please provide me any link which ll to fix this issue.

Re: Atlas failed to start embedded kafka

Pls. share your atlas-application.properties here.

Re: Atlas failed to start embedded kafka

Explorer
# Generated by Apache Ambari. Thu May 18 02:42:00 2017
atlas.audit.hbase.tablename=ATLAS_ENTITY_AUDIT_EVENTS
atlas.audit.hbase.zookeeper.quorum=localhost4,localhost2,localhost3
atlas.audit.zookeeper.session.timeout.ms=1500
atlas.auth.policy.file=/etc/atlas/conf/policy-store.txt
atlas.authentication.keytab=/etc/security/keytabs/atlas.service.keytab
atlas.authentication.method.file=true
atlas.authentication.method.file.filename=/etc/atlas/conf/users-credentials.properties
atlas.authentication.method.kerberos=false
atlas.authentication.method.ldap=false
atlas.authentication.method.ldap.ad.base.dn=
atlas.authentication.method.ldap.ad.bind.dn=
atlas.authentication.method.ldap.ad.bind.password=
atlas.authentication.method.ldap.ad.default.role=ROLE_USER
atlas.authentication.method.ldap.ad.domain=
atlas.authentication.method.ldap.ad.referral=ignore
atlas.authentication.method.ldap.ad.url=
atlas.authentication.method.ldap.ad.user.searchfilter=(sAMAccountName={0})
atlas.authentication.method.ldap.base.dn=
atlas.authentication.method.ldap.bind.dn=
atlas.authentication.method.ldap.bind.password=
atlas.authentication.method.ldap.default.role=ROLE_USER
atlas.authentication.method.ldap.groupRoleAttribute=cn
atlas.authentication.method.ldap.groupSearchBase=
atlas.authentication.method.ldap.groupSearchFilter=
atlas.authentication.method.ldap.referral=ignore
atlas.authentication.method.ldap.type=none
atlas.authentication.method.ldap.url=
atlas.authentication.method.ldap.user.searchfilter=
atlas.authentication.method.ldap.userDNpattern=uid=
atlas.authentication.principal=atlas
atlas.authorizer.impl=simple
atlas.client.connectTimeoutMSecs=60000
atlas.client.readTimeoutMSecs=60000
atlas.cluster.name=hadoop
atlas.enableTLS=true
atlas.graph.index.search.backend=solr5
atlas.graph.index.search.solr.mode=cloud
atlas.graph.index.search.solr.zookeeper-url=localhost3:2181/infra-solr,localhost2:2181/infra-solr,localhost1/infra-solr
atlas.graph.storage.backend=hbase
atlas.graph.storage.hbase.table=atlas_titan
atlas.graph.storage.hostname=localhost4,localhost2,localhost3
atlas.kafka.auto.commit.enable=true
atlas.kafka.auto.commit.interval.ms=1000
atlas.kafka.bootstrap.servers=localhost2:6667,localhost3:9092
atlas.kafka.data==${sys:atlas.home}/data/kafka
atlas.kafka.hook.group.id=atlas
atlas.kafka.zookeeper.connect=localhost2:2181,localhost3:2181,localhost4:2181
atlas.kafka.zookeeper.connection.timeout.ms=250
atlas.kafka.zookeeper.session.timeout.ms=450
atlas.kafka.zookeeper.sync.time.ms=25
atlas.lineage.schema.query.hive_table=hive_table where __guid='%s'\, columns
atlas.lineage.schema.query.Table=Table where __guid='%s'\, columns
atlas.notification.create.topics=true
atlas.notification.embedded=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.rest.address=https://localhost2:21443
atlas.server.address.id1=localhost2:21443
atlas.server.bind.address=localhost2
atlas.server.ha.enabled=false
atlas.server.http.port=21000
atlas.server.https.port=21443
atlas.server.ids=id1
atlas.solr.kerberos.enable=false
cert.stores.credential.provider.path=jceks://file/atlas.jceks
client.auth.enabled=false
keystore.file=/etc/cert/keystore.jks
truststore.file=/etc/cert/keystore.jks

Re: Atlas failed to start embedded kafka

Expert Contributor
@Shafi Ahmad

Which HDP version you using, normally HDP use external kafka, did you change the atlas config to use embeded kafka ?

Can you attached atlas-application.properties from /etc/atlas/conf/

Nixon

Re: Atlas failed to start embedded kafka

Explorer
# Generated by Apache Ambari. Thu May 18 02:42:00 2017


atlas.audit.hbase.tablename=ATLAS_ENTITY_AUDIT_EVENTS
atlas.audit.hbase.zookeeper.quorum=localhost4,localhost2,localhost3
atlas.audit.zookeeper.session.timeout.ms=1500
atlas.auth.policy.file=/etc/atlas/conf/policy-store.txt
atlas.authentication.keytab=/etc/security/keytabs/atlas.service.keytab
atlas.authentication.method.file=true
atlas.authentication.method.file.filename=/etc/atlas/conf/users-credentials.properties
atlas.authentication.method.kerberos=false
atlas.authentication.method.ldap=false
atlas.authentication.method.ldap.ad.base.dn=
atlas.authentication.method.ldap.ad.bind.dn=
atlas.authentication.method.ldap.ad.bind.password=
atlas.authentication.method.ldap.ad.default.role=ROLE_USER
atlas.authentication.method.ldap.ad.domain=
atlas.authentication.method.ldap.ad.referral=ignore
atlas.authentication.method.ldap.ad.url=
atlas.authentication.method.ldap.ad.user.searchfilter=(sAMAccountName={0})
atlas.authentication.method.ldap.base.dn=
atlas.authentication.method.ldap.bind.dn=
atlas.authentication.method.ldap.bind.password=
atlas.authentication.method.ldap.default.role=ROLE_USER
atlas.authentication.method.ldap.groupRoleAttribute=cn
atlas.authentication.method.ldap.groupSearchBase=
atlas.authentication.method.ldap.groupSearchFilter=
atlas.authentication.method.ldap.referral=ignore
atlas.authentication.method.ldap.type=none
atlas.authentication.method.ldap.url=
atlas.authentication.method.ldap.user.searchfilter=
atlas.authentication.method.ldap.userDNpattern=uid=
atlas.authentication.principal=atlas
atlas.authorizer.impl=simple
atlas.client.connectTimeoutMSecs=60000
atlas.client.readTimeoutMSecs=60000
atlas.cluster.name=hadoop
atlas.enableTLS=true
atlas.graph.index.search.backend=solr5
atlas.graph.index.search.solr.mode=cloud
atlas.graph.index.search.solr.zookeeper-url=localhost3:2181/infra-solr,localhost2:2181/infra-solr,localhost1/infra-solr
atlas.graph.storage.backend=hbase
atlas.graph.storage.hbase.table=atlas_titan
atlas.graph.storage.hostname=localhost4,localhost2,localhost3
atlas.kafka.auto.commit.enable=true
atlas.kafka.auto.commit.interval.ms=1000
atlas.kafka.bootstrap.servers=localhost2:6667,localhost3:9092
atlas.kafka.data==${sys:atlas.home}/data/kafka
atlas.kafka.hook.group.id=atlas
atlas.kafka.zookeeper.connect=localhost2:2181,localhost3:2181,localhost4:2181
atlas.kafka.zookeeper.connection.timeout.ms=250
atlas.kafka.zookeeper.session.timeout.ms=450
atlas.kafka.zookeeper.sync.time.ms=25
atlas.lineage.schema.query.hive_table=hive_table where __guid='%s'\, columns
atlas.lineage.schema.query.Table=Table where __guid='%s'\, columns
atlas.notification.create.topics=true
atlas.notification.embedded=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.rest.address=https://localhost2:21443
atlas.server.address.id1=localhost2:21443
atlas.server.bind.address=localhost2
atlas.server.ha.enabled=false
atlas.server.http.port=21000
atlas.server.https.port=21443
atlas.server.ids=id1
atlas.solr.kerberos.enable=false
cert.stores.credential.provider.path=jceks://file/atlas.jceks
client.auth.enabled=false
keystore.file=/etc/cert/keystore.jks
truststore.file=/etc/cert/keystore.jks



Re: Atlas failed to start embedded kafka

Expert Contributor

@Shafi Ahmad

I reviewed the atlas-application properties file

From the atlas-application properties it looks you are trying to using embeded kafka

Do you intent to use embeded kafka or external Kafka for Atlas ?

the below property is confusing

atlas.kafka.bootstrap.servers=localhost2:6667,localhost3:9092

I can suggest you properties depending on whether it embeded kafka or external Kafka for Atlas.

Let me know