Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Unable to Create entity in Atlas

Labels:
avatar
author-rank muja_owez
Explorer

Created ‎11-28-2018 03:02 PM

Hi Team,

We are using HDP-2.6.5.0 and Ambari-2.6.2.0. While creating new entity in Atlas we are getting following error in application.log. Currently we are not using Ranger. we trying to create new entity without ranger. 

Caused by: org.apache.atlas.notification.NotificationException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_ENTITIES-0: 30009 ms has passed since batch creation plus linger time
        at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:220)
        at org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:182)
        at org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:117)
        at org.apache.atlas.notification.NotificationEntityChangeListener.notifyOfEntityEvent(NotificationEntityChangeListener.java:208)
        at org.apache.atlas.notification.NotificationEntityChangeListener.onEntitiesUpdated(NotificationEntityChangeListener.java:102)

        at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.notifyListeners(AtlasEntityC

hangeNotifier.java:176)
        ... 100 more
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_ENTITIES-0: 30009 ms has passed since batch creation plus linger time
        at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:94)
        at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:64)
        at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:29)
        at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:207)
        ... 105 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_ENTITIES-0: 30009 ms has passed since batch creation plus linger time
2018-11-28 01:35:02,638 DEBUG - [pool-2-thread-10 - 2b585636-30dd-4484-a6a6-e4c05ae50167:] ~ ==> releaseLockedObjects(): lockedGuids.size: 0 (GraphTransactionInterceptor:187)
2018-11-28 01:35:02,638 DEBUG - [pool-2-thread-10 - 2b585636-30dd-4484-a6a6-e4c05ae50167:] ~ <== releaseLockedObjects(): lockedGuids.size: 0 (GraphTransactionInterceptor:197)
2018-11-28 01:35:02,638 DEBUG - [pool-2-thread-10 - 2b585636-30dd-4484-a6a6-e4c05ae50167:] ~ PERF|EntityREST.createOrUpdate()|30228 (EntityREST:77)
2018-11-28 01:35:02,638 ERROR - [pool-2-thread-10 - 2b585636-30dd-4484-a6a6-e4c05ae50167:] ~ Error handling a request: 168f59a001543090 (ExceptionMapperUtil:32)
org.apache.atlas.exception.AtlasBaseException: Failed to notify NotificationEntityChangeListener for change UPDATE
        at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.notifyListeners(AtlasEntityChangeNotifier.java:183)

I can describe topic for ATLAS_ENTITIES

/usr/hdp/2.6.5.0-292/kafka/bin/kafka-topics.sh --describe --topic ATLAS_ENTITIES --zookeeper hadmgrndcc03-1.lifeway.org:2181,hadmgrndcc03-2.lifeway.org:2181,hadmgrndcc03-3.lifeway.org:2181
Topic:ATLAS_ENTITIES    PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: ATLAS_ENTITIES   Partition: 0    Leader: 1003    Replicas: 1003  Isr: 1003
ACL Permission:-
 /usr/hdp/2.6.5.0-292/kafka/bin/kafka-acls.sh --add --allow-principal user:* --producer --topic ATLAS_ENTITIES --authorizer-properties zookeeper.connect=hadmgrndcc03-1.lifeway.org:2181,hadmgrndcc03-2.lifeway.org:2181,hadmgrndcc03-3.lifeway.org:2181
Adding ACLs for resource `Topic:ATLAS_ENTITIES`:
        user:* has Allow permission for operations: Write from hosts: *
        user:* has Allow permission for operations: Describe from hosts: *
Adding ACLs for resource `Cluster:kafka-cluster`:
        user:* has Allow permission for operations: Create from hosts: *
Current ACLs for resource `Topic:ATLAS_ENTITIES`:
        User: has Allow permission for operations: Read from hosts: *
        User: has Allow permission for operations: Describe from hosts: *
        user:* has Allow permission for operations: Write from hosts: *
        user:* has Allow permission for operations: Describe from hosts: *
        User:* has Allow permission for operations: Write from hosts: *
        user:* has Allow permission for operations: Write from hosts: application.log
        User:* has Allow permission for operations: Describe from hosts: *
        User:atlas has Allow permission for operations: Write from hosts: *
        user:* has Allow permission for operations: Read from hosts: application.log
        User:* has Allow permission for operations: Read from hosts: *
        User:atlas has Allow permission for operations: Describe from hosts: *

Could you please help in this issue and provide a solution for it.

Thanks in advance.

Regards,

Owez Mujawar

6,587 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank muja_owez
Explorer

Created ‎12-03-2018 05:52 AM

@Sarath Subramanian

now i am able publish a sample message to ATLAS_ENTITIES.

Resolution:

Deleted the older Atlas_entities and created a new one.

Thanks for the help.

Regards,

Owez Mujawar

View solution in original post

6,109 Views
0 Kudos
6 REPLIES 6

avatar
author-rank ssubramanian author-rank
Expert Contributor

Created ‎11-28-2018 07:13 PM

@Owez Mujawar

You see the NotificationException because Atlas is unable to send a message to kafka topic ATLAS_ENTITIES. Can you check if Kafka service is healthy? Another possible reason is when a new kafka broker was added and assigned leader and atlas doesn't know about it. To isolate this issue, can you try publishing a sample message to ATLAS_ENTITIES topic as atlas user and see if you see any permission issues.

6,109 Views
0 Kudos

avatar
author-rank muja_owez
Explorer

Created ‎11-29-2018 02:20 PM

@Sarath Subramanian

Hi,

As we are trying to execute the query but we are failing. below error for the same.

could you please help me to resolved this issue.

atlas@hadwkrndcc03-3:/var/log/atlas$ /usr/hdp/current/atlas-server/bin/quick_start.py
Enter username for atlas :- admin
Enter password for atlas :-
Creating sample types:
Created type [DB]
Created type [Table]
Created type [StorageDesc]
Created type [Column]
Created type [LoadProcess]
Created type [View]
Created type [JdbcAccess]
Created type [ETL]
Created type [Metric]
Created type [PII]
Created type [Fact]
Created type [Dimension]
Created type [Log Data]
Creating sample entities:
Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@514646ef failed with status 500 (Internal Server Error) Response Body ({"errorCode":"ATLAS-500-00-007","errorMessage":"Failed to notify NotificationEntityChangeListener for change CREATE"})
        at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:387)
        at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:320)
        at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:208)
        at org.apache.atlas.AtlasClientV2.createEntity(AtlasClientV2.java:281)
        at org.apache.atlas.examples.QuickStartV2.createInstance(QuickStartV2.java:339)
        at org.apache.atlas.examples.QuickStartV2.createDatabase(QuickStartV2.java:362)
        at org.apache.atlas.examples.QuickStartV2.createEntities(QuickStartV2.java:268)
        at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:150)
        at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132)
No sample data added to Apache Atlas Server.

Thanks in advance.

Regards,

Owez Mujawar

6,109 Views
0 Kudos

avatar
author-rank muja_owez
Explorer

Created ‎11-29-2018 03:06 PM

@Sarath Subramanian

Please find below issue while publishing message to ATLAS_ENTITIES.

atlas@hadwkrndcc03-3:/var/log/atlas$ /usr/hdp/2.6.5.0-292/kafka/bin/kafka-console-producer.sh --broker-list hadwkrndcc03-3.lifeway.org:6667 --topic ATLAS_ENTITIES
>olla
>[2018-11-29 05:50:15,694] ERROR Error when sending message to topic ATLAS_ENTITIES with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_ENTITIES-0: 1526 ms has passed since batch creation plus linger time

Could you please help in this issue and provide a solution for it.

Thanks in advance.

Regards,

Owez Mujawar

6,109 Views
0 Kudos

avatar
author-rank ssubramanian author-rank
Expert Contributor

Created ‎11-29-2018 08:59 PM

@Owez Mujawar

So you see the same exception when you try to publish a sample message to ATLAS_ENTITIES topic as atlas user. Some kafka config has been messed up - kafka brokers have been added or removed.

Please follow https://community.hortonworks.com/content/supportkb/150063/how-to-add-new-kafka-brokers-to-a-cluster...

to reassign ATLAS_ENTITIES/ATLAS_HOOK topic with new broker information.

6,109 Views
0 Kudos

avatar
author-rank ssubramanian author-rank
Expert Contributor

Created ‎11-29-2018 08:59 PM

@Owez Mujawar

So you see the same exception when you try to publish a sample message to ATLAS_ENTITIES topic as atlas user. Some kafka config has been messed up - kafka brokers have been added or removed.

Please follow https://community.hortonworks.com/content/supportkb/150063/how-to-add-new-kafka-brokers-to-a-cluster...

to reassign ATLAS_ENTITIES/ATLAS_HOOK topic with new broker information.

6,109 Views
0 Kudos

avatar
author-rank muja_owez
Explorer

Created ‎12-03-2018 05:52 AM

@Sarath Subramanian

now i am able publish a sample message to ATLAS_ENTITIES.

Resolution:

Deleted the older Atlas_entities and created a new one.

Thanks for the help.

Regards,

Owez Mujawar

6,110 Views
0 Kudos
Announcements
What's New @ Cloudera
[RELEASED] Cloudera Streams Messaging - Kubernetes Operator ...
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics 1.14 for Cloudera Pu...
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
View More Announcements