Atlas rest api timeout

[#1]

I am creating new entity in Atlas using rest api. It seems to timeout out and spits out kafka timeout messages. Any ideas?

$ curl -X POST -d @entity.json1 -u admin:admin -H 'Content-Type: application/json; charset=UTF-8' x.x.x.x:21000/api/atlas/entities {"error":"java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.","stackTrace":"org.apache.atlas.notification.NotificationException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.\n\tat org.apache.atlas.kafka.KafkaNo

This is on sandbox. Do I need to have a Kafka instance up or does Atlas have its own?

[#2 · Accepted solution]

@Sunile Manjee

Atlas works with both embedded as well as standalone Kafka, it depends on how Atlas is configured.

When Atlas config property "atlas.notification.embedded" is set to true, Kafka is started in embedded mode(as part of Atlas process) else it expects a standalone Kafka running in your cluster.

Regarding the Kafka timeout exception, it seems like topic does not exists. So please create the topics like below.

$KAFKA_HOME/bin/kafka-topics.sh --zookeeper $ZK_ENDPOINT --topic ATLAS_HOOK --create --partitions 1 --replication-factor $KAFKA_REPL_FACTOR
$KAFKA_HOME/bin/kafka-topics.sh --zookeeper $ZK_ENDPOINT --topic ATLAS_ENTITIES --create --partitions 1 --replication-factor $KAFKA_REPL_FACTOR

Let me know if you are still seeing this issue. Hope this helps.

[#3]

@Ayub Pathan I assume created the topic above is using non-embedded mode. What if I am using embbeded?

[#4]

In case of embedded kafka, the auto create of topics is a configuration parameter in Kaka, which is set to true by default. With this setting, when a producer produces message for the first time, Kafka will automatically create the topic. No need for anything specific to be done by the client.

[#5]

@Sunile Manjee Is this resolved for you?

[#6]

Hi,

How did you solve this problem? I am facing the same issue.

Thank you ...