Member since
09-11-2015
269
Posts
281
Kudos Received
55
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3307 | 03-15-2017 07:12 AM | |
1875 | 03-14-2017 07:08 PM | |
2322 | 03-14-2017 03:36 PM | |
1910 | 02-28-2017 04:32 PM | |
1366 | 02-28-2017 10:02 AM |
10-12-2016
11:45 AM
Glad that you tried resetting the property to check the actual issue. I think the actual issue was, the way you tried to access the hive entities from UI. You could have tried DSL search like below.
... View more
10-12-2016
11:16 AM
1 Kudo
@Floris Smit From the atlas application log, the metadata is consumed by atlas and everything looks fine.. 2016-10-12 10:05:16,487 INFO - [NotificationHookConsumer thread-0:] ~ Updated entities: {
"entities": {
"updated": [
"52bf585a-8e66-4402-864a-84bc201daffc",
"f67d81d9-4e2e-4396-99cf-2f2166e9fd83",
"43c64fde-38c6-4dbc-b75e-86d804e619c0",
"f5cdd132-0066-4a37-82a3-837d9316650d",
"53015e6b-2588-4c3c-925b-1cc2596eebc5",
"3ae5d014-147e-468d-a442-e4253d647fdc",
"ae0dacc8-66ac-4c5f-8673-35a3d1ca678c",
"fdc38fcc-edb9-48f6-a2f8-282a9032793a",
"c33fc14c-7aeb-45cb-b933-8e3c3d9f5a6e",
"52c78367-162c-4e91-b5f1-5335c8f2ad16",
"59af504a-192c-4814-b657-80e3c3a5bb8b",
"367ea90f-4bb7-4fc8-a43a-0fba1250cd29"
]
}
} (EntityResource:210) Even the hive logs reflect the same. Also, atlas.hook.hive.synchronous is setting used to publish messages at the user convenience(sync/async). In your case, even after setting atlas.hook.hive.synchronous to false, the metadata should have been published to atlas. atlas.hook.hive.synchronous - boolean, true to run the hook synchronously. default false. Recommended to be set to false to avoid delays in hive query completion. source: http://atlas.incubator.apache.org/Bridge-Hive.html
... View more
10-12-2016
09:32 AM
2 Kudos
@Floris Smit Can you please share the snapshot(attach the log) of the atlas application log? Also, try creating a sample table in hive and capture the hiveserver2.log and atlas application log for the sample table creation time. Please paste this log as well. Above logs are needed for debugging. Also, please check if all the dependent services are up and running(HBase(regionservers), Kafka, Ambari-infra).
... View more
10-11-2016
05:36 AM
@Mohan V Not sure what went wrong while creating the topic. It could be related to disabling kerberos, as this might have some stale bits. Can you check if the topic is actually created using kafka command line tools? Also, try creating the topic using the FQDN(zookeeper quorum) for zookeeper instead of localhost. for example: <strong>./kafka-topics.sh --create --zookeeper <host1:port>,<host2:port> --replication-factor 1 --partitions 1 --topic test</strong> If none of the above helps, can you try deploying a new cluster without kerberos if possible?
... View more
10-10-2016
03:35 PM
1 Kudo
@Mohan V
Can you please provide more details on
how you created the topic(the exact command)? Do you have permissions to access that topic?
Is ranger enabled on the cluster? - if yes, please check if you have policy for enabling permissions to create kafka topic. Is the cluster kerberized? If yes - Are you performing the producer operations as a proper user with proper 'kinit'? There is a similar question asked in the community, please check this also if it helps(https://community.hortonworks.com/questions/42793/kafka-producer-error-no-partition-metadata-for-top.html)..
... View more
10-08-2016
12:50 AM
1 Kudo
@Sunile Manjee From the stack trace, it seems like Atlas is not able to write to ATLAS_ENTITIES topic. Since you have already done a check for topic existence, can you please provide more details about your cluster? Is it a secure cluster? If yes, does it have ranger enabled? If yes, what policies are configured for both kafka topics? If no, do you have proper ACL/permissions configured for Atlas to read/write to the topics? Also, can you check if anything is written to the topic ATLAS_ENTITIES using below command? /usr/bin/kafka-console-consumer --zookeeper <zk_host> --topic <topic_name> --from-beginning Kafka however advertises its "hostname" to producers and consumers to use, that name needs to resolve from where you're connecting. Also, check advertised port numbers are accessible.
... View more
10-08-2016
12:13 AM
1 Kudo
@Michael Young Thanks for sharing your findings, this is helpful.
... View more
10-07-2016
02:47 PM
2 Kudos
@Michael Young Regarding your question - "Is there anything shared between docker containers?", I was just reading an article on the same and found that.. Your data doesn’t live in the container, it lives in a named volume that is shared between 1-N containers that you define. You backup the data volume, and forget about the container. Optimallyyour containers are completely stateless and immutable. Source: https://blog.docker.com/2016/03/containers-are-not-vms/ More indepth details: https://docs.docker.com/engine/userguide/storagedriver/imagesandcontainers/ Not sure, if this answers your question but thought to add it here.
... View more
10-07-2016
01:41 PM
3 Kudos
@Michael Young To remove all the types/metadata from metadata store, please follow hte below steps.. Stop Atlas Disable hbase table 'atlas_titan' - Hope you dont have any metadata there.. Drop hbase table 'atlas_titan' Restart Atlas This should actually remove all the existing types from backend store(hbase).
... View more
10-03-2016
06:18 AM
@Sundar Lakshmanan Is this issue resolved?
... View more