I'm not able to see my tables with Atlas

nisrine001 · ‎07-20-2018

I upgraded my HDP TO HDP-2.6 and all my services works but i can't see any new created tables neither the databases with altas so i tried to import-hive.sh.

i'm not sure that how it works so What should i do? any suggesion please

thanks you

JonathanSneep · ‎07-20-2018

Did you already run import-hive.sh? Could you share the output of the shell script if you have?

This is a good article showing how to make use of import-hive: https://community.hortonworks.com/articles/61274/import-hive-metadata-into-atlas.html

JonathanSneep · ‎07-20-2018

@nisrine elloumi
Would you be able to share some output of running the hive-import and command syntax you've used?
Please note the bottom piece of the article I previously linked, where it explains the additional parameters required to specify for the shell script when running the import on a kerberized cluster.

nisrine001 · ‎07-22-2018

@Jonathan Sneep

Using Hive configuration directory [/etc/hive/conf]

Log file for import is /usr/hdp/current/atlas-server/logs/import-hive.log

log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.

log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.

Exception in thread "main" org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed.

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650)

Caused by: org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClient$API_V1@2807bdebfailed with status 403 (Forbidden) Response Body (<html>

<head>

<title>Error 403 {"AuthorizationError":"You are not authorized for READ on [ENTITY] : *"}</title>

</head>

<body><h2>HTTP ERROR 403</h2>

<p>Problem accessing /api/atlas/entities. Reason:

<pre> {"AuthorizationError":"You are not authorized for READ on [ENTITY] : *"}</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>

<br/>

</body>

</html>

)

at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:387)

at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:320)

at org.apache.atlas.AtlasBaseClient.callAPIWithRetries(AtlasBaseClient.java:471)

at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:967)

at org.apache.atlas.AtlasClient.getEntity(AtlasClient.java:651)

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReference(HiveMetaStoreBridge.java:352)

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseReference(HiveMetaStoreBridge.java:230)

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:165)

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:140)

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134)

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647)

Failed to import Hive Data Model!!!

JonathanSneep · ‎07-23-2018

Hi @nisrine elloumi
it looks as if there may be issues with the ranger permissions causing "You are not authorized for READ on [ENTITY]". We have to be sure there is a policy in ranger for the user who is running the import script (with a secure environment, the import script will run as the user executing the script) that gives appropriate permissions for the user.

nisrine001 · ‎07-23-2018

hello @Jonathan Sneep thanks for your reponse

i added the following with ranger in Kafka policies:

topic=ATLAS_HOOK
permission=publish, create; group=public
permission=consume, create; user=atlas (for non-kerberized environments, set group=public)
topic=ATLAS_ENTITIES
permission=publish, create; user=atlas (for non-kerberized environments, set group=public)
permission=consume, create; group=public

nisrine001 · ‎07-20-2018

hi thank you @Jonathan Sneep

yes i run import-hive.sh with kerberos but i have a problem it demande my Kerberos username and password and that weird !!!!

nisrine001 · ‎07-20-2018

Yes @Jonathan Sneep i followed your linked article with running this commande

/usr/hdp/current/atlas-server/hook-bin/import-hive.sh -Dsun.security.jgss.debug=true -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/etc/atlas/2.6.4.0-91/0/atlas_jaas.conf

nisrine001 · ‎07-20-2018

Using Hive configuration directory [/etc/hive/conf]

Log file for import is /usr/hdp/current/atlas-server/logs/import-hive.log

log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.

log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.

Kerberos username [root]:

Kerberos password for -Djava.security.auth.login.config=/etc/atlas/2.6.4.0-91/0/atlas_jaas.conf:

Kerberos username [root]:

nisrine001 · ‎07-20-2018

@Jonathan Sneep

Using Hive configuration directory [/etc/hive/conf]

Log file for import is /usr/hdp/current/atlas-server/logs/import-hive.log

log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.

log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.

Exception in thread "main" org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed.

at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650)

Caused by: org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClient$API_V1@2807bdeb failed with status 403 (Forbidden) Response Body (<html>