Member since
10-23-2018
43
Posts
0
Kudos Received
0
Solutions
12-20-2018
03:09 PM
Hello I'm an newbie with kerberos and trying to achieve a connection with kafka-kerberos. So, i get this issue "SASL authentication failure: Disconnected" while running my python code whitch in my producer i specifie the sasl.kerberos.principal, sasl.kerberos.keytab, sasl.kerberos.service.name as kafka, security.protocol as SASL_PLAINTEXT and the sasl.mechanisms as GSSAPI Any suggesion. thank you in advance
... View more
Labels:
12-20-2018
01:36 PM
Hello I'm using kerberos with lib "confluent_kafka_python" . So, i get this issue while running my code : SASL authentication
failure: Disconnected and my code is : from time import sleep
from time import sleep
from json import dumps
#from kafka import KafkaProducer
from confluent_kafka import Producer
import logging
logging.basicConfig(level=logging.DEBUG)
p= Producer({
'bootstrap.servers': '..........',
'sasl.kerberos.principal': 'svchdp-admin@AP.CDC.FR',
'sasl.kerberos.keytab': '/etc/security/keytabs/svchdp-admin.keytab',
'sasl.kerberos.service.name': 'kafka',
'security.protocol': 'SASL_PLAINTEXT',
'sasl.mechanisms': 'GSSAPI',
'debug': 'security'
})
... View more
Labels:
12-10-2018
04:40 PM
Thank you @Bryan Bende for this response My ConsumeKafka_0_10 works fine and i obviously don't have an errors like @Eric Lloyd but i can't see my data in the processor. Is that what we call back-pressure ? could this can be resolved with increasing the Maximum Timer Driven Thread Count and Maximum Event Drive Thread Count ? what does that means ? Could you please give some suggesion. Thanks you in advance.
... View more
11-26-2018
06:21 PM
Hello I found a problem while using rsyslog (Omkafka) to send log to kafka. So i search in the logs and i found this. Any idea,any suggesion Please. [2018-11-26 17:06:29,702] INFO [GroupCoordinator 1001]: Preparing to restabilize group console-consumer-65890 with old generation 1 (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:06:29,702] INFO [GroupCoordinator 1001]: Group console-consumer-65890 with generation 2 is now empty (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:06:56,563] INFO [GroupCoordinator 1001]: Preparing to restabilize group console-consumer-95312 with old generation 0 (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:06:56,563] INFO [GroupCoordinator 1001]: Stabilized group console-consumer-95312 generation 1 (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:06:56,566] INFO [GroupCoordinator 1001]: Assignment received from leader for group console-consumer-95312 for generation 1 (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:07:01,405] INFO [GroupCoordinator 1001]: Preparing to restabilize group console-consumer-95312 with old generation 1 (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:07:01,406] INFO [GroupCoordinator 1001]: Group console-consumer-95312 with generation 2 is now empty (kafka.coordinator.GroupCoordinator)
[2018-11-26 17:10:42,034] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-11-26 17:20:42,034] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-11-26 17:30:42,034] INFO [Group Metadata Manager on Broker 1001]: Removed 0 expired offsets in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
... View more
Labels:
11-22-2018
01:27 PM
Hello @Bryan Bende @Timothy Spann I'm a newbee to syslog and i wanna use it with kafka. Can i have any idea how it works. Please, regards.
... View more
11-14-2018
10:17 AM
Hi, I'm an newbie with kafka connect and i wanna launch it with many servers mode. I already done an exemple with the standalone mode but now i wanna extract source "logs" from another server (it is site to site exemple). Both of my servers have a kafka-brokers and that one for source and the other for a sink. Any suggesion and Thank you in advance.
... View more
Labels:
11-06-2018
10:46 AM
Hello. I have in issue with NIFI and MINIFI connection. 1- I installed NIFI 1.0.0 with 3.0 HDP with a secure properties and CA. 2- I wanna use a NIFI agent so i used this tuto https://nifi.apache.org/minifi/minifi-java-agent-quick-start.html and connecting to NIFI using https://nifi.apache.org/minifi/getting-started.html but i get this errors in minifi logs when starting MINIFI: WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@ Unable to refresh Remote Group's peers due to sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target so MINIFI Couldn't find a valid peer to communicate with for my Remote Process Group 3- Is MINIFI need a Self Signed Certificates ? how could i use site to site connection ? Any suggesion. Thank you in advance.
... View more
Labels:
11-05-2018
04:10 PM
Hello @Abhinav Joshi and @Andy LoPresto I wanna install minifi and i already installed NIFI. The issue is minifi can't connect to nifi due to TLShandshake problem with nifi whom already hev a keystore.jks et truststore.jks. Any suggesion Thanks you in advance
... View more
10-19-2018
12:24 PM
Hello I have the same issue with creating taxonemy (create Sub term) did any of you find any solution for this problem? thank you
... View more
10-17-2018
05:11 PM
Hello, this application.log 2018-10-17 17:39:38,095 INFO - [pool-2-thread-29 - 47593d06-ea3b-4f9d-ae54-1f52ed931dfc:] ~ Audit: UNKNOWN/.......... performed request GET http://..........:21000/api/atlas/admin/status (....) at time 2018-10-17T15:39Z (AUDIT:117) this for gc-worker.log.0.current {Heap before GC invocations=87 (full 0):
par new generation total 552960K, used 501664K [0x0000000080000000, 0x00000000a5800000, 0x00000000a5800000)
eden space 491520K, 100% used [0x0000000080000000, 0x000000009e000000, 0x000000009e000000)
from space 61440K, 16% used [0x00000000a1c00000, 0x00000000a25e8390, 0x00000000a5800000)
to space 61440K, 0% used [0x000000009e000000, 0x000000009e000000, 0x00000000a1c00000)
concurrent mark-sweep generation total 1482752K, used 192764K [0x00000000a5800000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 100066K, capacity 101828K, committed 102016K, reserved 1138688K
class space used 12079K, capacity 12406K, committed 12416K, reserved 1048576K
97088.938: [GC (Allocation Failure) 97088.938: [ParNew
Desired survivor size 31457280 bytes, new threshold 6 (max 6)
- age 1: 308504 bytes, 308504 total
- age 2: 9640 bytes, 318144 total
- age 3: 33576 bytes, 351720 total
- age 4: 2168 bytes, 353888 total
: 501664K->2871K(552960K), 0.0178481 secs] 694429K->195636K(2035712K), 0.0179751 secs] [Times: user=0.32 sys=0.19, real=0.02 secs] yes it is a kerberized cluster. thank you for your response.
... View more
10-17-2018
01:32 PM
Hello, I'm working with HDP 2.6 and i have a problem logging with Atlas dashboard UI. It's inaccesible. Any suggession. Thank you in advance
... View more
Labels:
08-01-2018
04:06 PM
i found a difficulty while connecting to hive while using gatling thank you any suggesion
... View more
Labels:
07-31-2018
01:33 PM
hi @Jonathan Sneep i found this security.inter.broker.protocol=PLAINTEXTSASL in the 'advanced kafka-broker' and i changed to SASL_PLAINTEXT so i get an error in all my kafka brokers thank you in advance
... View more
07-31-2018
01:00 PM
hi @Jonathan Sneep i found this security.inter.broker.protocol=PLAINTEXTSASL in the 'advanced kafka-broker' and i changed to SASL_PLAINTEXT so i get an error in all my kafka brokers thank you in advance
... View more
07-31-2018
08:58 AM
hello @dbains what should i do ? please thanks you
... View more
07-30-2018
10:00 AM
hello @dbains i checked the ATLAS_ENTITIES and ATLAS_HOOK and i found this : Exception in thread "main" org.apache.kafka.common.KafkaException: Exception while loading Zookeeper JAAS login context 'Client' i folowed this policies to add ranger kafka atlas plocies : https://community.hortonworks.com/articles/90962/atlas-install-validation-and-troubleshooting-tips.html thank you
... View more
07-27-2018
02:22 PM
hello @dbains The kafka broker atlas entities failed
... View more
07-27-2018
01:05 PM
hello @dbains atlas.kafka.sasl.kerberos.service.name : kafka atlas.kafka.security.protocol : PLAINTEXTSASL
... View more
07-27-2018
01:05 PM
hello @dbains atlas.kafka.sasl.kerberos.service.name : kafka atlas.kafka.security.protocol : PLAINTEXTSASL
... View more
07-27-2018
12:54 PM
hello @dbains how could i check the security.protocol in atlas properties to connect to secure Kafka thank you
... View more
07-26-2018
02:15 PM
Hello everyone, I tried to export metadata and this is my logs from atlas graph rollback due to exception (GraphTransactionInterceptor:71)
org.apache.atlas.exception.AtlasBaseException: Failed to notify NotificationEntityChangeListener for change UPDATE
at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.notifyListeners(AtlasEntityChangeNotifier.java:175)
at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.onEntitiesMutated(AtlasEntityChangeNotifier.java:84)
at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.createOrUpdate(AtlasEntityStoreV1.java:165)
at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.createOrUpdate(AtlasEntityStoreV1.java:173)
at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$FastClassBySpringCGLIB$80c00649.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:60)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$EnhancerBySpringCGLIB$fea0e85f.createOrUpdate(<generated>)
at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:423)
at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:310)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.atlas.notification.NotificationException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 5 record(s) for ATLAS_ENTITIES-0 due to 30090 ms has passed since batch creation plus linger time
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:239)
at org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:212)
at org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:116)
at org.apache.atlas.notification.NotificationEntityChangeListener.notifyOfEntityEvent(NotificationEntityChangeListener.java:191)
at org.apache.atlas.notification.NotificationEntityChangeListener.onEntitiesUpdated(NotificationEntityChangeListener.java:89)
at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.notifyListeners(AtlasEntityChangeNotifier.java:168)
... 19 more
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 5 record(s) for ATLAS_ENTITIES-0 due to 30090 ms has passed since batch creation plus linger time
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:65)
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:52)
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25)
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:230)
... 24 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 5 record(s) for ATLAS_ENTITIES-0 due to 30090 ms has passed since batch creation plus linger time thank you Any suggesion please
... View more
07-24-2018
12:14 PM
HELLO, After an upgrade to my HDP 2.6, i'm not able to see my new tables and databases. All my services works very well. So i tried to import hive.sh /usr/hdp/current/atlas-server/hook-bin/import-hive.sh but i got this errors Using Hive configuration directory [/etc/hive/conf] Log file for import is /usr/hdp/current/atlas-server/logs/import-hive.log log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout. log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout. Exception in thread "main" org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed. at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClient$API_V1@2807bdebfailed with status 403 (Forbidden) Response Body (<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 403 {"AuthorizationError":"You are not authorized for READ on [ENTITY] : *"}</title> </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /api/atlas/entities. Reason: <pre> {"AuthorizationError":"You are not authorized for READ on [ENTITY] : *"}</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body> </html> ) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:387) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:320) at org.apache.atlas.AtlasBaseClient.callAPIWithRetries(AtlasBaseClient.java:471) at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:967) at org.apache.atlas.AtlasClient.getEntity(AtlasClient.java:651) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReference(HiveMetaStoreBridge.java:352) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseReference(HiveMetaStoreBridge.java:230) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:165) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:140) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) Failed to import Hive Data Model!!! thnks any suggestion !!! Please
... View more
Labels:
07-24-2018
09:40 AM
HELLO @Jonathan Sneep i allways have the same issues Any suggestion please
... View more
07-23-2018
08:59 AM
hello @Jonathan Sneep thanks for your reponse i added the following with ranger in Kafka policies:
topic=ATLAS_HOOK
permission=publish, create; group=public permission=consume, create; user=atlas (for non-kerberized environments, set group=public) topic=ATLAS_ENTITIES
permission=publish, create; user=atlas (for non-kerberized environments, set group=public) permission=consume, create; group=public
... View more
07-22-2018
06:50 PM
@Jonathan Sneep Using Hive configuration directory [/etc/hive/conf] Log file for import is /usr/hdp/current/atlas-server/logs/import-hive.log log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout. log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout. Exception in thread "main" org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed. at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClient$API_V1@2807bdebfailed with status 403 (Forbidden) Response Body (<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 403 {"AuthorizationError":"You are not authorized for READ on [ENTITY] : *"}</title> </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /api/atlas/entities. Reason: <pre> {"AuthorizationError":"You are not authorized for READ on [ENTITY] : *"}</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body> </html> ) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:387) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:320) at org.apache.atlas.AtlasBaseClient.callAPIWithRetries(AtlasBaseClient.java:471) at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:967) at org.apache.atlas.AtlasClient.getEntity(AtlasClient.java:651) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReference(HiveMetaStoreBridge.java:352) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseReference(HiveMetaStoreBridge.java:230) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:165) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:140) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) Failed to import Hive Data Model!!!
... View more
07-20-2018
06:20 PM
thank you but the only message i have in the application.log is 2018-07-20 14:07:52,071 INFO - [pool-2-thread-31 - a43d6555-0b77-4fbd-b093-c71c0d45a080:] ~ Audit: UNKNOWN/10.0.120.250-10.0.120.250 performed request GET http://localhost:21000/api/atlas/admin/status (10.0.120.250) at time 2018-07-20T12:07Z (AUDIT:117)
... View more
07-20-2018
06:20 PM
THANK YOU I have this message in all the application.log 2018-07-20 14:07:52,071 INFO - [pool-2-thread-31 - a43d6555-0b77-4fbd-b093-c71c0d45a080:] ~ Audit: UNKNOWN/10.0.120.250-10.0.120.250 performed request GET http://dbmnprodza51001.serv.cdc.fr:21000/api/atlas/admin/status (10.0.120.250) at time 2018-07-20T12:07Z (AUDIT:117)
... View more
07-20-2018
06:19 PM
Hello @William Brooks i found a solution for the permission denied but i have now another problem log4j:WARN No such property [maxFileSize] in
org.apache.log4j.PatternLayout. log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout. Exception in thread "main"
org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed.
... View more
07-20-2018
06:19 PM
Hi @Nixon Rodrigues i found this after the import-hive.sh Using Hive configuration directory
[/etc/hive/conf]
[20/1880] Log file for import is
/usr/hdp/current/atlas-server/logs/import-hive.log log4j:WARN No such property [maxFileSize] in
org.apache.log4j.PatternLayout. log4j:WARN No such property [maxBackupIndex] in
org.apache.log4j.PatternLayout. Exception in thread "main"
org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed. at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.Session HiveMetaStoreClient at
org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1334) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:138) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) Caused by: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1566) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3528) at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3560) at
org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1332) ... 3 more Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) ... 9 more Caused by: MetaException(message:Could not connect to meta
store using any of the URIs provided. Most recent failure:
org.apache.thrift.transport.TTransp ortException: GSS initiate failed at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:487) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:282) at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) ... 9 more Caused by: MetaException(message:Could not connect to meta
store using any of the URIs provided. Most recent failure:
org.apache.thrift.transport.TTransportException: GSS initiate failed at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at
org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:487) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:282) at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3528) at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3560) at
org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1332) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:138) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) ) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:534) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:282) at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) ... 14 more Failed to import Hive Data Model!!!
... View more
07-20-2018
04:03 PM
@Jonathan Sneep Using Hive configuration directory [/etc/hive/conf] Log file for import is
/usr/hdp/current/atlas-server/logs/import-hive.log log4j:WARN No such property [maxFileSize] in
org.apache.log4j.PatternLayout. log4j:WARN No such property [maxBackupIndex] in
org.apache.log4j.PatternLayout. Exception in thread "main"
org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.main() failed. at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:650) Caused by: org.apache.atlas.AtlasServiceException: Metadata
service API org.apache.atlas.AtlasClient$API_V1@2807bdeb
failed with status 403 (Forbidden) Response Body (<html> <head> <meta http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1"/> <title>Error 403
{"AuthorizationError":"You are not authorized for READ on
[ENTITY] : *"}</title> </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /api/atlas/entities. Reason: <pre>
{"AuthorizationError":"You are not authorized for READ on
[ENTITY] : *"}</pre></p><hr
/><i><small>Powered by
Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body> </html> ) at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:387) at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:320) at
org.apache.atlas.AtlasBaseClient.callAPIWithRetries(AtlasBaseClient.java:471) at
org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:967) at
org.apache.atlas.AtlasClient.getEntity(AtlasClient.java:651) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReference(HiveMetaStoreBridge.java:352) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseReference(HiveMetaStoreBridge.java:230) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:165) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:140) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) Failed to import Hive Data Model!!!
... View more