Member since
11-07-2017
43
Posts
3
Kudos Received
0
Solutions
10-22-2018
07:24 AM
3 Kudos
@Marek Martofel Following the steps in below link fixed this issue: Enable System Service Mode On an Upgraded Cluster
... View more
10-18-2018
11:03 AM
@Akhil S Naik Thanks for the quick reply. The missing '.' was the issue and the web ui is up now.
... View more
10-18-2018
10:42 AM
After the error "Error:Could not find or load main class org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator", following the steps in below link fixed the issue. Enable System Service Mode On an Upgraded Cluster
... View more
10-18-2018
10:36 AM
Hi, I recently upgraded from HDP 2.6.3 to HDP 3.0.1.0-187 & Ambari from 2.6 to 2.7. After the upgrade Atlas UI is not coming up though the service is up and running fine, listening on https port 21443. There is no errors or warnings thrown in the application.log file. The below exception is from the .err log file. Anyone facing the same kind of issues in Altas after the upgrade? Any help would be appreciable. Atlas version is 1.0.0 Exception in thread "main" org.apache.atlas.exception.AtlasBaseException: EmbeddedServer.Start: failed!
at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:101)
at org.apache.atlas.Atlas.main(Atlas.java:133)
Caused by: MultiException[java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
*NULL.*
^, java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
*NULL.*
^]
at org.eclipse.jetty.server.Server.doStart(Server.java:386)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:98)
... 1 more
Suppressed: java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
*NULL.*
^
at java.util.regex.Pattern.error(Pattern.java:1957)
at java.util.regex.Pattern.sequence(Pattern.java:2125)
at java.util.regex.Pattern.expr(Pattern.java:1998)
at java.util.regex.Pattern.compile(Pattern.java:1698)
at java.util.regex.Pattern.<init>(Pattern.java:1351)
at java.util.regex.Pattern.compile(Pattern.java:1028)
at org.eclipse.jetty.util.ssl.SslContextFactory.removeExcludedCipherSuites(SslContextFactory.java:1204)
at org.eclipse.jetty.util.ssl.SslContextFactory.selectCipherSuites(SslContextFactory.java:1163)
at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:320)
at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:217)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:72)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:268)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:401)
... 3 more
Caused by: java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
*NULL.*
^
at java.util.regex.Pattern.error(Pattern.java:1957)
at java.util.regex.Pattern.sequence(Pattern.java:2125)
at java.util.regex.Pattern.expr(Pattern.java:1998)
at java.util.regex.Pattern.compile(Pattern.java:1698)
at java.util.regex.Pattern.<init>(Pattern.java:1351)
at java.util.regex.Pattern.compile(Pattern.java:1028)
at org.eclipse.jetty.util.ssl.SslContextFactory.removeExcludedCipherSuites(SslContextFactory.java:1204)
at org.eclipse.jetty.util.ssl.SslContextFactory.selectCipherSuites(SslContextFactory.java:1163)
at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:320)
at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:217)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:72)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:268)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:401)
... 3 more
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Atlas
10-17-2018
12:47 PM
@Daniel Zafar Hi, I'm also facing the same error, were you able to find anything? # ./ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/lib64/qt-3.3/bin:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/maven/bin:/root/bin:/opt/maven/bin:/opt/maven/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.1.0-187/hadoop-yarn/timelineservice/^*; /usr/hdp/3.0.1.0-187/hbase/bin/hbase --config /usr/hdp/3.0.1.0-187//hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'
Error: Could not find or load main class org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
... View more
08-30-2018
03:07 PM
Hi All, In my case, i have configured my cluster with Kerberos, SSL & set the hive transport mode to http. Could you please let me know how the connection string should look like? I'm using the below connection string but still getting the same "HTTP Response code: 401 (state=08S01,code=0)" error: Note: I do have a valid kerberos ticket for accessing the service. # beeline -u "jdbc:hive2://xxx.xxx.local:10001/;principal=hive/xxx.xxx.local@xxx.LOCAL;ssl=true;sslTrustStore=/opt/hdp/security/pki/certs-new/jks/truststore.jks;trustStorePassword=changeit;transportMode=http;httpPath=cliservice" Connecting to jdbc:hive2://xxx.xxx.local:10001/;principal=hive/xxx.xxx.local@xxx.LOCAL;ssl=true;sslTrustStore=/opt/hdp/security/pki/certs-new/jks/truststore.jks;trustStorePassword=changeit;transportMode=http;httpPath=cliservice
18/08/30 13:11:33 [main]: ERROR jdbc.HiveConnection: Error opening session
org.apache.thrift.transport.TTransportException: HTTP Response code: 401
at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:262)
at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:158)
at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:150)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:622)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:221)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146)
at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211)
at org.apache.hive.beeline.Commands.connect(Commands.java:1204)
at org.apache.hive.beeline.Commands.connect(Commands.java:1100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:54)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:998)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:717)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:779)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:493)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:476)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Error: Could not establish connection to jdbc:hive2://xxx.xxx.local:10001/;principal=hive/xxx.xxx.local@xxx.LOCAL;ssl=true;sslTrustStore=/opt/hdp/security/pki/certs-new/jks/truststore.jks;trustStorePassword=changeit;transportMode=http;httpPath=cliservice: HTTP Response code: 401 (state=08S01,code=0)
Beeline version 1.2.1000.2.6.5.0-292 by Apache Hive
0: jdbc:hive2://xxx.xxx.local:10001/ (closed)>
... View more
08-27-2018
12:11 PM
Hi, @Ronak bansal, Thanks for the suggestion. The property atlas.kafka.security.protocol is already set in Atlas and the issue was because of a typo in atlas.kafka.security.protocol configured at the Hive service end. Thanks, Cibi
... View more
08-24-2018
12:50 PM
Hi, I have setup two clusters (1.LLAP Cluster,2.Data Governance Cluster), i have enabled Atlas hook in the hive service available in LLAP Cluster where as Atlas, Kafka has been setup in Data Governance cluster. Both the cluster have Kerberos setup for authentication. There is an issue with Hive Atlas hook bridge which fails to create a producer to the Kafka broker on the Data Governance Cluster. I suspect the issue is because of the incorrect Producer properties during the producer creation. I have provided the needed configurations in the atlas-applcation.properties file through ambari, but still some of the configurations are not reflecting during kafka producer creation for example security.protocol property of the producer (security.inter.broker.protocol in kafka service set to PLAINTEXTSASL while the security.protocol property used by the hive atlas hook uses PLAINTEXT). I have also tried modifying the producer.properties file avaiable under /etc/kafka/2.6.5.0-292/0/ path. Still there is an issue with producer creation. Getting below error: 2018-08-24 10:21:10,441 DEBUG [kafka-producer-network-thread | producer-1]: network.Selector (LogContext.java:debug(189)) - [Producer clientId=producer-1] Connection with dgpoc-m2.xyz.local/10.4.0.18 disconnected
java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:124)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:93)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:235)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:196)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:538)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:482)
at org.apache.kafka.common.network.Selector.poll(Selector.java:412)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:460)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:748) Below are the kafka producer properties used by the hive atlas hook: 2018-08-24 10:21:10,090 INFO [HiveServer2-Background-Pool: Thread-144]: producer.ProducerConfig (AbstractConfig.java:logAll(223)) - ProducerConfig values:
acks = 1
batch.size = 16384
bootstrap.servers = [dgpoc-m2.xyz.local:6667]
buffer.memory = 33554432
client.id =
compression.type = none
connections.max.idle.ms = 540000
enable.idempotence = false
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 0
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.StringSerializer When referring to the below hortonworks doc link, i got to know the difference in security protocol and the error caused was similar to the one showed in the link. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_secure-kafka-ambari/content/ch_secure-kafka-produce-events.html Any help will be much appreciated! Thanksm Cibi
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache Hive
-
Apache Kafka
05-31-2018
06:39 AM
Hi @pdarvasi, Currently i'm using Cloudbreak 2.7.0, i have enabled LDAP authentication for Cloudbreak UI. I would like to know whether resources can shared in version 2.7.0 and if so could you please help me reference articles on how to setup resource sharing? Thanks in Advance!
... View more
12-27-2017
08:02 AM
Thanks for the details @mmolnar. Appreciate your help on this.
... View more