Member since
04-11-2017
15
Posts
1
Kudos Received
0
Solutions
06-11-2018
09:57 AM
csguna, Here is the connection related to zookeeper, it looks that our created topic ciovInput_v3 and ciovInput_v1 are there. thanks # bin/kafka-topics.sh --describe --zookeeper localhost:2181 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/KAFKA-3.0.0-1.3.0.0.p0.40/lib/kafka/libs/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/KAFKA-3.0.0-1.3.0.0.p0.40/lib/kafka/libs/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Topic:__consumer_offsets PartitionCount:50 ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer Topic: __consumer_offsets Partition: 0 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 1 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 2 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 3 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 4 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 5 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 6 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 7 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 8 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 9 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 10 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 11 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 12 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 13 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 14 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 15 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 16 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 17 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 18 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 19 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 20 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 21 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 22 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 23 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 24 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 25 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 26 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 27 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 28 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 29 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 30 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 31 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 32 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 33 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 34 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 35 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 36 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 37 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 38 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 39 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 40 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 41 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 42 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 43 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic: __consumer_offsets Partition: 44 Leader: 256 Replicas: 255,256,257 Isr: 256 Topic: __consumer_offsets Partition: 45 Leader: 256 Replicas: 256,255,257 Isr: 256 Topic: __consumer_offsets Partition: 46 Leader: 256 Replicas: 257,256,255 Isr: 256 Topic: __consumer_offsets Partition: 47 Leader: 256 Replicas: 255,257,256 Isr: 256 Topic: __consumer_offsets Partition: 48 Leader: 256 Replicas: 256,257,255 Isr: 256 Topic: __consumer_offsets Partition: 49 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic:ciovGSInput PartitionCount:1 ReplicationFactor:3 Configs: Topic: ciovGSInput Partition: 0 Leader: 256 Replicas: 257,255,256 Isr: 256 Topic:ciovInput PartitionCount:1 ReplicationFactor:1 Configs: Topic: ciovInput Partition: 0 Leader: 256 Replicas: 256 Isr: 256 Topic:ciovInput_v1 PartitionCount:1 ReplicationFactor:1 Configs: Topic: ciovInput_v1 Partition: 0 Leader: 256 Replicas: 256 Isr: 256 Topic:ciovInput_v3 PartitionCount:1 ReplicationFactor:1 Configs: Topic: ciovInput_v3 Partition: 0 Leader: 255 Replicas: 255 Isr: 255
... View more
02-15-2018
12:15 PM
ben, thanks. the log file is way too big, I just checked again and found the following error: Can't open /run/cloudera-scm-agent/process/1431-hive-HIVEMETASTORE/supervisor.conf: Permission denied. all files in directory /run/cloudera-scm-agent/process/ is owned by hive except this file. -rw------- 1 root root 3430 Feb 15 14:30 supervisor.conf I have two hive servers, the one works has the same permission issue. but it doesn't prevent it from running successfully. Here is the more detailed log file information: + exec /opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hive/bin/hive --config /run/cloudera-scm-agent/process/1431-hive-HIVEMETASTORE --service metastore -p 9083
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/02/15 14:30:16 ERROR conf.Configuration: error parsing conf core-default.xml
javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2541)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2503)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2409)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:982)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1032)
at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1433)
at org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:67)
at org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:81)
at org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:96)
at org.apache.hadoop.hbase.util.MapreduceDependencyClasspathTool.main(MapreduceDependencyClasspathTool.java:70)
Exception in thread "main" java.lang.RuntimeException: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2659)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2503)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2409)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:982)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1032)
at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1433)
at org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:67)
at org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:81)
at org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:96)
at org.apache.hadoop.hbase.util.MapreduceDependencyClasspathTool.main(MapreduceDependencyClasspathTool.java:70)
Caused by: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2541)
... 9 more
18/02/15 14:30:17 ERROR conf.Configuration: error parsing conf core-default.xml
javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2541)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2503)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2409)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1144)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1116)
at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:525)
at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:543)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:437)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2652)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:2611)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:74)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:58)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6083)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Exception in thread "main" java.lang.RuntimeException: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2659)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2503)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2409)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1144)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1116)
at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:525)
at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:543)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:437)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2652)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:2611)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:74)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:58)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6083)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2541)
... 18 more
+ date it looks that the following log files do not have any informationl /var/log/hive/hadoop-cmf-hive-HIVESERVER2-cahive-master01.log.out and /var/log/hive/hadoop-cmf-hive-HIVESERVER2-cahive-master01.log.out also, I tried to start the server from command line and it dosn't work. $ sudo service hive-server2 start Redirecting to /bin/systemctl start hive-server2.servic Failed to start hive-server2.service: Unit not found. thanks
... View more
06-12-2017
07:06 PM
go to machine's Status page, Click Action, it has only two options clickable : "Initialize" and "enter maintenance mode", unable to click "start/stop" etc options. I clicked "initialize", still I am unable to start the zookeeper. Thanks
... View more