[root@hadoop2 admin]# ps -ef |grep hive hive 6178 1 0 14:21 ? 00:00:08 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Xmx1024m -Dhdp.version=2.4.3.0-227 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.4.3.0-227 -Dhadoop.log.dir=/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.4.3.0-227/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.3.0-227/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -XX:MaxPermSize=512m -Xmx1489m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.4.3.0-227/hive/lib/hive-service-1.2.1000.2.4.3.0-227.jar org.apache.hadoop.hive.metastore.HiveMetaStore -hiveconf hive.log.file=hivemetastore.log -hiveconf hive.log.dir=/var/log/hive hive 8525 1 1 14:22 ? 00:00:11 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Xmx1024m -Dhdp.version=2.4.3.0-227 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.4.3.0-227 -Dhadoop.log.dir=/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.4.3.0-227/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.3.0-227/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -XX:MaxPermSize=512m -Xmx4467m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.4.3.0-227/hive/lib/hive-service-1.2.1000.2.4.3.0-227.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris= -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/var/log/hive hcat 8973 1 0 14:22 ? 00:00:06 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Xmx1024m -Dhdp.version=2.4.3.0-227 -Djava.net.preferIPv4Stack=true -Dwebhcat.log.dir=/var/log/webhcat/ -Dlog4j.configuration=file:///usr/hdp/2.4.3.0-227/hive-hcatalog/sbin/../etc/webhcat/webhcat-log4j.properties -Dhdp.version=2.4.3.0-227 -Dhadoop.log.dir=/var/log/hadoop/hcat -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.4.3.0-227/hadoop -Dhadoop.id.str=hcat -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.3.0-227/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -XX:MaxPermSize=512m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.4.3.0-227/hive-hcatalog/sbin/../share/webhcat/svr/lib/hive-webhcat-1.2.1000.2.4.3.0-227.jar org.apache.hive.hcatalog.templeton.Main ... View more

I checked and both hiveserver2 and metastore processes are running on the server hadoop2, I still can't connect via beeline , and HDC is showing a critical alert on HIVE. [root@hadoop2 admin]# ps -ef | grep metastore hive 6178 1 1 14:21 ? 00:00:08 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Xmx1024m -Dhdp.version=2.4.3.0-227 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.4.3.0-227 -Dhadoop.log.dir=/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.4.3.0-227/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.3.0-227/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -XX:MaxPermSize=512m -Xmx1489m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.4.3.0-227/hive/lib/hive-service-1.2.1000.2.4.3.0-227.jar org.apache.hadoop.hive.metastore.HiveMetaStore -hiveconf hive.log.file=hivemetastore.log -hiveconf hive.log.dir=/var/log/hive hive 8525 1 1 14:22 ? 00:00:10 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Xmx1024m -Dhdp.version=2.4.3.0-227 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.4.3.0-227 -Dhadoop.log.dir=/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.4.3.0-227/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.3.0-227/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -XX:MaxPermSize=512m -Xmx4467m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.4.3.0-227/hive/lib/hive-service-1.2.1000.2.4.3.0-227.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris= -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/var/log/hive root 12000 11237 0 14:32 pts/0 00:00:00 grep metastore [root@hadoop2 admin]# [root@hadoop2 admin]# [root@hadoop2 admin]# [root@hadoop2 admin]# ps -ef | grep hiveserver2 hive 8525 1 1 14:22 ? 00:00:10 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Xmx1024m -Dhdp.version=2.4.3.0-227 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.4.3.0-227 -Dhadoop.log.dir=/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.4.3.0-227/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.3.0-227/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -XX:MaxPermSize=512m -Xmx4467m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.4.3.0-227/hive/lib/hive-service-1.2.1000.2.4.3.0-227.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris= -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/var/log/hive root 12009 11237 0 14:32 pts/0 00:00:00 grep hiveserver2 [root@hadoop2 admin]# [root@hadoop2 admin]# ... View more

service is not running ? which service is using this 10000 port? I read the solution of the similar issue suggesting the following but I don't know how to do it Kill your hiveserver2 & metastore & Restart them again. ???? check hiveserver2 & metastore are running using 'jps' command. ???? [root@hadoop1 admin]# telnet hadoop2 10000 Trying 10.100.44.16... telnet: connect to address 10.100.44.16: Connection refused [root@hadoop1 admin]# here is the output of jps command on hadoop2 where HIVE is installed. 7829 NodeManager 6194 gateway.jar 5679 Main 7292 Main 6014 Kafka 6530 HistoryServer 14238 Jps 2618 AmbariServer 6922 Main 4290 QuorumPeerMain 5868 HMaster 8753 Main 4178 DataNode 7167 Main 4462 NameNode 7478 HRegionServer [root@hadoop1 admin]# jps 7829 NodeManager 6194 gateway.jar 5679 Main 7292 Main 6014 Kafka 6530 HistoryServer 2618 AmbariServer 6922 Main 4290 QuorumPeerMain 5868 HMaster 14398 Jps 8753 Main 4178 DataNode 7167 Main 4462 NameNode 7478 HRegionServer ... View more

Thanks Pierre now its beginning to make some sense , so the 5 GenerateFF processors are there to take care of the 5 countries I guess. I want to read my own log file , which processor would I use? I want to start with a simple task as read my log file , parse out some values by using the Regexp language and then save the parsed values to HIVE. ... View more

hi Pierre attributes as I understand are key value pairs right ? so what keys does the GenerateFF generate? I looked at the configuration of the five processors and they look exactly same .whats the difference? ... View more

GenerateFlowFile processor (document is saying) generates files with random data, but no where in the processor configuration are the fields mentioned , so where do we tell or how do we know what fields this file has ? since the ReplaceText text connector is connected to GenerateFlowFile and has to know which fields to modify right? ... View more

hi Pierre I am reading these documents but I am still not understanding why use 5 identical processors ... View more

nifi-screenshot.jpg ... View more