Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Beeswax and Hive CLI not consistent.

Beeswax and Hive CLI not consistent.

Explorer

Hi,

 

I am having a strange problem.  The View of data in Beeswax does seem to match the view I get using the Hive CLI.  in the CLI  if I run the following:

 

 select * from jira_image_errors;

 

I get back two records

 

OK
CWO-527837    2014-02-20 22:35:03    Quality issue    2014-05-13
CWO-528111    2014-02-22 05:17:15    Quality Issue    2014-05-13
Time taken: 1.616 seconds

 

running the same query against the same db using beeswax returns no records.  Also looking at the Metastore manager in Hue I am seeing three tables in this particular database, viewed from the CLI,  I only see one.

 

I am running CDH 4.5 with a single server cluster running CENTOS

 

looking at what is running, I only see the one intstance of the metastore

 

hive     10125  2495  0 20:56 ?        00:00:08 /usr/java/jdk1.6.0_31/bin/java -Xmx256m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/usr/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xms268435456 -Xmx268435456 -Djava.net.preferIPv4Stack=true -Xms268435456 -Xmx268435456 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/lib/hive/lib/hive-service-0.10.0-cdh4.5.0.jar org.apache.hadoop.hive.metastore.HiveMetaStore -p 9083
postgres 10224  1976  0 20:56 ?        00:00:00 postgres: hive metastore 127.0.0.1(51726) idle    
postgres 10260  1976  0 20:56 ?        00:00:00 postgres: hive metastore 127.0.0.1(51730) idle    
hive     10355  2495  0 20:56 ?        00:00:07 /usr/java/jdk1.6.0_31/bin/java -Xmx1000m -Djava.net.preferIPv4Stack=true -Dwebhcat.log.dir=/var/log/hcatalog -Dlog4j.configuration=file:/var/run/cloudera-scm-agent/process/166-hive-WEBHCAT/webhcat-log4j.properties -Dhadoop.log.dir=/usr/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xms268435456 -Xmx268435456 -Djava.net.preferIPv4Stack=true -Xms157931397 -Xmx157931397 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/lib/hcatalog/sbin/../share/webhcat/svr//webhcat-0.5.0-cdh4.5.0.jar org.apache.hcatalog.templeton.Main
root     11403  8732  0 20:59 pts/6    00:00:02 /usr/java/jdk1.6.0_31/bin/java -Xmx256m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/usr/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xms268435456 -Xmx268435456 -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/lib/hive/lib/hive-service-0.10.0-cdh4.5.0.jar org.apache.hadoop.hive.service.HiveServer
root     16553  8732  0 21:13 pts/6    00:00:00 grep hive

 And I am only running one instance of Postgres

 

[root@qa-big-data-219 hive]# ps -ef | grep postgres
postgres  1976     1  0 May04 ?        00:00:12 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
postgres  1978  1976  0 May04 ?        00:00:11 postgres: logger process                          
postgres  1980  1976  0 May04 ?        00:01:00 postgres: writer process                          
postgres  1981  1976  0 May04 ?        00:00:51 postgres: wal writer process                      
postgres  1982  1976  0 May04 ?        00:00:21 postgres: autovacuum launcher process             
postgres  1983  1976  0 May04 ?        00:00:24 postgres: stats collector process                 
postgres 10224  1976  0 20:56 ?        00:00:00 postgres: hive metastore 127.0.0.1(51726) idle    
postgres 10260  1976  0 20:56 ?        00:00:00 postgres: hive metastore 127.0.0.1(51730) idle    
root     18632  8732  0 21:21 pts/6    00:00:00 grep postgres

 I have stopped all the services and restarted everything, with no impact.   From what I can see the BeesWAX config looks correct

 

hive.metastore.uristhrift://qa-big-data-219:9083 
hive.metastore.warehouse.dir/user/hive/warehouse

 

 

qa-big-data-219 is the server I am running the local queries on.    One odd thing I did notice was that the beeswax queries show up in the hadoop-cmf-hive1-HIVEMETASTORE-qa-big-data-219.log.out  but the queries run via the cli do not.   Does anyone have any suggestions as to how I could further troubleshoot this issue.  I am not so much worried about Beeswax per say,  it more an issue that our external services aren't seeing consistent results.  

 

Thanks

 

Andrew

Don't have an account?
Coming from Hortonworks? Activate your account here