Re: Using Rest API calls how would I call a lineag...

DanBur · ‎04-03-2018

I access hive tables through the hive metastore api. I'm trying to get the hive table row count via the below lines of code. The problem I get is that some of the values returned are inaccurate. How do I get the updated/accurate table row count value. The numRows property does not seem to update as the hive table updates. HCatTable tbl; //table object Map<String, String> tableProps =tbl.getTblProps(); //table property map object String value = tableProps.get("numRows");

DanBur · ‎06-01-2017

Thanks!. Looks like your suggested call successfully returned.

DanBur · ‎05-31-2017

I'm connecting to Navigator and attempting to make rest api calls. I keep getting { "message" : "Invalid id : 4500 5267" } when trying to make a lineage call with multiple entityids. The url i'm using is below http://vm-cloudera-59:7187/api/v10/lineage3/?entityIds=4500%205267

DanBur · ‎04-05-2017

I've attached the audit.log and the application.log application.txtaudit.txt

DanBur · ‎04-03-2017

Taxonomy and subterms are missing from the Taxonomy tab. This issue started once I imported a large amount of hive objects into atlas. Before the import, I was able to access the terms via rest api calls (http://host:port/api/atlas/v1/taxonomies/Catalog/terrms) now the calls hang. I've restarted the services in this order (HDFS,YARN,Zookeeper,HBase,Kafka,Ambari Infra,Hive and Atlas) with no improvement. We are currently using HDP-2.5.0.0.

DanBur · ‎08-29-2016

String hbasePolicyPath = "hbase-policy.xml"; String hbaseSitepath ="hbase-site.xml"; config.set("hadoop.security.authentication", "kerberos"); UserGroupInformation.setConfiguration(config); System.out.print("Enter principal"); System.out.println(); String principal = in.nextLine(); //System.out.println("Attempting to authenticate with principal: : " + principal); String keytab = "hbase.keytab"; if(principal != null && !principal.isEmpty() && keytab != null && !keytab.isEmpty()){ logger.info("Login user " + principal + " from keytab " + keytab + " ...."); UserGroupInformation userGroupInformation = UserGroupInformation.loginUserFromKeytabAndReturnUGI(pri ncipal, keytab); UserGroupInformation.setLoginUser(userGroupInformation); logger.info("User logged in from keytab...."); } config.addResource(new Path(hbasePolicyPath)); config.addResource(new Path(hbaseSitepath)); connection = ConnectionFactory.createConnection(config); System.out.println(connection.getConfiguration().get("hbase.zookeeper.quorum")); logger.info("Connection Accepted"); admin = connection.getAdmin(); System.out.println(admin.getClusterStatus().getMaster()); System.out.println("Dead Server count:" + admin.getClusterStatus().getDeadServers()); yes the clusters are using the root znode of /habse-secure. I've also attached the hbase-site.xml

DanBur · ‎08-29-2016

Exception in thread "main" org.apache.hadoop.net.ConnectTimeoutException: Call From rw1.rw.com/177.277.22.177 to rw2.rw.com:8020 failed on socket timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending local=177.277.22.177:43214 remote=rw2.rw.com/17.17.33.7:8020]; For more details see: http://wiki.apache.org/hadoop/SocketTimeout

DanBur · ‎08-29-2016

Caused by: org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase-secure This is the error I get while attempting to run the following snippet config.addResource(new Path(hbasePolicyPath)); config.addResource(new Path(hbaseSitepath)); connection = ConnectionFactory.createConnection(config); admin = connection.getAdmin(); //hangs here for some minutes and then returns the above error. Please note I'm attempting to connect to multiple zookeeper quorums.

DanBur · ‎06-02-2016

Thank you. I haven't had a chance to look at the logs. Other tasks came up. I'll update the question once I've done that

DanBur · ‎06-01-2016

Is that accessible through ambari or do I have to get it from my SSH Horton client

DanBur · ‎06-01-2016

I set the hadoop home directory. I'm still getting the same zookeper connection errors

DanBur · ‎06-01-2016

I also added the following to my local windows host file 127.0.0.1 localhost 10.0.2.15 sandbox sandbox.hortonworks.com

DanBur · ‎06-01-2016

I've put the hbase-site.xml in the classpath. I get an updated log which I've atatchederrorlog.txt Thank you for your help

DanBur · ‎06-01-2016

Yes I'm trying to connect to an hbase cluster to perform CRUD oeprations on tables. My java program currently looks like this Configuration conf = HBaseConfiguration.create(); try { HBaseAdmin.checkHBaseAvailable(conf); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); } HBase Version 1.1.2.2.4.0.0-169, revision=61dfb2b344f424a11f93b3f086eab815c1eb0b6a ZooKeeper Client Version 3.4.6-169, revision=-1 Zookeeper Quorum sandbox.hortonworks.com:2181 Zookeeper Base Path /hbase-unsecure Hadoop Version 2.7.1.2.4.0.0-169, revision=26104d8ac833884c8776473823007f176854f2eb All information gotten from Ambari Dashboard. The hbase-site xml were also downloaded from the Ambari dashboard. I'm currently using hbase-1.1.5 bin version jars in my local java project. I also had to set a local hadoop home directory in vm arguments because I got the below error 2016-06-01 10:12:17,088 ERROR [main] util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

DanBur · ‎06-01-2016

No. There is not zk client running on my local machine. I think my java program fails to connect to zookeper. I get the below error as well 2016-05-31 16:04:05,827 INFO [main-SendThread(0:0:0:0:0:0:0:1:2181)] zookeeper.ClientCnxn: Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 2016-05-31 16:04:06,830 WARN [main-SendThread(0:0:0:0:0:0:0:1:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2016-05-31 16:04:06,935 INFO [main-SendThread(127.0.0.1:2181)] zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2016-05-31 16:04:06,945 WARN [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid 2016-05-31 16:04:07,937 WARN [main-SendThread(127.0.0.1:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information

DanBur · ‎06-01-2016

How would I ensure it picks up the values from my VM.

DanBur · ‎06-01-2016

The below environment variables are wrong and seem to be from my local machine. I would expect Zookeeper to list the variables from my Sandbox on VirtualBox 2016-05-31 16:04:05,796 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2016-05-31 16:04:05,796 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=daniel 2016-05-31 16:04:05,796 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.7.0_72 2016-05-31 16:04:05,796 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2016-05-31 16:04:05,796 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=C:\Java\jdk1.7.0_72\jre

Online	Offline
Last Visited	‎06-01-2017 11:38 AM

Date Registered	‎05-31-2017 03:55 PM
Last Visited	‎06-01-2017 11:38 AM
Total Messages Posted	17
Total Kudos Received	1

Re: Using Rest API calls how would I call a lineag...

How Do I get the accurate Hive Table row count val...

Re: Using Rest API calls how would I call a lineag...

Using Rest API calls how would I call a lineage ca...

Re: Catalog Taxonomy and subterms fail to load on ...

Catalog Taxonomy and subterms fail to load on the ...

Re: HBase java client failed to connect to multipl...

HDFS failed on socket timeout exception

HBase java client failed to connect to multiple zo...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Re: Wrong Zookeeper Client Information listed in E...

Wrong Zookeeper Client Information listed in Eclip...