brijesh_jaggi1
Explorer

Re: Best way to merge multi part file into single ...

by Explorer in Support Questions
‎05-25-2018 02:17 AM
‎05-25-2018 02:17 AM
this seems like a better solution to me https://community.hortonworks.com/questions/8010/hives-alter-table-partition-concatenate-not-workin.html ... View more

Re: How to reduce the small problem in spark using...

by Explorer in Support Questions
‎05-25-2018 01:04 AM
‎05-25-2018 01:04 AM
https://community.hortonworks.com/questions/8010/hives-alter-table-partition-concatenate-not-workin.html this solution seems better... ... View more

Re: Empty Lines in Json/CSV creates Empty Row in H...

by Explorer in Support Questions
‎05-17-2018 04:04 PM
‎05-17-2018 04:04 PM
made some changes to the serde but still no luck, guess its a hive limtation https://github.com/rcongiu/Hive-JSON-Serde/issues/207 ... View more

Empty Lines in Json/CSV creates Empty Row in Hive

by Explorer in Support Questions
‎05-16-2018 11:35 PM
‎05-16-2018 11:35 PM
Hello, this post is a duplicate to the Post in the link. Reposting it as the solution there doesn't work. I tried with Json and with Text image attached. I created a Avro and insert into table stops at first insert (link) Interestingly when I put this line in SparkConf and it works fine : sparkConf(set textinputformat.record.delimiter='\n\n'); ... View more
Labels:

Zeppelin MultiUser to use their own Keytab for spa...

by Explorer in Support Questions
‎04-30-2018 03:39 PM
‎04-30-2018 03:39 PM
Hello, we have a multi user environment setup on Zeppelin. Zeppelin has to communicate to a kerberized cluster. However, every user should be using its own keytab. This doesnt seem to be possible with Zeppelin, as the interpreter configuration is shared across all the users. I tried to use the shell script in the image below. But now the problem is if multiple users have a kerberos ticket valid, dont know which one the zeppelin or spark job will use. Can i specify in my spark job to use one keytab from the list ? ... View more

Optimize LLAP Cache & Realtime Stats

by Explorer in Support Questions
‎01-30-2018 04:03 PM
‎01-30-2018 04:03 PM
Hello, is it possible to have a little more control over what goes in LLAP cache ?. Especially when new data is loaded to HDFS, and LLAP is queried for the first time. Our use case expects the most recently added data to be in the cache ( right now : the property set to true is hive.llap.io.use.lrfu ). But we want a combination of newly added data in cache & LRFU. Will @Marcos Da Silva suggestion work for data loaded every few hours?. Or is there a better generalized/approach possible? select column1,column2 from table where partition_column in (select max(distinct partition_column)from table)" Also I don't see grafanna on my Ambari, do I have to install it on all the nodes of the cluster?. To see realtime stats of LLAP?. Reference URL : https://community.hortonworks.com/questions/85330/how-to-optimize-hive-access-to-the-latest-partitio.html ... View more
Labels:

Configure OJDBC Hive driver to Connect to AWS HDP ...

by Explorer in Support Questions
‎01-18-2018 07:31 PM
‎01-18-2018 07:31 PM
Hello, I am trying to connect my Tableau client to AWS-Hortonworks2.6 and I am constantly getting error [Hortonworks][Hardy] (34) Error from server: connect() failed: Operation timed out. Unable to connect to the server "XX.XX.XX.XX". Check that the server is running and that you have access privileges to the requested database. also had a new errno = 10060 I can connect to hive on my sandbox but not on the AWS, I referred the following link for connecting to AWS-HIVE. https://hortonworks.github.io/hdp-aws/hive-jdbc/index.html#download-ssl-certificate But While generating the certificate I am constantly getting this error (tried port 443 & 10000 as found in Ambari): openssl s_client -servername ${GATEWAY_HOSTNAME} -connect ${GATEWAY_ADDRESS}:443 -showcerts </dev/null | openssl x509 -outform PEM > gateway.pem connect: Operation timed outconnect:errno=60 unable to load certificate140735832372104:error:0906D06C:PEM routines:PEM_read_bio:no start line:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22/libressl/crypto/pem/pem_lib.c:704:Expecting: TRUSTED CERTIFICATE openssl s_client -servername ${GATEWAY_HOSTNAME} -connect ${GATEWAY_ADDRESS}:10000 -showcerts </dev/null | openssl x509 -outform PEM > gateway.pem connect: Operation timed out connect:errno=60 unable to load certificate I tried to keep my sandbox setting same as AWS, but not sure why I am getting a different error. Can I disable any security at the moment ?, not sure if that is the reason of failure. When I test with beeline I get this error !connect jdbc:hive2://ec2-54-80-219-49.compute-1.amazonaws.com:10000/default;transportMode=http;httpPath=cliserviceConnecting to jdbc:hive2://ec2-54-80-219-49.compute-1.amazonaws.com:10000/default;transportMode=http;httpPath=cliservice Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response ERROR jdbc.HiveConnection: Error opening session org.apache.thrift.transport.TTransportException: org.apache.http.client.ClientProtocolException at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297) Caused by: org.apache.http.client.ClientProtocolException <br> ... View more
Labels:

HDP on AWS Stuck on Scaling

by Explorer in Support Questions
‎01-16-2018 07:10 PM
‎01-16-2018 07:10 PM
Hello, our Hortonworks cluster is stuck on scaling for last 24 hours and I can't seem to do anything. I tried restarting ambari on master and also the EC2 instance, but its still stuck there. My last action on the HDP Data Cloud UI to increase its size by 2 more nodes caused this, I can't seem to undo or fix this. AWS console shows the 2 new instances requested from HDP UI are not setup, not sure why HDP is stuck. Help will be appreciated. The state of one or more instances couldn't be determined. Try syncing later. [this is the only information repeating on HDP Cloud UI] 2018-01-16 11:54:55 Success Couldn't retrieve the cluster's status, starting to sync. 2018-01-15 14:14:53 Success The state of one or more instances couldn't be determined. Try syncing later. ... View more

Re: Hive JDBC "uber" jar

by Explorer in Code Repositories
‎01-09-2018 11:37 PM
‎01-09-2018 11:37 PM
I tried various options but still getting this error. Class.forName("org.apache.hive.jdbc.HiveDriver"); Connection con = DriverManager.getConnection("jdbc:hive2://35.168.235.17:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"); Connection con2 = DriverManager.getConnection("jdbc:hive2://35.168.235.17:10000/default connectString=35.168.235.17:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@307f6b8c 2018-01-08 22:47:08,080 INFO [main-SendThread(35.168.235.17:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(975)) - Opening socket connection to server 35.168.235.17/35.168.235.17:2181. Will not attempt to authenticate using SASL (unknown error) 2018-01-08 22:47:23,131 ERROR [main] curator.ConnectionState (ConnectionState.java:checkTimeouts(201)) - Connection timed out for connection string (35.168.235.17:2181) and timeout (15000) / elapsed (15093) org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:198) at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88) at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115) at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:474) at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214) at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38) at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:63) at org.apache.hive.jdbc.Utils.configureConnParams(Utils.java:509) at org.apache.hive.jdbc.Utils.parseURL(Utils.java:429) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:135) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:270) at HiveJDBC.main(HiveJDBC.java:15) ... View more
Contact Me
Online
Offline
Last Visited
‎05-25-2018 02:16 AM
Public Statistics
Date Registered ‎01-09-2018 03:44 AM
Last Visited ‎05-25-2018 02:16 AM
Total Messages Posted 9