Member since
01-09-2018
9
Posts
0
Kudos Received
0
Solutions
05-25-2018
02:17 AM
this seems like a better solution to me https://community.hortonworks.com/questions/8010/hives-alter-table-partition-concatenate-not-workin.html
... View more
05-25-2018
01:04 AM
https://community.hortonworks.com/questions/8010/hives-alter-table-partition-concatenate-not-workin.html this solution seems better...
... View more
05-17-2018
04:04 PM
made some changes to the serde but still no luck, guess its a hive limtation https://github.com/rcongiu/Hive-JSON-Serde/issues/207
... View more
05-16-2018
11:35 PM
Hello, this post is a duplicate to the Post in the link. Reposting it as the solution there doesn't work. I tried with Json and with Text image attached. I created a Avro and insert into table stops at first insert (link) Interestingly when I put this line in SparkConf and it works fine : sparkConf(set textinputformat.record.delimiter='\n\n');
... View more
Labels:
04-30-2018
03:39 PM
Hello, we have a multi user environment setup on Zeppelin. Zeppelin has to communicate to a kerberized cluster. However, every user should be using its own keytab. This doesnt seem to be possible with Zeppelin, as the interpreter configuration is shared across all the users. I tried to use the shell script in the image below. But now the problem is if multiple users have a kerberos ticket valid, dont know which one the zeppelin or spark job will use. Can i specify in my spark job to use one keytab from the list ?
... View more
Labels:
01-30-2018
04:03 PM
Hello, is it possible to have a little more control over what goes in LLAP cache ?. Especially when new data is loaded to HDFS, and LLAP is queried for the first time. Our use case expects the most recently added data to be in the cache ( right now : the property set to true is hive.llap.io.use.lrfu ). But we want a combination of newly added data in cache & LRFU. Will @Marcos Da Silva suggestion work for data loaded every few hours?. Or is there a better generalized/approach possible? select column1,column2 from table where partition_column in
(select max(distinct partition_column)from table)" Also I don't see grafanna on my Ambari, do I have to install it on all the nodes of the cluster?. To see realtime stats of LLAP?. Reference URL : https://community.hortonworks.com/questions/85330/how-to-optimize-hive-access-to-the-latest-partitio.html
... View more
Labels:
01-18-2018
07:31 PM
Hello, I am trying to connect my Tableau client to AWS-Hortonworks2.6 and I am constantly getting error [Hortonworks][Hardy] (34) Error from server: connect() failed: Operation timed out.
Unable to connect to the server "XX.XX.XX.XX". Check that the server is running and that you have access privileges to the requested database.
also had a new errno = 10060 I can connect to hive on my sandbox but not on the AWS, I referred the following link for connecting to AWS-HIVE. https://hortonworks.github.io/hdp-aws/hive-jdbc/index.html#download-ssl-certificate But While generating the certificate I am constantly getting this error (tried port 443 & 10000 as found in Ambari): openssl s_client -servername ${GATEWAY_HOSTNAME} -connect ${GATEWAY_ADDRESS}:443 -showcerts </dev/null | openssl x509 -outform PEM > gateway.pem
connect: Operation timed outconnect:errno=60
unable to load certificate140735832372104:error:0906D06C:PEM routines:PEM_read_bio:no start line:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22/libressl/crypto/pem/pem_lib.c:704:Expecting: TRUSTED CERTIFICATE
openssl s_client -servername ${GATEWAY_HOSTNAME} -connect ${GATEWAY_ADDRESS}:10000 -showcerts </dev/null | openssl x509 -outform PEM > gateway.pem
connect: Operation timed out
connect:errno=60
unable to load certificate
I tried to keep my sandbox setting same as AWS, but not sure why I am getting a different error. Can I disable any security at the moment ?, not sure if that is the reason of failure. When I test with beeline I get this error !connect jdbc:hive2://ec2-54-80-219-49.compute-1.amazonaws.com:10000/default;transportMode=http;httpPath=cliserviceConnecting to jdbc:hive2://ec2-54-80-219-49.compute-1.amazonaws.com:10000/default;transportMode=http;httpPath=cliservice
Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
ERROR jdbc.HiveConnection: Error opening session
org.apache.thrift.transport.TTransportException: org.apache.http.client.ClientProtocolException
at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297)
Caused by: org.apache.http.client.ClientProtocolException
<br>
... View more
Labels:
01-16-2018
07:10 PM
Hello, our Hortonworks cluster is stuck on scaling for last 24 hours and I can't seem to do anything. I tried restarting ambari on master and also the EC2 instance, but its still stuck there. My last action on the HDP Data Cloud UI to increase its size by 2 more nodes caused this, I can't seem to undo or fix this. AWS console shows the 2 new instances requested from HDP UI are not setup, not sure why HDP is stuck. Help will be appreciated. The state of one or more instances couldn't be determined. Try syncing later. [this is the only information repeating on HDP Cloud UI]
2018-01-16 11:54:55
Success
Couldn't retrieve the cluster's status, starting to sync.
2018-01-15 14:14:53
Success
The state of one or more instances couldn't be determined. Try syncing later.
... View more
01-09-2018
11:37 PM
I tried various options but still getting this error. Class.forName("org.apache.hive.jdbc.HiveDriver"); Connection con = DriverManager.getConnection("jdbc:hive2://35.168.235.17:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2");
Connection con2 = DriverManager.getConnection("jdbc:hive2://35.168.235.17:10000/default
connectString=35.168.235.17:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@307f6b8c
2018-01-08 22:47:08,080 INFO [main-SendThread(35.168.235.17:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(975)) - Opening socket connection to server 35.168.235.17/35.168.235.17:2181. Will not attempt to authenticate using SASL (unknown error)
2018-01-08 22:47:23,131 ERROR [main] curator.ConnectionState (ConnectionState.java:checkTimeouts(201)) - Connection timed out for connection string (35.168.235.17:2181) and timeout (15000) / elapsed (15093)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:198)
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88)
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:474)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:63)
at org.apache.hive.jdbc.Utils.configureConnParams(Utils.java:509)
at org.apache.hive.jdbc.Utils.parseURL(Utils.java:429)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:135)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at HiveJDBC.main(HiveJDBC.java:15)
... View more