Member since
06-15-2016
12
Posts
4
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
27055 | 07-29-2016 07:42 PM |
03-05-2018
06:44 PM
Hi Ali, Did you tried to pass "hive.execution.engine=tez" as session config property in JDBC connection URL. Please try that and let me if that works. e.g. con = DriverManager.getConnection("jdbc:hive2://host:10000/default?hive.execution.engine=tez")
... View more
09-22-2016
11:06 PM
@Predrag Minovic I also tried this work around it is working partially for me as i have to recover all the table partition because somehow it doesn't detect s3 sub directories .
... View more
09-22-2016
11:02 PM
@Rajesh Balamohan I have aws access id & secret stored in in hive-site and hdfs-site config file.
... View more
09-22-2016
11:00 PM
@hduraiswamy I am able to execute alter query, I didn't get any errors . Yes i have tried extended describe command as well to get location info.
... View more
09-21-2016
10:02 PM
1 Kudo
I want to change my external table hdfs location to new path location which is Amazon S3 in my case. I tried following query. ALTER TABLE table_name set location 's3n://bucket/path/to/data' But some how it is still pointing to old hdfs external path. Is there any query I need to use in order to update hive metastore with new external data path location. Any kind of help would be greatly appreciated .
... View more
Labels:
- Labels:
-
Apache Hive
08-24-2016
09:26 PM
@Wes Floyd @srai @mqureshi Thank you so much for your quick responses to my question. As I already have staging table to orc table implementation structure I will try to use csv column headers to create staging tables and then I will load staging data to actual table.
... View more
08-24-2016
08:55 PM
Consider the following scenario
Hive table with 5 columns (col1, col2, col3, col4, col5)
CSV file with 3 columns (col1, col3, col5)
Now I want to load CSV file data into hive table with exact csv to hive column mapping as follows.
hive csv data
col1 <-> col1
col2 empty
col3 <-> col3
col5 <-> col5
col4 empty
Any kind of help would be greatly appreciate.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
07-29-2016
07:42 PM
1 Kudo
I solved it by passing the below connection objects with my user name. con = DriverManager.getConnection("jdbc:hive2://host:10000/default?hive.execution.engine=tez", "<user-name>",""); Thanks you all of you for your help and advice.
... View more
07-29-2016
06:31 AM
@mqureshi I am able to run same query through hive view and hive cli but not through jdbc .
Cluster configuration is as below: Name node and Secondary node (8 Core / 28 GB RAM each)
3 Data Nodes (8 Core / 28 GB RAM each)
I tried to set hive.execution.engine=tez by appending it to jdbc connect url but it doesn't work.
... View more
07-28-2016
12:34 AM
@Takahiko Saito Stack trace just after jdbc query failure. Container exited with a non-zero exit code 127
Failing this attempt. Failing the application.
2016-07-28 00:11:22,769 INFO rmapp.RMAppImpl (RMAppImpl.java:handle(767)) - application_1466711437570_0051 State change from FINAL_SAVING to FAILED
2016-07-28 00:11:22,769 WARN resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(323)) - USER="girish" OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1466711437570_0051 failed 2 times due to AM Container for appattempt_1466711437570_0051_000002 exited with exitCode: 127
For more detailed output, check application tracking page then click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e02_1466711437570_0051_02_000001
Exit code: 127
Exception message: /bin/bash: /mnt/resource/usercache/girish/appcache/application_1466711437570_0051/container_e02_1466711437570_0051_02_000001/default_container_executor_session.sh: No such file or directory
Stack trace: ExitCodeException exitCode=127: /bin/bash: /mnt/resource/usercache/girish/appcache/application_1466711437570_0051/container_e02_1466711437570_0051_02_000001/default_container_executor_session.sh: No such file or directory
at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
at org.apache.hadoop.util.Shell.run(Shell.java:487)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 127
Failing this attempt. Failing the application. APPID=application_1466711437570_0051
2016-07-28 00:11:22,769 INFO capacity.ParentQueue (ParentQueue.java:removeApplication(372)) - Application removed - appId: application_1466711437570_0051 user: "girish" leaf-queue of parent: root #applications: 0
2016-07-28 00:11:23,446 INFO resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(170)) - USER="girish" IP=10.0.0.5 OPERATION=Kill Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1466711437570_0051 CALLERCONTEXT=HIVE_SSN_ID:d8a476e9-84c4-4633-a980-3f8a9dd7dac8
... View more