Member since
04-11-2018
5
Posts
0
Kudos Received
0
Solutions
08-26-2020
05:37 AM
Hi All, We are getting below error in CDH6.3.3 after upgrade from CDH5.16.1. ALL: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session 7f0bc92b-a23d-4fe8-9235-da337db530d6_0: java.util.concurrent.TimeoutException: Client '7f0bc92b-a23d-4fe8-9235-da337db530d6_0' timed out waiting for connection from the Remote Spark Driver.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
08-26-2020
04:42 AM
I am unable to set below parameter when running hive query on spark engine using hive jdbc.
Cannot modify hive.spark.client.server.connect.timeout at runtime. It is in the list of parameters that can't be modified at runtime or is prefixed by a restricted variable.
set hive.spark.client.server.connect.timeout=90000
set hive.spark.client.connect.timeout=900000;
Both parameter is giving error in CDH6.3.3 but it was working in CDH5.16.1.
Kindly let us know the option to avoid timeout error.
... View more
Labels:
04-07-2020
01:39 AM
does hive version 1.1.0-cdh5.16.1 supports acid transaction?
... View more
Labels:
- Labels:
-
Apache Hive
03-29-2020
10:38 AM
Yes, by adding jar in classpath, i am able to import without using -libjars option also as given below.
export HADOOP_CLASSPATH=/app/hadoop_users/Mahfooz/sqoop/mysql-connector-java-5.1.48.jar
sqoop import \
--connect "jdbc:mysql://localhost:3306/move?verifyServerCertificate=false&zeroDateTimeBehavior=round" \
--username "something" \
--password "something" \
--delete-target-dir \
--table users \
--fields-terminated-by "," \
--hive-import \
--create-hive-table \
--hive-table test.users \
-- \
--schema "move"
But what is the use of this -libjars option. I am still confused.
... View more
03-28-2020
08:09 PM
sqoop import -libjars /app/hadoop_users/Mahfooz/sqoop/mysql-connector-java-5.1.48.jar --connect "jdbc:mysql://wwwwwwwww01.xxxx.example.com:3306/move?verifyServerCertificate=false" --username "root" --password "root" --table cities --fields-terminated-by "," --hive-import --create-hive-table --hive-table test.cities_mysql -- --schema "move" Warning: /app/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 20/03/29 03:07:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.1 20/03/29 03:07:52 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 20/03/29 03:07:52 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 20/03/29 03:07:52 INFO tool.CodeGenTool: Beginning code generation 20/03/29 03:07:52 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:874) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:762) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:785) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:288) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:259) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:245) at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:333) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
... View more
Labels:
- Labels:
-
Apache Sqoop