csguna, appreciate your inputs. will try this. ... View more

Thanks Saranvisa for this explanation ... View more

thinking of avoidng duplicates while insertion if this won't cause performacne issue. ... View more

thanks for the suggession, will try this. Is there any way to avoid the above error? ... View more

Actually trying in java, as below   DataFrame updated = joined.selectExpr("id", "cast(col_1 as STRING) col_1", "cast(col_2 as DOUBLE) col_2", "cast(col_11 as STRING) col_11", "cast(col_22 as DOUBLE) col_22" ); updated.write().jdbc(DB_CONNECTION, DB_TABLE3, props);   Still shows the same error, any issue over here ?   ... View more

Thanks for the reply, The peace of code is mentioned below.   DataFrame right = sqlContext.read().jdbc(DB_CONNECTION, "testDB.tab2", props); DataFrame joined = sqlContext.read().jdbc(DB_CONNECTION, "testDB.tab1", props).join(right, "id"); joined.write().jdbc(DB_CONNECTION, DB_TABLE3, props);   Its default file comma delimited format. Please find the full exception is mentioned below.  Exception in thread "main" java.sql.SQLException: [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:AnalysisException: Syntax error in line 1: ....tab3 (id INTEGER , col_1 TEXT , col_2 DOUBLE PRECISIO... ^ Encountered: IDENTIFIER Expected: ARRAY, BIGINT, BINARY, BOOLEAN, CHAR, DATE, DATETIME, DECIMAL, REAL, FLOAT, INTEGER, MAP, SMALLINT, STRING, STRUCT, TIMESTAMP, TINYINT, VARCHAR CAUSED BY: Exception: Syntax error ), Query: CREATE TABLE testDB.tab3 (id INTEGER , col_1 TEXT , col_2 DOUBLE PRECISION , col_3 TIMESTAMP , col_11 TEXT , col_22 DOUBLE PRECISION , col_33 TIMESTAMP ). at com.cloudera.hivecommon.api.HS2Client.executeStatementInternal(Unknown Source) at com.cloudera.hivecommon.api.HS2Client.executeStatement(Unknown Source) at com.cloudera.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.executeHelper(Unknown Source) at com.cloudera.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.execute(Unknown Source) at com.cloudera.jdbc.common.SStatement.executeNoParams(Unknown Source) at com.cloudera.jdbc.common.SStatement.executeUpdate(Unknown Source) at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:302) Caused by: com.cloudera.support.exceptions.GeneralException: [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:AnalysisException: Syntax error in line 1: ....tab3 (id INTEGER , col_1 TEXT , col_2 DOUBLE PRECISIO... ^ Encountered: IDENTIFIER Expected: ARRAY, BIGINT, BINARY, BOOLEAN, CHAR, DATE, DATETIME, DECIMAL, REAL, FLOAT, INTEGER, MAP, SMALLINT, STRING, STRUCT, TIMESTAMP, TINYINT, VARCHAR CAUSED BY: Exception: Syntax error ), Query: CREATE TABLE testDB.tab3 (id INTEGER , col_1 TEXT , col_2 DOUBLE PRECISION , col_3 TIMESTAMP , col_11 TEXT , col_22 DOUBLE PRECISION , col_33 TIMESTAMP ). ... 7 more     ... View more