question Re: spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters in Support Questions

spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

tqthang_hn — Thu, 06 Oct 2016 08:35:50 GMT

1. I'm using a ETL tool and connecting to Spark2-HiveThriftServer2 over connection string URL: "jdbc:hive2://10.30.164.132:10000/nat"

2. I'm facing with below error, if the sql have existed wild card characters "*" : LOAD DATA LOCAL INPATH '${SOURCE_DB_FILE}/V_SOURCE_BTS/${YYYYMMDD}/s0_V_SOURCE_BTS_\*_part_\*' INTO TABLE ${SCHEMA_BI}V_SOURCE_BTS; Couldn't execute SQL: LOAD DATA LOCAL INPATH '/u02/CDR/HAITI/MakeFile/V_SOURCE_BTS/20160927/s0_V_SOURCE_BTS_\*_part_\*' INTO TABLE nat.V_SOURCE_BTS 2016/10/05 14:38:43 - V_SOURCE_BTS - 2016/10/05 14:38:43 - V_SOURCE_BTS - org.apache.spark.sql.AnalysisException: LOAD DATA input path does not exist: /u02/CDR/HAITI/MakeFile/V_SOURCE_BTS/20160927/s0_V_SOURCE_BTS_\*_part_\*;

3. The sql query can execute without error if I changed the sql and remove wild cards to: LOAD DATA LOCAL INPATH '/u02/CDR/HAITI/MakeFile/V_SOURCE_BTS/20160927/s0_V_SOURCE_BTS_20160927_part_0000000' INTO TABLE ${SCHEMA_BI}V_SOURCE_BTS;

4. The problem's happened from Spark 2.0.0.

Re: spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

mqureshi — Thu, 06 Oct 2016 10:16:42 GMT

@Tran Quyet Thang

According to the hive documentation, "filepath can refer to a file (in which case Hive will move the file into the table) or it can be a directory (in which case Hive will move all the files within that directory into the table)."

I don't think, wild cards are allowed in the path.

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables

Re: spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

tqthang_hn — Thu, 06 Oct 2016 10:34:43 GMT

I've posted to spark community. It's a bug of SparkSQL.

https://issues.apache.org/jira/browse/SPARK-17796.

whenever the bug is fixed , It'll be updated to HDP 2.5. with spark 2.0.0 version.

Re: spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

tqthang_hn — Mon, 10 Oct 2016 09:23:58 GMT

Hi Mqureshi,

It's already fixed in the dfdfc46. patched.

When it'll merged to HDP 2.5 (spark 2.0) ?