Member since
02-07-2015
23
Posts
3
Kudos Received
0
Solutions
03-20-2015
05:29 PM
Thanks. Actually based on that very same paragraph I ended up integrating jinja2 in my pipeline and have now a very nice framework for generating dynamic workflows and make parameters available anywhere 🙂 Thanks once again.
... View more
03-15-2015
04:33 PM
I can't get parameter substitution to work on my impala sql script even when I have it being passed through beeline (using it via hive2 action on oozie): Script [customs_jp.sql] content:
------------------------
--
--
--
...
CREATE TABLE customers LIKE PARQUET '${INPUT}/_metadata' STORED AS PARQUET;
.. ------------------------
Parameters:
------------------------
INPUT=/user/etl/transfer/customers/output1
------------------------
Beeline command arguments :
-u
jdbc:impala://localhost:21050/;auth=noSasl
-n
etl
-p
DUMMY
-d
org.apache.hive.jdbc.HiveDriver
--hivevar
INPUT=/user/etl/transfer/customers/output1
-f
customers.sql
-a
delegationToken
================================================================= on stderr: Connected to: Impala (version 2.1.2-cdh5)
Driver: ImpalaJDBC (version 02.05.05.1007)
Error: [Simba][JDBC](11975) Unsupported transaction isolation level: 4. (state=HY000,code=11975)
No rows affected (0.145 seconds)
No rows affected (0.009 seconds)
No rows affected (0.043 seconds)
Error: [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:AnalysisException: URI path must be absolute: ${INPUT}/_metadata), Query: CREATE TABLE customsers LIKE PARQUET '${INPUT}/_metadata' STORED AS PARQUET. (state=HY000,code=500051)
i.e. ${INPUT} is not being expanded.... (also tried ${hivevar:INPUT} without success...); I see this bug here which seems related https://issues.cloudera.org/browse/IMPALA-1067 , but discusses impala-shell rather than beeline.... Any ideas ?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
-
Apache Oozie
03-08-2015
06:52 AM
I know this question is long answered, but would use it for a quick follow up : If my cluster has an edge server where I only run Hue, how can I force the the push of client configuration to the server (e.g. for defaultFS), so that hdfs command line has the intended value ? When I deploy at cluster level I see it only pushed to servers with Yarn, Hive and HDFS roles... Thanks
... View more