Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

parameter in beeline script

avatar
Master Collaborator

Hi:

I am trying to execute this and pass the parameter "20160301" to beeline script but doesnt work:

beeline -u jdbc:hive2://hostname:10000 -n xxxx -p xxxx -f /home/hdfs/scripts/hive/store_wordcount.hql 20160301

Any idea how to pas this parameter??

Thanks

1 ACCEPTED SOLUTION

avatar
Master Guru

The correct way has changed a bit. You shouldn't use hiveconf as is shown in the link since it sets parameters and is restricted when you use something like SQLStdAuth.

The correct way to do it is

beeline -u jdbc:hive2://hostname:10000 -n xxxx -p xxxx -f /home/hdfs/scripts/hive/store_wordcount.hql --hivevar day=20160301

You can then use this variable as ${day} in the SQL script. hivevars are only variables, hiveconf could also be settings like mapreduce.map.memory.mb.

View solution in original post

4 REPLIES 4

avatar
Master Guru

The correct way has changed a bit. You shouldn't use hiveconf as is shown in the link since it sets parameters and is restricted when you use something like SQLStdAuth.

The correct way to do it is

beeline -u jdbc:hive2://hostname:10000 -n xxxx -p xxxx -f /home/hdfs/scripts/hive/store_wordcount.hql --hivevar day=20160301

You can then use this variable as ${day} in the SQL script. hivevars are only variables, hiveconf could also be settings like mapreduce.map.memory.mb.

avatar
Master Collaborator

Hi:

I tried it but not working well look:

beeline -u jdbc:hive2://hostname:10000 -n xxxx -p xxxx -f /home/hdfs/scripts/hive/store_wordcount.hql --hivevar day=2016/3/01

and inside de hql script that
LOAD DATA INPATH '/RSI/staging/output/wordcount/${day}' OVERWRITE INTO TABLE temp_wordcount_v3;

the output is.

Error: Error while compiling statement: FAILED: SemanticException Line 1:17 Invalid path ''/RSI/staging/output/wordcount//3/'': No files matching path hdfs://lnxbig05.cajarural.gcr:8020/RSI/staging/output/wordcount/3 (state=42000,code=40000)

avatar
Master Collaborator

Hi:

finally its works like this: with double comillas

RUTAHIVE=$ANYODIR"/"${BATCH:2:1}"/"${BATCH:3}""

beeline -u jdbc:hive2://hostname:10000 -n hive -p temporal01 -f /home/hdfs/scripts/hive/store_wordcount.hql --hivevar date=$RUTAHIVE


avatar
Master Guru

Good that you figured it out. You weren't using special characters in the original question. So yes the parameter needs to be in quotes if the values are not normal letters. You might also have to escape things sometimes.

i.e.

--hivevar "day=2016/3/01"

--hivevar "regex=.*\\|" ( if you want the regex .*\| )

And if you use it in shell scripts you sometimes have to escape more. I once needed 32 \\ in oozie to have 1 backslash in a pig script.