How to run beeline command against file on HDFS?

[#1]

Hi all,

Sorry for the basic question i've had little success searching online, i just need clarification whether something is possible.

Can i run a beeline command against a file that is in HDFS. I know we use -f in beeline to specify a file when its on the local file system but can this also be done against a file on HDFS.

My use case is that i'd like to run a beeline command through a shell action in Oozie. I'm hitting some issues using Hive2 actions so i wanted to try using a shell action instead

Any help is much appreciated,

Thanks

[#2 · Accepted solution]

@Daniel Perry

Beeline expects the HQL file to be local file system. So If your HQL file is in HDFS, you should first download it to local file system, then feed it to Beeline.

[#3]

Thought as much, thanks for confirming

[#4]

@Daniel Perry

You can try to use expression and cat command, something like this (but it won't work if you have comments in hql file, so remove all comments):

beeline -e “$(hdfs dfs -cat <hdfs_filename>)”

if you still want to keep comments you can pipeline the output to sed, keep in mind that comments should start from a new line:

beeline -e “$(hdfs dfs -cat <hdfs_filename> | sed -e 's/^--.*//g' )' )”