Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to run beeline command against file on HDFS?

avatar
Rising Star

Hi all,

Sorry for the basic question i've had little success searching online, i just need clarification whether something is possible.

Can i run a beeline command against a file that is in HDFS. I know we use -f in beeline to specify a file when its on the local file system but can this also be done against a file on HDFS.

My use case is that i'd like to run a beeline command through a shell action in Oozie. I'm hitting some issues using Hive2 actions so i wanted to try using a shell action instead

Any help is much appreciated,

Thanks

1 ACCEPTED SOLUTION

avatar
Contributor

@Daniel Perry

Beeline expects the HQL file to be local file system. So If your HQL file is in HDFS, you should first download it to local file system, then feed it to Beeline.

View solution in original post

3 REPLIES 3

avatar
Contributor

@Daniel Perry

Beeline expects the HQL file to be local file system. So If your HQL file is in HDFS, you should first download it to local file system, then feed it to Beeline.

avatar
Rising Star

Thought as much, thanks for confirming

avatar
New Contributor

@Daniel Perry

You can try to use expression and cat command, something like this (but it won't work if you have comments in hql file, so remove all comments):

beeline -e “$(hdfs dfs -cat <hdfs_filename>)”

if you still want to keep comments you can pipeline the output to sed, keep in mind that comments should start from a new line:

beeline -e “$(hdfs dfs -cat <hdfs_filename> | sed -e 's/^--.*//g' )' )”