Support Questions
Find answers, ask questions, and share your expertise

How to run beeline command against file on HDFS?

Solved Go to solution

How to run beeline command against file on HDFS?

Contributor

Hi all,

Sorry for the basic question i've had little success searching online, i just need clarification whether something is possible.

Can i run a beeline command against a file that is in HDFS. I know we use -f in beeline to specify a file when its on the local file system but can this also be done against a file on HDFS.

My use case is that i'd like to run a beeline command through a shell action in Oozie. I'm hitting some issues using Hive2 actions so i wanted to try using a shell action instead

Any help is much appreciated,

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to run beeline command against file on HDFS?

Cloudera Employee

@Daniel Perry

Beeline expects the HQL file to be local file system. So If your HQL file is in HDFS, you should first download it to local file system, then feed it to Beeline.

View solution in original post

3 REPLIES 3

Re: How to run beeline command against file on HDFS?

Cloudera Employee

@Daniel Perry

Beeline expects the HQL file to be local file system. So If your HQL file is in HDFS, you should first download it to local file system, then feed it to Beeline.

View solution in original post

Re: How to run beeline command against file on HDFS?

Contributor

Thought as much, thanks for confirming

Re: How to run beeline command against file on HDFS?

New Contributor

@Daniel Perry

You can try to use expression and cat command, something like this (but it won't work if you have comments in hql file, so remove all comments):

beeline -e “$(hdfs dfs -cat <hdfs_filename>)”

if you still want to keep comments you can pipeline the output to sed, keep in mind that comments should start from a new line:

beeline -e “$(hdfs dfs -cat <hdfs_filename> | sed -e 's/^--.*//g' )' )”