Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to dump the output to a file from Beeline?

Solved Go to solution

Re: How to dump the output to a file from Beeline?

New Contributor

Hi 苏 斌,

I have not tried INSERT OVERWRITE DIRECTORY when i switch to beeline. Try to use this command:-

beeline -u jdbc:hive2://somehost_ip/ --silent=true --outputformat=csv2 -f hive.hql >op.csv

Re: How to dump the output to a file from Beeline?

New Contributor

Hi Team,

I am trying to print output along with number records selected or number record affected messages in the output file but I am unable to print it.

I have tried with --verbose=true also but it is not helping.

<p>

beeline -u "jdbc:hive2://abc.nsroot.net:10000/;principal=hive/abc.nsroot.net@ABC.NSROOT.NET;ssl=true " -f /home/test1.hql > testresult1.txt

</p>

hql files has only one query "select * from mytable;"

can any one suggest me on this?

Thanks & regards,

Kamleshkumar Gujarathi

Re: How to dump the output to a file from Beeline?

New Contributor

@Benjamin Leonhardi I am facing a problem while fetching ouput in csv2 format. Since my data itself contains comma hence that specific column is encapsulated in special character "^@". Can you please help of how to avoid it or if I can get the output in a pipe delimited form?

Re: How to dump the output to a file from Beeline?

Contributor

Also, just wanted to point out that, depending on the version of Hive being used, may default to buffering within the beeline client.  Be sure to enable 'incremental' fetches of data from the Hive server when dealing with large result sets.

 

--incremental=[true/false]

Defaults to true from Hive 2.3 onwards, before it defaulted to false. When set to false, the entire result set is fetched and buffered before being displayed, yielding optimal display column sizing. When set to true, result rows are displayed immediately as they are fetched, yielding lower latency and memory usage at the price of extra display column padding. Setting --incremental=true is recommended if you encounter an OutOfMemoryException.

 

https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients

Highlighted

Re: How to dump the output to a file from Beeline?

Contributor

Another option for large data sets, if ordering doesn't matter, is to create an EXTERNAL table with the necessary delimiters and issue an INSERT statement into the table instead of a SELECT statement in beeline.  To copy the data locally, issue a:

 

hdfs dfs -cat /my/table/*

"Order doesn't matter' because the cat application will not necessarily read the files in proper order.  If an ORDER BY is included in the query, the contents of each file will be in order, but the files may be read out of order by the 'cat' application.

 

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/FileSystemShell.html#cat

Don't have an account?
Coming from Hortonworks? Activate your account here