I am trying to print output along with number records selected or number record affected messages in the output file but I am unable to print it.
I have tried with --verbose=true also but it is not helping.
beeline -u "jdbc:hive2://abc.nsroot.net:10000/;principal=hive/abc.nsroot.net@ABC.NSROOT.NET;ssl=true " -f /home/test1.hql > testresult1.txt
hql files has only one query "select * from mytable;"
can any one suggest me on this?
Thanks & regards,
@Benjamin Leonhardi I am facing a problem while fetching ouput in csv2 format. Since my data itself contains comma hence that specific column is encapsulated in special character "^@". Can you please help of how to avoid it or if I can get the output in a pipe delimited form?
Also, just wanted to point out that, depending on the version of Hive being used, may default to buffering within the beeline client. Be sure to enable 'incremental' fetches of data from the Hive server when dealing with large result sets.
Defaults to true from Hive 2.3 onwards, before it defaulted to false. When set to false, the entire result set is fetched and buffered before being displayed, yielding optimal display column sizing. When set to true, result rows are displayed immediately as they are fetched, yielding lower latency and memory usage at the price of extra display column padding. Setting --incremental=true is recommended if you encounter an OutOfMemoryException.
Another option for large data sets, if ordering doesn't matter, is to create an EXTERNAL table with the necessary delimiters and issue an INSERT statement into the table instead of a SELECT statement in beeline. To copy the data locally, issue a:
hdfs dfs -cat /my/table/*
"Order doesn't matter' because the cat application will not necessarily read the files in proper order. If an ORDER BY is included in the query, the contents of each file will be in order, but the files may be read out of order by the 'cat' application.
Please try this, this will only output the stadard output of the not error or logs:
beeline -u jdbc:hive2://somehost_ip/ -f hive.hql 2> /dev/null >op.txt
Please check below command, here 2> /dev/null will consume all the logs and error. It will now allow standard output to be shown:
beeline -u jdbc:hive2://somehost_ip/ -f 2> /dev/null hive.hql >op.txt
if you like this please give me kudos. Thanks!!!