Support Questions
Find answers, ask questions, and share your expertise

I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

@colorsoflife@gmail.com

6 REPLIES 6

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

The most common way to do bulk load into hive is not with JDBC but by ingesting files into HDFS and then use the LOAD INTO or CREATE EXTERNAL TABLE/INSERT INTO commands. ( The first one if your table has exactly the format of your data, the second one if you want to do transformations. ). The newest version of HDP also has Hive Transactions enabled which would allow you to insert large amounts of data through JDBC however they are more suited for streaming reads and update situations and are still pretty new.

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

I want to get row wise log for the Bulk insert, at least for failed insertions, like by any key value, which row failed. Any idea, how I can achieve that??

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

@Mamta Chawla What do you mean by row insert failure for table you are trying to handle and capture i.e. bad records or wrong format etc.?

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Yes. For whatever reason, Insertion fails, I want to get in log file

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Is there anyone, who know the solution??

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Expert Contributor

I create a file containing like 1000 create table statements and run beeline -f filename. The logs are stored in hive.log .we can check the log files for "Parsing command - create table .." to "Updated the size of the table" and other logs