Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Highlighted

I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

@colorsoflife@gmail.com

6 REPLIES 6
Highlighted

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

The most common way to do bulk load into hive is not with JDBC but by ingesting files into HDFS and then use the LOAD INTO or CREATE EXTERNAL TABLE/INSERT INTO commands. ( The first one if your table has exactly the format of your data, the second one if you want to do transformations. ). The newest version of HDP also has Hive Transactions enabled which would allow you to insert large amounts of data through JDBC however they are more suited for streaming reads and update situations and are still pretty new.

Highlighted

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

I want to get row wise log for the Bulk insert, at least for failed insertions, like by any key value, which row failed. Any idea, how I can achieve that??

Highlighted

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

@Mamta Chawla What do you mean by row insert failure for table you are trying to handle and capture i.e. bad records or wrong format etc.?

Highlighted

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Yes. For whatever reason, Insertion fails, I want to get in log file

Highlighted

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Is there anyone, who know the solution??

Highlighted

Re: I am using, JDBC to connect with hive and execute a query for bulk insert. I want to capture row wise, insert statement log. How can it be done??

Expert Contributor

I create a file containing like 1000 create table statements and run beeline -f filename. The logs are stored in hive.log .we can check the log files for "Parsing command - create table .." to "Updated the size of the table" and other logs

Don't have an account?
Coming from Hortonworks? Activate your account here