Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Scoop import text data problem

Highlighted

Scoop import text data problem

Explorer

Dear All,

Sir we a importing SQL data through scoop in hive of text which has all special characters and when we take a count of that table is 5000from SQL table its and cross checking the same in hive it gives 6143 there is a difference of 1143 records which are null how to avoid this please help us getting this sorted.

2 REPLIES 2
Highlighted

Re: Scoop import text data problem

Contributor

Hello,

 

Can you please share the exact sqoop command you are running at your end.

 

Also, I think your Hive table is a TextFormat table(correct me if I am wrong).

 

If you have the table in Textformat, this means your data from Source SQL DB contains change line characters such as "\n" in some of your fields.

 

To resolve the issue, import the data to an ORC Hive table.

Example on how to create ORC table in Hive:

CREATE TABLE test_details_txt(
visit_id INT,
store_id SMALLINT) STORED AS ORC; 
Highlighted

Re: Scoop import text data problem

Cloudera Employee

You can pass arguments to handle null strings :

 --null-string '\\N' --null-non-string '\\N'

Refer https://sqoop.apache.org/docs/1.4.3/SqoopUserGuide.html for more detail 

Don't have an account?
Coming from Hortonworks? Activate your account here