Support Questions

Find answers, ask questions, and share your expertise

Using impala-shell to export data to dat format with no delimeter and no extra quotes

avatar
New Contributor

This question has been put out on 2018.03.19, and i met this question again.

The link:https://community.cloudera.com/t5/Support-Questions/Export-query-to-csv-with-no-delimeter-and-no-ext...

 

My sql  query :

select name from ${table};

the hue shows:

1.png

and i try to export data to dat format by using impala-shell.

 

Here is my query 

impala-shell -i ${load_balancer} -u ${username} -q "SELECT IFNULL(CAST(NAME AS STRING),'') FROM DBNAME.TBNAME LIMIT 100" -B -o ${file_path};

the result data.dat shows:

2.png

I try to change query to:

impala-shell -i ${load_balancer} -u ${username} -q "SELECT IFNULL(CAST(NAME AS STRING),'') FROM DBNAME.TBNAME LIMIT 100" -B -o ${file_path} '--output_delimiter=,'

the result is same.

 

I want :   KURDISTAN ("THE PKK")

NOT:"KURDISTAN (""THE PKK"")"

 

How can I accomplish this through impala-shell?

Thank you

1 REPLY 1

avatar
Master Collaborator

Hi,

 

Where you are redirecting the output to? for csv file?

What is the output from impala-shell terminal if you do not redirect the output?

 

It looks like caused by the csv module when impala is using it to export the data.

 

    # csv.writer expects a file handle to the input.

    # cStringIO is used as the temporary buffer.

    temp_buffer = StringIO()

    writer = csv.writer(temp_buffer, delimiter=self.field_delim,

                        lineterminator='\n', quoting=csv.QUOTE_MINIMAL)

    writer.writerows(rows)

 

Seems to be we cannot change this since it is needs to modified at code level.

 

https://github.com/apache/impala/blob/014c455aaaa38010ae706228f7b439c080c0bc7d/shell/shell_output.py...

 

Regards,

Chethan YM