- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
HIVE - Concatenates all columns easily
- Labels:
-
Apache Hive
Created ‎07-09-2018 08:04 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to write hql that concatenates all columns in a table with comma separated.
ex)
target_table
col1 | col2 | col3 | col4 |
a01 | b01 | 100 | X1 |
a02 | b02 | 200 | X2 |
a03 | b03 | 300 | X3 |
Execute SELECT query
SELECT COL1 ,COL2 ,COL3 ,COL4 ,CONCAT???(T1.*) AS CONCAT_COL -- SQL for Concatenates all columns FROM target_table AS T1
Expected result
col1 | col2 | col3 | col4 | CONCAT_COL |
a01 | b01 | 100 | X1 | a01,b01,100,X1 |
a02 | b02 | 200 | X2 | a02,b02,100,X2 |
a03 | b03 | 300 | X3 | a03,b03,300,X3 |
- CONCAT(COL1, ',', COL2, ',', COL2, ...) is a one of idea ,however if target table has too many columns or number of columns will increase in the future, I have to write long hql and it is difficult to manage.
- CONCAT_WS(',' , T1.*) FROM target_tbl AS T1 is FAILED 'NullPointerException null'
Is there anyone has good idea for this ??
Thank you.
Created ‎07-09-2018 02:23 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm not sure what your original goal is - the following approach may help; or possibly not...
Beeline has some output format features which enables it to output the resultset in csv/tsv2 format. see: https://community.hortonworks.com/questions/25789/how-to-dump-the-output-from-beeline.html?childToVi...
I think with `concat_ws` you are very close; but I'm afraid its not possible to convert a struct into an array (or at least into a map - in which case you might be able to use `map_values` to obtain the array)
Created ‎11-12-2018 06:06 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You need to use CONCAT_WS (separator,col1,col2,...colX) :
select CONCAT_WS ('',c12,c13,c14,c15,c16) FROM your_table;
,Use CONCAT_WS , where you give the separator as 1st term and then the list of columns to concat :
select CONCAT_WS (' ',c12,c13,c14,c15,c16) from your_table) ;
