Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Filename extract and insert into column values hive

avatar
Rising Star
 
1 REPLY 1

avatar
Master Guru

@Gayathri Devi

You can use INPUT__FILE__NAME(gives all input filenames of the table) virtual column and construct your query then store the results of your query to final table.

You need to create a temp table and keep your akolp9app1a_170905_0000.txt file in that table location.

Then use

hive> select INPUT__FILE__NAME from table; //this statement results your akolp9app1a_170905_0000.txt filename
+---------------------------------------------------------------------------------+--+
|                                input__file__name                                |
+---------------------------------------------------------------------------------+--+
| /apps/hive/warehouse/sales/akolp9app1a_170905_0000.txt  			|
+---------------------------------------------------------------------------------+--+

So then you can use all your string functions like substring on the input_file_name filed and keep your hostname,date fileds extracted from the input__file__name field.

hive> select substring(INPUT__FILE__NAME,20,30) hostname,substring(INPUT__FILE__NAME,40,50) `date` from table;

Then you can have final table that you can insert the above select statement hostname,date values.

hive> insert into finaltable select substring(INPUT__FILE__NAME,20,30) hostname,substring(INPUT__FILE__NAME,40,50) `date` from table;

For more references:-

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+VirtualColumns