Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Set destination filenames during dynamic partition Inserts in Hive

Highlighted

Set destination filenames during dynamic partition Inserts in Hive

New Contributor

Hello,

When we insert data from staging table into a production table using dynamic partition inserts, the files created at the partition directory are like: 0000_0.

However, say, for a process where data is loaded on a daily basis, after the first data insertion in a partition, the file names are like 0000_0_copy_1 for the second day, 0000_0_copy_2 for the third day and so on...

I want to create a filename like so: partitionName_datestamp [ex. IND_20173107] so that it helps to maintain a logical and relevant file structure for any manual intervention needs.

I am aware that we can achieve this by executing a shell script after Hive jobs.

But, can we control this from within Hive?

Regards,

Anirban.

Don't have an account?
Coming from Hortonworks? Activate your account here