Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Retrieve File Name using Oozie and/or Shell script executed by oozie

Retrieve File Name using Oozie and/or Shell script executed by oozie

New Contributor

Hello,

 

 

My goal is to use oozie, to start a shell action and retrieve an unknown FILENAME of a parquet file in a specific directory.

 

Ultimately, I will like to create a impala/hive table off of the newly discovered file via an impala-shell all with a few parameters passed to it.

 

My error lies with retrieving that file name via oozie or a shell action.

 

ex. Oozie -> Shell Action -> retrieves the (unknown parquet) file name from directory -> passes filename into the .SQL file that is run in impala-shell (the passing of other parameters is already working.

 

Has anyone successfully done this? I have the oozie workflow sqooping data from SQL into parquet, not knowing what the file name is for the rest of the oozie workflow steps is what is giving me a road block at this point.

 

Any help would be greatly appreciated!

 

thanks!

 

Matt G.

 

 

 

Don't have an account?
Coming from Hortonworks? Activate your account here