Reply
New Contributor
Posts: 1
Registered: ‎12-04-2017

Retrieve File Name using Oozie and/or Shell script executed by oozie

[ Edited ]

Hello,

 

 

My goal is to use oozie, to start a shell action and retrieve an unknown FILENAME of a parquet file in a specific directory.

 

Ultimately, I will like to create a impala/hive table off of the newly discovered file via an impala-shell all with a few parameters passed to it.

 

My error lies with retrieving that file name via oozie or a shell action.

 

ex. Oozie -> Shell Action -> retrieves the (unknown parquet) file name from directory -> passes filename into the .SQL file that is run in impala-shell (the passing of other parameters is already working.

 

Has anyone successfully done this? I have the oozie workflow sqooping data from SQL into parquet, not knowing what the file name is for the rest of the oozie workflow steps is what is giving me a road block at this point.

 

Any help would be greatly appreciated!

 

thanks!

 

Matt G.

 

 

 

Announcements