New Contributor
Posts: 1
Registered: ‎12-04-2017

Retrieve File Name using Oozie and/or Shell script executed by oozie

[ Edited ]




My goal is to use oozie, to start a shell action and retrieve an unknown FILENAME of a parquet file in a specific directory.


Ultimately, I will like to create a impala/hive table off of the newly discovered file via an impala-shell all with a few parameters passed to it.


My error lies with retrieving that file name via oozie or a shell action.


ex. Oozie -> Shell Action -> retrieves the (unknown parquet) file name from directory -> passes filename into the .SQL file that is run in impala-shell (the passing of other parameters is already working.


Has anyone successfully done this? I have the oozie workflow sqooping data from SQL into parquet, not knowing what the file name is for the rest of the oozie workflow steps is what is giving me a road block at this point.


Any help would be greatly appreciated!




Matt G.




New solutions