Hello, I'm designing ETL for my company. The source database is Oracle. We have HDP 2.6. To make things simple, I have 100s of tables to extract data from. I wrote a parameterised sqoop workflow that I call 100s times - each time with different table name as parameter. The calls are from a fork in another oozie workflow. The problem I'm having is that all the resources of the cluster is being taken up by the oozie launcher and oozie action is not started as there are no resources left. Currently I'm using the default queue for everything. So the question is; Is my design/understanding wrong? Any help would be appreciated. <fork name="startAllExtracts"> <path start="sqoop_action_list"/> <path start="sqoop_card"/> ... </fork> <join name="endAllExtracts" to="end"/> <action name="sqoop_action_list"> <sub-workflow> <app-path>${pathWF}/pwc_05_sqoop</app-path> <propagate-configuration/> <configuration> <property> <name>table_name</name> <value>action_list</value> </property> </configuration> </sub-workflow> <ok to="endAllExtracts"/><error to="fail"/> </action> <action name="sqoop_card"> <sub-workflow> <app-path>${pathWF}/pwc_05_sqoop</app-path> <propagate-configuration/> <configuration> <property> <name>table_name</name> <value>card</value> </property> </configuration> </sub-workflow> <ok to="endAllExtracts"/><error to="fail"/> </action> ... ... View more

Hello, I've been struggling with parameters in command line. Parameters work in interactive shell but I cannot figure out what is wrong. This is the code I'm running. Output should be 9911221199, but only 99 is displayed. [hive@bitest ~]$ beeline -u jdbc:hive2:// --hivevar rdate=112211 -e "select 9${hivevar:rdate}9" Connecting to jdbc:hive2:// Connected to: Apache Hive (version 1.2.1000.2.5.0.0-1245) Driver: Hive JDBC (version 1.2.1000.2.5.0.0-1245) Transaction isolation: TRANSACTION_REPEATABLE_READ OK +------+--+ | _c0 | +------+--+ | 99 | +------+--+ 1 row selected (3.818 seconds) Beeline version 1.2.1000.2.5.0.0-1245 by Apache Hive Closing: 0: jdbc:hive2:// ... View more