Planned to setup hadoop ecosystem to consume data from RDMS and store it in hbase. perform transformation and data cleaning in hbase and load that data into hive warehouse. consume the warehouse data in ML (spark, flink and scala). so need to know which is best practise to set up for above requirement? 1. Single cluster which has a master and 5 to 10 slaves with maximum GB RAM and diskspace. 2. Three cluster (3 master and respective slaves) for Hbase, hive and ML seperately. Data will get increases every years as million to trillion. so need to know the best practice for above requirements ( 1 or 2 ) ? ... View more

i have schedule a sqoop job(import data from mysql to hive) in oozie workflow.xml and its is running fine. but after some time it throws an error like 1. No job jar file set. User classes may not be found. See Job or Job#setJar(String). 2. Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] I have searched many sites and found it is due to lib issue. so i have placed all required jar files inside share lib including jdbc connector jar and provided path in oozie. but its still throwing same error. i didn't get any idea on how to resolve that. Any suggestion would be really helpfull. ... View more