I have below questions. 1. Is sqoop creating sql query internally ? If yes , then How it is getting created and executed for multiple mapper? 2. Is sqoop using any staging node to load the data ? Or is sqoop loading data directly in data node ? How it behaves for different mapper? 3. How sqoop run parallel for multiple mapper ? Please explain with simple architecture. ... View more

I am planning for HCA - HORTONWORKS CERTIFIED ASSOCIATE Certification in next 2 months. I have already checked the exam objective and it is vast. I am reading below book for certification Apress.Pro.Apache.Hadoop.2nd.Edition Is this sufficient for passing the certification ? Can anyone tell me how to prepare for this certification ? Can someone share sample question being asked in this HCA certification? ... View more

I am trying execute one MapReduce job in java but it is getting stuck at the middle and finally it was timed out. Below is the log 16/09/26 18:46:42 INFO mapreduce.Job: Running job: job_1474692614849_0070 16/09/26 18:46:50 INFO mapreduce.Job: Job job_1474692614849_0070 running in uber mode : false 16/09/26 18:46:50 INFO mapreduce.Job: map 0% reduce 0% 16/09/26 18:47:01 INFO mapreduce.Job: map 33% reduce 0% 16/09/26 18:52:19 INFO mapreduce.Job: Task Id : attempt_1474692614849_0070_m_000000_0, Status : FAILED AttemptID:attempt_1474692614849_0070_m_000000_0 Timed out after 300 secs 16/09/26 18:52:20 INFO mapreduce.Job: map 0% reduce 0% 16/09/26 18:52:30 INFO mapreduce.Job: map 33% reduce 0% 16/09/26 18:57:49 INFO mapreduce.Job: Task Id : attempt_1474692614849_0070_m_000000_1, Status : FAILED AttemptID:attempt_1474692614849_0070_m_000000_1 Timed out after 300 secs Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 16/09/26 18:57:50 INFO mapreduce.Job: map 0% reduce 0% 16/09/26 18:57:59 INFO mapreduce.Job: map 33% reduce 0% 16/09/26 19:03:19 INFO mapreduce.Job: Task Id : attempt_1474692614849_0070_m_000000_2, Status : FAILED AttemptID:attempt_1474692614849_0070_m_000000_2 Timed out after 300 secs 16/09/26 19:03:20 INFO mapreduce.Job: map 0% reduce 0% 16/09/26 19:03:31 INFO mapreduce.Job: map 33% reduce 0% 16/09/26 19:08:50 INFO mapreduce.Job: map 100% reduce 100% 16/09/26 19:08:50 INFO mapreduce.Job: Job job_1474692614849_0070 failed with state FAILED due to: Task failed task_1474692614849_0070_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 16/09/26 19:08:50 INFO mapreduce.Job: Counters: 13 Job Counters Failed map tasks=4 Killed reduce tasks=1 Launched map tasks=4 Other local map tasks=3 Rack-local map tasks=1 Total time spent by all maps in occupied slots (ms)=1311303 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=1311303 Total time spent by all reduce tasks (ms)=0 Total vcore-seconds taken by all map tasks=1311303 Total vcore-seconds taken by all reduce tasks=0 Total megabyte-seconds taken by all map tasks=2685548544 Total megabyte-seconds taken by all reduce tasks=0 Is there any way to debug this mapreduce job ? Please help. ... View more

I am from java background and want to shift my carrier in Big Data world . Which one will give me the mileage - Scala or Python ? ... View more