Support Questions
Find answers, ask questions, and share your expertise

Hadoop: MiniCluster setup - questions

Hadoop: MiniCluster setup - questions

Explorer

minicluster.txtHi,

I would like to test my pig udf and creating test scripts.

The script will create a PigServer that runs in MapReduce mode. For this I am writing script to start and stop the MiniCluster.

I have taken code from the link:

https://wiki.apache.org/hadoop/HowToDevelopUnitTests. Could you please help me to understand the concept on MiniCluster. Below are my question:

1. How to start the DFS cluster? I do not see any start method. 2. The code creates an empty YarnConfiguration but not uses it. It uses hdfs conf to set the yarn properties. Is it correct?

Could you please respond as this needs to be resolved today.

My code looks like in the attachement. Is this the correct approach.

Thank you.