Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to set job name with MapReduceIndexerTool

avatar
Explorer

Hello,

 

I'm trying to set a name for indexing jobs launched with

cmd_hdp=$(
HADOOP_OPTS="-Djava.security.auth.login.config=jaas.conf" hadoop --config /etc/hadoop/conf.cloudera.yarn \
  jar /opt/cloudera/parcels/CDH/lib/solr/contrib/mr/search-mr-*-job.jar \
  org.apache.solr.hadoop.MapReduceIndexerTool \
  -D morphlineVariable.ZK_HOST=myZKHost:2181/solr \
  -D morphlineVariable.COLLECTION=myCollection \
  -D mapreduce.map.memory.mb=8192 \
  -D mapred.child.java.opts=-Xmx4096m \
  -D mapreduce.reduce.java.opts=-Xmx4096m \
  -D mapreduce.reduce.memory.mb=8192 \
  --output-dir hdfs://myHdfsHost:8020/tmp/my/data \
  --morphline-file morphlines/myMorphline.conf \
  --log4j log4j.properties \
  --go-live \
  --collection myCollection \
  --zk-host myZKHost:2181/solr \
 hdfs://myHdfsHost:8020/my/data/
)

 

Actual name is "org.apache.solr.hadoop.MapReduceIndexerTool/MorphlineMapper" but I'd like to change it.

I already tried setting -D mapreduce.job.name=myName but it does not change.

1 ACCEPTED SOLUTION

avatar
Mentor
Currently the MapReduceIndexerTool appears to hardcode the job names, so it does not appear configurable: https://github.com/cloudera/search/blob/cdh5.13.0-release/search-mr/src/main/java/org/apache/solr/ha... (and other such setJobName calls in the driver).

View solution in original post

1 REPLY 1

avatar
Mentor
Currently the MapReduceIndexerTool appears to hardcode the job names, so it does not appear configurable: https://github.com/cloudera/search/blob/cdh5.13.0-release/search-mr/src/main/java/org/apache/solr/ha... (and other such setJobName calls in the driver).