How to set job name with MapReduceIndexerTool

niser — Fri, 16 Sep 2022 12:22:43 GMT

Hello,

I'm trying to set a name for indexing jobs launched with

cmd_hdp=$(
HADOOP_OPTS="-Djava.security.auth.login.config=jaas.conf" hadoop --config /etc/hadoop/conf.cloudera.yarn \
  jar /opt/cloudera/parcels/CDH/lib/solr/contrib/mr/search-mr-*-job.jar \
  org.apache.solr.hadoop.MapReduceIndexerTool \
  -D morphlineVariable.ZK_HOST=myZKHost:2181/solr \
  -D morphlineVariable.COLLECTION=myCollection \
  -D mapreduce.map.memory.mb=8192 \
  -D mapred.child.java.opts=-Xmx4096m \
  -D mapreduce.reduce.java.opts=-Xmx4096m \
  -D mapreduce.reduce.memory.mb=8192 \
  --output-dir hdfs://myHdfsHost:8020/tmp/my/data \
  --morphline-file morphlines/myMorphline.conf \
  --log4j log4j.properties \
  --go-live \
  --collection myCollection \
  --zk-host myZKHost:2181/solr \
 hdfs://myHdfsHost:8020/my/data/
)

Actual name is "org.apache.solr.hadoop.MapReduceIndexerTool/MorphlineMapper" but I'd like to change it.

I already tried setting -D mapreduce.job.name=myName but it does not change.

Re: How to set job name with MapReduceIndexerTool

Harsh J — Mon, 16 Oct 2017 04:18:34 GMT

Currently the MapReduceIndexerTool appears to hardcode the job names, so it does not appear configurable: https://github.com/cloudera/search/blob/cdh5.13.0-release/search-mr/src/main/java/org/apache/solr/hadoop/MapReduceIndexerTool.java#L812 (and other such setJobName calls in the driver).

question How to set job name with MapReduceIndexerTool in Archives of Support Questions (Read Only)

How to set job name with MapReduceIndexerTool

Re: How to set job name with MapReduceIndexerTool