Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
Go to solution
How to set job name with MapReduceIndexerTool
Labels:
- Labels:
-
Apache Solr
-
MapReduce
Explorer
Created on ‎10-10-2017 04:57 AM - edited ‎09-16-2022 05:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I'm trying to set a name for indexing jobs launched with
cmd_hdp=$( HADOOP_OPTS="-Djava.security.auth.login.config=jaas.conf" hadoop --config /etc/hadoop/conf.cloudera.yarn \ jar /opt/cloudera/parcels/CDH/lib/solr/contrib/mr/search-mr-*-job.jar \ org.apache.solr.hadoop.MapReduceIndexerTool \ -D morphlineVariable.ZK_HOST=myZKHost:2181/solr \ -D morphlineVariable.COLLECTION=myCollection \ -D mapreduce.map.memory.mb=8192 \ -D mapred.child.java.opts=-Xmx4096m \ -D mapreduce.reduce.java.opts=-Xmx4096m \ -D mapreduce.reduce.memory.mb=8192 \ --output-dir hdfs://myHdfsHost:8020/tmp/my/data \ --morphline-file morphlines/myMorphline.conf \ --log4j log4j.properties \ --go-live \ --collection myCollection \ --zk-host myZKHost:2181/solr \ hdfs://myHdfsHost:8020/my/data/ )
Actual name is "org.apache.solr.hadoop.MapReduceIndexerTool/MorphlineMapper" but I'd like to change it.
I already tried setting -D mapreduce.job.name=myName but it does not change.
1 ACCEPTED SOLUTION
Mentor
Created ‎10-15-2017 09:18 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Currently the MapReduceIndexerTool appears to hardcode the job names, so it does not appear configurable: https://github.com/cloudera/search/blob/cdh5.13.0-release/search-mr/src/main/java/org/apache/solr/ha... (and other such setJobName calls in the driver).
1 REPLY 1
Mentor
Created ‎10-15-2017 09:18 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Currently the MapReduceIndexerTool appears to hardcode the job names, so it does not appear configurable: https://github.com/cloudera/search/blob/cdh5.13.0-release/search-mr/src/main/java/org/apache/solr/ha... (and other such setJobName calls in the driver).
