Installing Solr on yarn using Slider

[#1]

Hi,

I am trying to run solr on yarn using the link lucidworksSolrSlider, apart from taking help from slider.incubator.apache.org/docs/getting_started.html

Here is my folder structure:

[solrs@ip-10-0-0-217 solr-slider]$ ls -lrt
total 131744
-rw-rw-r--. 1 solrs solrs      3182 Dec 10 01:17 README.md
drwxrwxr-x. 4 solrs solrs        32 Dec 10 01:17 package
-rw-rw-r--. 1 solrs solrs      2089 Dec 10 01:17 metainfo.xml
-rw-rw-r--. 1 solrs solrs     11358 Dec 10 01:17 LICENSE
-rw-rw-r--. 1 solrs solrs 134874517 Dec 10 01:37 solr-on-yarn.zip
-rw-rw-r--. 1 solrs solrs       277 Dec 10 01:49 resources-default.json
-rw-rw-r--. 1 solrs solrs      1355 Dec 10 15:33 appConfig-default.json

appConfig-default.json:

{
  "schema": "http://example.org/specification/v2.0.0",
  "metadata": {
  },
  "global": {
    "application.def": "/user/solrs/.slider/package/solryarn/solr-on-yarn.zip",
    "java_home": "/usr/jdk64/jdk1.8.0_40",
    "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/solr-5.2.0-SNAPSHOT",
    "site.global.zk_host": "localhost:2181",
    "site.global.solr_host": "${SOLR_HOST}",
    "site.global.listen_port": "${SOLR.ALLOCATED_PORT}",
    "site.global.xmx_val": "1g",
    "site.global.xms_val": "1g",
    "site.global.gc_tune": "-XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewG$
    "site.global.zk_timeout": "15000",
    "site.global.server_module": "--module=http",
    "site.global.stop_key": "solrrocks",
    "site.global.solr_opts": ""
  },
  "components": {
    "slider-appmaster": {
      "jvm.heapsize": "512M"
    },
    "SOLR": {
    }
  }
}

resources-default.json:

{
  "schema" : "http://example.org/specification/v2.0.0",
  "metadata" : {
  },
  "global" : {
  },
  "components": {
    "slider-appmaster": {
    },
    "SOLR": {
      "yarn.role.priority": "1",
      "yarn.component.instances": "3",
      "yarn.memory": "1024"
    }
  }
}

Could you please suggest me what will be the value of below parameters in appConfig-default.json file:

"site.global.app_root": "${AGENT_WORK_ROOT}/app/install/solr-5.2.0-SNAPSHOT",
"site.global.solr_host": "${SOLR_HOST}",
"site.global.listen_port": "${SOLR.ALLOCATED_PORT}",

Basically where should I find "/app/install/solr-5.2.0-SNAPSHOT"??

My Environment: HDP 2.3, Slider Core-0.80.0.2.3.2.0-2950

Thanks, hoping a quick reply.

[#2]

I guess this is a design limitation of the LucidWorks solr app package. One thing you'd have to do to avoid losing data on restart is to configure Solr to store its data in HDFS. You might be able to do this by setting the solr_opts parameter in the appConfig:

-Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=none -Dsolr.hdfs.confdir=/etc/hadoop/conf -Dsolr.hdfs.home=${DEFAULT_DATA_DIR}

You would also need to configure automatic failover when creating the collection:

curl "http://server:port/solr/admin/collections?action=CREATE&name=collection_name&numShards=2&autoAddReplicas=true"

[#3]

Slider has to run the Solr servers on random ports because it can't control where they are started (more than one might be started on the same host). The next release of Slider will have an anti-affinity container placement option, so that might help. You should be able to connect to each of the Solr instances that Slider creates.

[#4]

@Rakesh Gupta excellent question, are you still having issues with this? Can you accept best answer or provide your own solution?