@tonybolt, Solr 7.0.1 was just released this afternoon. ... View more

Just to add to the previous reply, the version of Solr in HDP Search is *exactly* the same as the same version downloaded direct from the Apache Solr project. Besides Solr, the benefits of the package are the connectors for indexing from HDFS, Hive, Pig, HBase, plus Storm and Spark development kits, and the Ambari integration. ... View more

Solr also supports TTL, although I think if the docs are deleted from HBase they should be deleted in Solr automatically. In case you are interested in the Solr TTL feature, it's done through an UpdateRequestProcessor (URP). It's currently only documented in Solr's Javadocs: http://lucene.apache.org/solr/6_4_0/solr-core/org/apache/solr/update/processor/DocExpirationUpdateProcessorFactory.html (replace the '6_4_0' part of that URL to get to the javadocs for your version; this URP has existed since Solr 4.8.0). ... View more

I believe this is a known issue with .zip archives and the Solr ExtractingRequestHandler (aka Solr Cell): https://issues.apache.org/jira/browse/SOLR-2416. The short version of the story is that Tika in this case is not configured to parse the .zip recursively. One of the other suggestions for NiFi processing may be worth exploring in this case. ... View more

Great, thanks @Tony Bolt. I was able to trace the cause of the problem to a seemingly unrelated commit that occurred for the 6.4.0 release, and the good news is the fix has already been committed for an upcoming 6.4.2 release. The release process for that has already started, and we'd expect it to be out within 1-2 weeks. There is no patch to apply, but if you have the ability to build Solr from source, you could try to build locally with "branch_6_4", which is where 6.4.2 will come from, or "branch_6x", which also contains the same fix. If you can't do a local build for any reason, we do already have a 2nd confirmation that the problem is fixed with this upcoming release, so it's certainly not required or expected of you to test it at this point. ... View more

When speaking about scaling capabilities of Solr, there are really only two modes: 1. Master/slave, which is one or more Solr servers running in standalone mode with what the community calls "legacy scaling" features enabled. 2. SolrCloud, which is one or more Solr servers running with ZooKeeper coordinating activity between them. Solr doesn't make the two really obvious, meaning there's no single switch to turn on or off to choose which mode. Both are a set of configuration options that are different depending on the mode you want to use. It's important to note master/slave is not really different from standalone mode because you can work for years with Solr running on a single server and then add index replication (and/or distributed queries) if you need it. It's harder to move from standalone to SolrCloud (although not impossible). Some documentation from the Solr Reference Guide might help: Master/Slave: https://cwiki.apache.org/confluence/display/solr/Legacy+Scaling+and+Distribution SolrCloud: https://cwiki.apache.org/confluence/display/solr/SolrCloud ... View more