Potentially but no guarantees. You would lose those changes on upgrades and would be up to you to keep them in sync. I know it is something that will not be supported but you seem to acknowledge that in the question with "Though KNOX with YARNUI is not officially supported" ... View more

KNOX-1098 is the support for adding proxyUser when it is not there. This hasn't been merged yet. ... View more

This is fixed in at least HDP 2.6.5. I didn't check back farther. The underlying issue was that Knox didn't send the correct headers to Solr. It was passing through the original headers instead of rewriting them. This was fixed for Solr in Knox. ... View more

The amount of memory needed by Ambari Infra Solr by default is too high due to poor configuration choices. I wrote about what simple changes can be made to make Solr significantly more performant on less heap. We have Solr with less than 4GB of heap holding billions of documents. https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html ... View more

@Devendra Singh Here are all the details in a blog: https://risdenk.github.io/2018/02/07/hdf-ambari-mpack-upgrades.html ... View more

@Raffaele S Here are all the details in a blog: https://risdenk.github.io/2018/02/07/hdf-ambari-mpack-upgrades.html ... View more

@Raffaele S - We found a few more things that were broken which we were able to fix. I am working on writing up some more steps for this. We were able to get everything to work. I agree with you that the documentation isn't clear that mpack must go before Ambari upgrade. I'll update this question with more info hopefully soon. ... View more

See my answer about fixing stack_root. "I tried upgrading the management pack after, but didn't help." It looks like you HAVE to do mpack first then Ambari upgrade going from Ambari 2.5.x to Ambari 2.6.x. This will automatically update the stack_root. ... View more

Might want to see this question: https://community.hortonworks.com/answers/158963/view.html ... View more

The issue seems to be Ambari doesn't like the JSON coming back from Stack.get_stack_root(). The format looks like it changed between Ambari 2.5.1 and Ambari 2.5.2 with AMBARI-21430. The commit looks like this: f33a250c0 This is what the new Ambari 2.6 cluster has for stack_root. "stack_root" : "{\"HDF\":\"/usr/hdf\"}", This is what our upgraded Ambari 2.6 cluster has for stack_root. "stack_root" : "/usr/hdf", This can easily be updated with configs.py since this is just cluster-env -> stack_root. An example of doing this: /var/lib/ambari-server/resources/scripts/configs.py -l $(hostname -f) -t 8081 -s https -n CLUSTER_NAME -u USERNAME -p PASSWORD -a set -c cluster-env -k stack_root -v '{"HDF":"/usr/hdf"}' PS: It also looks like this requires that you have the 3.0.2 mpack installed (even if you have 3.0.1 HDF installed) since that is compatible with Ambari 2.6.x (see matrix here). Without the 3.0.2 mpack, we were getting different errors (like the ones in the original question). To install 3.0.2 mpack (make sure purge flag is set): sudo ambari-server install-mpack --mpack=https://s3.amazonaws.com/public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.0.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.0.2.0-76.tar.gz --purge --verbose --force ... View more

In Ambari under Solr there is a box for environment stuff. I don't have Ambari in front of me right now but you should be able to add a line like so: SOLR_OPTS="$SOLR_OPTS -javaagent...." ... View more

If the hostnames stay the same then you won't have any issue. Changing the hostnames will cause issues. For reference we moved physical clusters between data centers. In the new DC boxes got new IP addresses but had same hostnames. We turned the cluster back on in the new DC and everything worked. ... View more

This was expanded to a blog post with additional details: https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html There are two things I would check: 1) Check the field caches by navigating to Ambari Infra Solr UI - http://HOSTNAME:8886/solr/#/ranger_audits_shard1_replica1/plugins/cache?entry=fieldCache,fieldValueCache 2) If you were to take a heap dump, you could open the heap dump in Eclipse Memory Analyzer and check this biggest heap offender. My assumption is that the majority of the heap is being used by uninverting the _version_ field since it is being used for sorting or other things instead of indexing. This isn't a misuse by your part but a problem with how Ranger is using Solr. I was able to fix this by turning on DocValues for the _version_ field. I am currently working on opening a Ranger JIRA and posting to the Ranger user list to try to address this. The change to DocValues will require you to delete the collection and recreate. We have been running with 4GB heap with no issues. Previously we needed a heap of almost 20GB to handle the uninverted cached values (this grew overtime). ... View more

Is this infra-solr or a regular Solr instance? Assuming from the tags this is infra-solr. Are you putting Ranger audit logs in Infra Solr? ... View more

@Sean Roberts - Was Solr completely initialized when you were hitting it with curl? Did you restart Solr or reload the ranger_audit collection? One way to get a better error message: http://hostname:8886/solr/ranger_audits_shard1_replica1/query?debug=query&q=*:*&distrib=false This will query only a single shard and not try to get bounced around. On my Ambari Infra server, if I reboot and issue queries before the collections are completely loaded I get the request is a replay hitting /solr/ranger_audits/ but if I hit a single shard I get this error message: { "responseHeader":{ "status":503, "QTime":0, "params":{ "q":"*:*", "debug":"query"}}, "error":{ "metadata":[ "error-class","org.apache.solr.common.SolrException", "root-error-class","org.apache.solr.common.SolrException"], "msg":"no servers hosting shard: shard2", "code":503}} The shard2 could be any shard that is still initializing. You should also be able to see this in the Solr Admin UI -> Cloud and see which shards aren't green. ... View more

I'm pretty sure the Capacity Scheduler permissions query against the Ambari API is incorrect. The change that introduced this is here: https://github.com/apache/ambari/commit/82ccf224f0142b0b63c13e6ac0d58d45f55dd5ab The related JIRA: https://issues.apache.org/jira/browse/AMBARI-16866 The correct url: http://AMBARI/api/v1/users/USERNAME/privileges?PrivilegeInfo/permission_name=AMBARI.ADMINISTRATOR|(PrivilegeInfo/permission_name.in(CLUSTER.ADMINISTRATOR,CLUSTER.OPERATOR)&PrivilegeInfo/cluster_name=CLUSTER) This returns the list of privileges. The incorrect url: http://AMBARI/api/v1/users/USERNAME?privileges/PrivilegeInfo/permission_name=AMBARI.ADMINISTRATOR|(privileges/PrivilegeInfo/permission_name.in(CLUSTER.ADMINISTRATOR,CLUSTER.OPERATOR)&privileges/PrivilegeInfo/cluster_name=CLUSTER) This returns a blank page. ... View more

The OS detection for the solr script was improved in Solr 6.3 with SOLR-9475. Prior to this, the solr script from bin/solr wouldn't detect the OS properly when running under Docker. There are some other OS detection improvements as well from that JIRA. ... View more

Do you have Ambari Infra running on the same node as where you installed the Solr service? ... View more

@Andrew Bumstead opened SOLR-8538. This was resolved as not a problem since JDK 1.7.0_79 fixed the issue. If you use JDK 8 and Solr <6.2.0 you will most likely have to manually update hadoop-commons to 2.6.1+. The packaged Solr 5.5.0 and 5.5.1 for HDP 2.3, 2.4, and 2.5 seem to have the original packaged Hadoop 2.6.0 dependencies. If you don't upgrade the Hadoop dependencies under Solr to 2.6.1+, you will most likely get Kerberos ticket renewal issues or you have to follow the steps outlined below by @Jonas Straub to enable full Kerberos authentication. ... View more

Would be good to link to the Apache Solr documentation for this specifically: https://cwiki.apache.org/confluence/display/solr/Parallel+SQL+Interface https://cwiki.apache.org/confluence/display/solr/Solr+JDBC+-+Apache+Zeppelin ... View more

Can we store index in local file system as well as in HDFS? Simultaneaously. Solr supports both indices on local file system and HDFS. It just depends on the directory factory being used for the collection. A single collection cannot span different file systems. What should be possible (haven't tested) is to have say two collections (one local filesystem, one hdfs filesystem) and then use Solr collection aliases to search both collections at once. Also can we set a custom job periodically to upload index from local file system to HDFS? There is nothing different between an index on HDFS and a local index. Moving an index between the two can be done carefully making sure that the correct index ends up in the right location. If the index is moved improperly (ie: shards don't line up) then you will get bad results. ... View more

@Abdelkrim Hadjidj & @Sunile Manjee - I posted more about the Apache Solr and Apache Zeppelin (incubating) integration here: https://www.linkedin.com/pulse/apache-solr-jdbc-zeppelin-incubating-kevin-risden http://blogs.avalonconsult.com/blog/search/apache-solr-jdbc-apache-zeppelin-incubating/ There is a whole series about the Solr 6.0 JDBC driver. ... View more