Member since
06-09-2016
529
Posts
129
Kudos Received
104
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1098 | 09-11-2019 10:19 AM | |
7139 | 11-26-2018 07:04 PM | |
1485 | 11-14-2018 12:10 PM | |
3022 | 11-14-2018 12:09 PM | |
2168 | 11-12-2018 01:19 PM |
06-18-2018
01:44 PM
@Satya Nittala To delete all documents single core: curl 'http://localhost:8080/solr/update' -H "Content-type: text/xml" --data-binary '<delete><query>*:*</query></delete>'
curl 'http://localhost:8080/solr/update' -H "Content-type: text/xml" --data-binary '<commit />' mutliple core: curl 'http://localhost:8080/solr/<core-name>/update' -H "Content-type: text/xml" --data-binary '<delete><query>*:*</query></delete>' curl http://localhost:8080/solr/update -H "Content-type: text/xml" --data-binary '<commit />' If you like to delete using a query filter instead of all, just use the filter criteria you like. For example the following will delete only the documents for city=Santiago <delete>
<query>city:Santiago</query>
</delete> Also note the above curl commands are not specifying --negotiate -u : please add those if you have kerberos enabled cluster. HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.
... View more
06-18-2018
01:36 PM
@Robert Cornell if the above answer helped you please take a moment to login and click the "accept" link on the answer.
... View more
06-18-2018
01:33 PM
@Avinash A if the above answer helped you please take a moment to login and click the "accept" link on the answer.
... View more
06-18-2018
01:33 PM
@Ivan Diaz if the above answer helped you please take a moment to login and click the "accept" link on the answer.
... View more
06-18-2018
01:11 PM
1 Kudo
@ilia kheifets The difference may come from yarn.scheduler.minimum-allocation-mb, spark memory overhead and jvm. For more information you may want to read the following article: https://blog.csdn.net/oufuji/article/details/50387104 HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.
... View more
06-16-2018
01:18 PM
@JAy PaTel Try using full path to your spark-submit command in shell script: /usr/hdp/current/spark2-client/bin/spark-submit --class org.apache.<main>--master local[2]<jar_file_path><HDFS_input_path><HDFS_output_path> HTH
... View more
06-16-2018
01:11 PM
1 Kudo
@Robert Cornell In my experience the verbosity level in spark 2 has greatly been reduced compared to 1.6 - specially in the interactive interpreters like spark-shell. . Please check default log4j in ambari > spark2 > conf and make sure the global log4j file is not setting any loggers to INFO. . If you wish to point to a specific log4j file, depending on the master and deployment mode you need to use one or more properties: #yarn-client mode bin/spark-submit --master yarn --deploy-mode client --files /path/to/log4j/log4j.properties --conf "spark.executor.extraJavaOptions='-Dlog4j.configuration=log4j.properties'" --driver-java-options "-Dlog4j.configuration=file:/path/to/log4j/log4j.properties" #yarn-cluster mode bin/spark-submit --master yarn --deploy-mode cluster --files /path/to/log4j/log4j.properties --conf "spark.executor.extraJavaOptions='-Dlog4j.configuration=log4j.properties'" --conf "spark.driver.extraJavaOptions='-Dlog4j.configuration=log4j.properties'" HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.
... View more
06-15-2018
04:16 PM
1 Kudo
@Avinash A Spark shell is only intended to be use for testing and perhaps development of small applications - is only an interactive shell and should not be use to run production spark applications. For production application deployment you should use spark-submit. The last one will also allow you to run applications in yarn-cluster mode. HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.
... View more
06-15-2018
01:49 PM
@Tom C the above service name was <clustername>_hadoop right? Perhaps is a configuration issue as you mentioned. Since its not working as it should you could also turn log4j.logger.org.apache.ranger.authorization=DEBUG in the hdfs log4j to get more information from logs.
... View more
06-15-2018
01:25 PM
@Tom C Perhaps policies are not sync correctly to the Namenode host. You can quickly check by: 1. In Ranger Admin UI perform a change on any hdfs policy - Add a user for example/ 2. Navigate to Ranger Admin UI > Audit > Plugin tab. 3. You should see a new entry at the top - make sure export date is for current time and plugin id should be hdfs (if you dont see this entry showing here after time you made the change this means the hdfs ranger plugin is not downloading policies and probably this explains why is not working as expected) HTH
... View more