Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

unable to create Solr collection / index

avatar
Explorer

Hi Team,

I've downloaded CDH quickstart vm and started to play with HUE - single node.

When I tried to create a Solr collection by running 

solrctl --zk quickstart.cloudera:2181/solr collection --create hue_solr -s 1

but i got an error that i want to use --solr as the cluster can't detect the solr automatically, I've googled this error and found people talking abut specifying the solr url as well as the zookeeper url, when I run the command as solrctl --zk quickstart.cloudera:2181/solr --solr quickstart.cloudera:8983/solr collection --create hue_solr -s 1 and this time I got another error "curl: option --negotiate: the installed libcurl version doesn't support this
curl: try 'curl --help' or 'curl --manual' for more information
Error: A call to SolrCloud WEB APIs failed:"

found this kb article https://www.cloudera.com/documentation/enterprise/5-7-x/topics/search_solrctl_examples.html talking about disabling proxy so I run it and the command became NO_PROXY='*' solrctl --zk quickstart.cloudera:2181/solr --solr quickstart.cloudera:8983/solr collection --create hue_solr -s 1 but still getting same error as above.

I came from Windows background so my Linux knowledge is a bit limited.

I  tried to restart Solr service but this didn't help.

Any help would be appreciated.

Thanks in advance

 

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor
Hi Osama,

Sorry, didn't realize you were managing using Cloudera Manager (CM). In
that case you should be using CM to start and stop the Solr services. I
would try to get thing reset by stopping Solr via CM, make sure it's
stopped on the system, and then start it with CM. If it doesn't stop via
CM control then you may have to manually stop it this time. I am not
familiar with how the manual init scripts interact w/ the CM methods for
service control so you may be in an odd state.

* Stop Solr from CM
* Check from the command line "ps -ef | grep solr"
* If needed, stop Solr from the command line "service solr-server stop"
* Check from the command line "ps -ef | grep solr"
* Start Solr from CM
* Check from the command line "ps -ef | grep solr"

One you're satisfied that Solr is actually up and running test your
collection creation step. If it fails, check to see that Solr is still
running. I think you're just having some sort of stability problem with
Solr. You may need to increase the heap for Solr a little.

Nick

View solution in original post

25 REPLIES 25

avatar
New Contributor

I am facing the same issue when creating SOLR collection in my quickstart VM (error below) -

 

solrctl collection --create findrcoll -s 1
Error: can't discover Solr URI. Please specify it explicitly via --solr.

 

I also verified that solr server daemon is running. I am not using Cloudera Manager in my VM.
[cloudera@quickstart findrcoll]$ sudo service solr-server status
Solr server daemon is running                              [  OK  ]

 

Any help is appreciated. Thanks

avatar
Contributor

Is this a bug in CDH 5.15?

 

The same happens for me.

 

This works

solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr instancedir --create ds-collection /tmp/ds-collection

  This gives me Solr url not found error:

solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --create ds-collection -s 2 -r 2

The above command used to work in older 5.1x versions. Please advise.  

avatar
Contributor

@bgooley  Please see my post above. Thanks.

avatar
Contributor

Anyone?

avatar
Can you run with the --trace option and see if theres any indication of why the ZK_ENSEMBLE is not being used?

-pd

avatar
Contributor
solrctl --zk
ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr
collection --delete ds-collection --trace
Error: can't discover Solr URI. Please specify it explicitly via --solr.

avatar
Contributor

[centos@ip-172-31-81-167 data]$ solrctl --trace --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --delete ds-collection
+ shift 1
+ test 5 '!=' 0
+ case "$1" in
+ '[' 5 -gt 1 ']'
+ SOLR_ZK_ENSEMBLE=ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr
+ shift 2
+ test 3 '!=' 0
+ case "$1" in
+ break
+ '[' -n '' ']'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ '[' -z ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr ']'
+ SOLR_ADMIN_ZK_CMD='ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS} LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties ${SOLR_HOME}/bin/zkcli.sh -zkhost $SOLR_ZK_ENSEMBLE 2>&3'
+ '[' 3 -gt 0 ']'
+ test 3 '!=' 0
+ case "$1" in
+ '[' --delete = --list ']'
+ '[' --delete = --list-snapshots ']'
+ '[' 3 -gt 2 ']'
+ case "$2" in
++ echo --delete
++ tr '[a-z]-' '[A-Z] '
+ COL_ACTION=' DELETE'
++ echo DELETE
+ eval solr_webapi ''\''/admin/collections?action=DELETE&name=ds-collection'\'''
++ solr_webapi '/admin/collections?action=DELETE&name=ds-collection'
++ '[' -z '' ']'
+++ get_solr_protocol
++++ eval 'ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS}' 'LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties' '${SOLR_HOME}/bin/zkcli.sh' -zkhost '$SOLR_ZK_ENSEMBLE' '2>&3' -cmd get /clusterprops.json
+++++ ZKCLI_JVM_FLAGS=
+++++ LOG4J_PROPS=/etc/solr/conf/log4j.properties
+++++ /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/solr//bin/zkcli.sh -zkhost ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr -cmd get /clusterprops.json
+++ SOLR_STATE='{"urlScheme":"http"}'
+++ echo '{"urlScheme":"http"}'
+++ grep -i urlScheme
+++ grep -q -i https
+++ echo http
++ local SOLR_PROTOCOL=http
+++ get_solr_state /live_nodes
+++ sed -ne 's#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'
++++ eval 'ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS}' 'LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties' '${SOLR_HOME}/bin/zkcli.sh' -zkhost '$SOLR_ZK_ENSEMBLE' '2>&3' -cmd ls /live_nodes
+++++ ZKCLI_JVM_FLAGS=
+++++ LOG4J_PROPS=/etc/solr/conf/log4j.properties
+++++ /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/solr//bin/zkcli.sh -zkhost ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr -cmd ls /live_nodes
+++ SOLR_STATE='/live_nodes (5)
/live_nodes/ip-172-31-81-167.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-90-165.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-82-140.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-92-169.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-93-54.ec2.internal:8983_solr (0)'
+++ echo '/live_nodes (5)
/live_nodes/ip-172-31-81-167.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-90-165.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-82-140.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-92-169.ec2.internal:8983_solr (0)
/live_nodes/ip-172-31-93-54.ec2.internal:8983_solr (0)'
+++ grep -v '^/ '
++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT'
++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-81-167.ec2.internal:8983/solr
++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT'
++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-90-165.ec2.internal:8983/solr
++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT'
++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-82-140.ec2.internal:8983/solr
++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT'
++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-92-169.ec2.internal:8983/solr
++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT'
++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-93-54.ec2.internal:8983/solr
++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT'
++ curl -i --retry 5 -s -L -k --negotiate -u : http://localhost:/solr
++ '[' -n '' ']'
++ die 'Error: can'\''t discover Solr URI. Please specify it explicitly via --solr.'
++ '[' '!' -z 'Error: can'\''t discover Solr URI. Please specify it explicitly via --solr.' ']'
++ echo Error: 'can'\''t' discover Solr URI. Please specify it explicitly via --solr.
Error: can't discover Solr URI. Please specify it explicitly via --solr.

 

 

## Seems like the collection command is not working in CDH 5.15. Instancedir works fine with zk quoram.

avatar
It looks like its failing contacting the solr nodes. are you able to run this successfully from the host where the solrctl command is running?

curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-82-140.ec2.internal:8983/solr

-pd

avatar
Contributor

Yes, running all commands on NN2 which happens to be a Solr node also. I was able to create the collections using the collections api, but solrctl works for some commands and just not the collections command. Maybe some bug, please check the code. Thanks

avatar
Does the curl command I noted return an actual web page? From the output, it is possible there is something wrong with the curl binaries that you are using...

-pd