Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

SolrCloud 7.3 Backup and Restore of Indexes and Collections

SolrCloud 7.3 Backup and Restore of Indexes and Collections

New Contributor

Solrcloud 7.3

I have been tasked with writing a backup and restore demo for indexes and collections. I have a question regarding the location when taking a backup. In my address box I have the following URL to get my backup started... http://IPADDRESS:8983/solr/#/collections?action=BACKUP&name=test_catalog&location= ...so the documentation says I should back up to a shared drive, but shared drive where? Is the shared drive on the zookeeper, or on my local machine? If on the zookeeper, do I need to create a backup directory first? And if on my local machine, what do I need to type in as the name and IP Address would not work?

I am also interested to know if it is possible to backup to Azure storage, as that would be ideal?

What is the difference between the location and repository? I would really appreciate any guidance, as there appears to be no step by step on the internet (hard to believe I know), if there is I cannot find it. The official documentation is vague at best (at least for me), any help will be appreciated.

Thank you.

1 REPLY 1

Re: SolrCloud 7.3 Backup and Restore of Indexes and Collections

New Contributor

Hello,

OK, so things have moved on, and now I am hoping someone can help now things have moved on. I have created a mount point on the head node of my HDInsight cluster, this mount point goes to Azure Blob storage (File Share) and I am achieving partial backup, or at least this is what the error is telling me. I think what it really means is that it is not able to do any backups at all as I have yet to see anything appear in the blob storage.

I created a file share using steps from these URLs, the first tells me how to create a file share, the second tells me how to create mount point on an Linux (ubuntu) node to that file share.

https://docs.microsoft.com/en-us/azure/storage/files/storage-how-to-create-file-share

https://docs.microsoft.com/en-us/azure/storage/files/storage-how-to-use-files-linux

I have tested the link to the file share by uploading a txt file to it, which then appears on the Linux node hosting the mount point.

If I execute the following line in the Solr UI, I receive the subsequent error...

http://1STPARTOFIPADDRESS.213:8983/solr/admin/collections?action=BACKUP&name=prod_cat5&collection=pr...

{ "responseHeader":{ "status":500, "QTime":216}, "success":{ "1STPartofIPaddress.202:8983_solr":{ "responseHeader":{ "status":0, "QTime":6}}}, "failure":{ "1STPartofIPaddress.207:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://1STPartofIPaddress.207:8983/solr: Failed to backup core=products_yan_1_replica_n4 because org.apache.solr.common.SolrException: Directory to contain snapshots doesn't exist: file:///mnt/solrbkp/prod_cat5"}, "Operation backup caused exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not backup all replicas", "exception":{ "msg":"Could not backup all replicas", "rspCode":500}, "error":{ "metadata":[ "error-class","org.apache.solr.common.SolrException", "root-error-class","org.apache.solr.common.SolrException"], "msg":"Could not backup all replicas", "trace":"org.apache.solr.common.SolrException: Could not backup all replicas\n\tat org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:258)\n\tat org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:230)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)\n\tat java.lang.Thread.run(Thread.java:748)\n", "code":500}}

If I run the same backup command again, I receive...

{
  "responseHeader":{
    "status":400,
    "QTime":145},
  "Operation backup caused exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: The backup directory already exists: file:///mnt/solrbkp/prod_cat5/",
  "exception":{
    "msg":"The backup directory already exists: file:///mnt/solrbkp/prod_cat5/",
    "rspCode":400},
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.common.SolrException"],
    "msg":"The backup directory already exists: file:///mnt/solrbkp/prod_cat5/",
    "code":400}}

So here it says the location already exists, but it does not exist in the blob store file share where I would expect to find it, and the previous error states it does not exist. So where is it? If I specify the backup name, is the last part of the desired backup location supposed to be created automatically or I'm I correct in specifying it?

Do I need to create the same mount point on multiple nodes? If so, just the nodes that are hosting collections/indexes/shards? Or also on the Head and Zookeeper nodes?

I have tested the link to the file share by uploading a txt file to it, which then appears on the Linux node hosting the mount point.

Thank you in advance for any help or advice.

Regards,

Paul.

Don't have an account?
Coming from Hortonworks? Activate your account here