Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Solr Indexing using Morphline with Kerberos

Highlighted

Solr Indexing using Morphline with Kerberos

Explorer

Hi, 

I am trying to index some data using SolrCtl. 

I have create the Solr index "testindex" using the solrctl commands (instancedir & collection). I could see the solr index in both Hue Search->Indexes and also in Solr Admin page.

 

I am trying to index the solr using Morphline. For that I created Morphline conf file "testindex.conf". Below is the content of the conf file.

 

SOLR_LOCATOR {
zkHost : "ZookperHost1:2181,ZookperHost2:2181,ZookperHost3:2181/solr"
collection : testindex
}

morphlines : [
{
id: morphline_0
importCommands: ["org.kitesdk.**", "org.apache.solr.**"]
commands : [
{
readAvroParquetFile {
projectionSchemaString: """{
"type" : "record",
"name" : "cloudera_solr_avro",
"namespace" : "batch1",
"fields" : [ {
"name" : "name",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "age",
"type" : [ "null", "string" ],
"default" : null
}
]
}"""
}
}
{
extractAvroPaths {
flatten : true
paths : {

Data_Element : /name
Supplier_Channel : /age
}
}
}
{
generateUUID {
field: id
}
}
{
sanitizeUnknownSolrFields {
# Location from which to fetch Solr schema
solrLocator : ${SOLR_LOCATOR}
}
}
{
loadSolr {
solrLocator : ${SOLR_LOCATOR}
}
}
]
}
]

 

 Since my cluster is Kerberosed with SSL & TSL enabled, I also using the jaas-client.conf file. Below is the content of the jaas-client.conf file

 

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="PATH of the keytab file"
storeKey=true
useTicketCache=false
debug=true
principal="fully qualified principal name";
};

 

Below is the command on how I am executing the whole script

 

HADOOP_OPTS="-Djava.security.auth.login.config= LocalPath/jaas-client.conf" hadoop jar $SearchMrjar $className -D "mapreduce.job.maps=16" -D "mapred.child.java.opts=-Xmx8192m" -D "mapreduce.map.memory.mb=8192" -D "mapreduce.reduce.memory.mb=8192" --morphline-file testindex.conf --output-dir $outputDirPath --zk-host $zookeeperHosts --collection $CollectionName --go-live $ParquetPath

 

When I run the above script, I am getting the below error

 

16/12/27 05:09:56 INFO hadoop.MapReduceIndexerTool: Done. Indexing 1 files using 1 real mappers into 1 reducers took 32.0959 secs
16/12/27 05:09:56 INFO hadoop.GoLive: Live merging of output shards into Solr cluster...
16/12/27 05:09:56 INFO hadoop.GoLive: Live merge hdfs:<HDFSPATH>/results/part-00000 into https:<SOLRHTTPSPORT>/solr
16/12/27 05:09:56 INFO impl.HttpClientUtil: Setting up SPNego auth with config: <LOCALJAASPATH>/jaas-client.conf
16/12/27 05:09:56 WARN client.TargetAuthenticationStrategy: Authentication scheme Basic not supported
Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is <KEYTABPATH>/<KEYTABNAME> refreshKrb5Config is false principal is <PRINCIPALNAME> tryFirstPass is false useFirstPass is false storePass is false clearPass is false
principal is <PRINCIPALNAME>
Will use keytab
Commit Succeeded

16/12/27 05:09:57 ERROR hadoop.GoLive: Error sending live merge command
java.util.concurrent.ExecutionException: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: User:solr not allowed to do 'DECRYPT_EEK' on 'testKey'
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.hadoop.GoLive.goLive(GoLive.java:126)
at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:954)
at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:681)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.solr.hadoop.MapReduceIndexerTool.main(MapReduceIndexerTool.java:668)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: User:solr not allowed to do 'DECRYPT_EEK' on 'sesameKey'
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:620)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:228)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:224)
at org.apache.solr.client.solrj.request.CoreAdminRequest.process(CoreAdminRequest.java:510)
at org.apache.solr.hadoop.GoLive$1.call(GoLive.java:100)
at org.apache.solr.hadoop.GoLive$1.call(GoLive.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

 

I am not sure what's wrong in my approach and what I am missing. Any help is greatly appreciated.

 

 

6 REPLIES 6

Re: Solr Indexing using Morphline with Kerberos

Super Collaborator

Does the "solr" user has enough permission on the keytab for this to work ?

 

Since we are using the jaas configuration with "useTicketCache=true", I can't really help you more than that.

Re: Solr Indexing using Morphline with Kerberos

Explorer

Thanks for the guidance, I also tried with the use useTicketCache=true as below, but that didn't change the error.

 

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
useTicketCache=true
debug=true
principal="fully.qualified.domain.name@<YOUR-REALM";
};

Re: Solr Indexing using Morphline with Kerberos

Super Collaborator
It looks like you are using hdfs encryption zones. If so, does your solr user have permissions to DECRYPT_EEK in the kms-acls.xml safety valve? (look in the KMS Service)

-pd

Re: Solr Indexing using Morphline with Kerberos

Explorer
I will check and get back to you.

Re: Solr Indexing using Morphline with Kerberos

Super Collaborator

Hey Ravi,

 

You can't use these two properties at the same time :

useKeyTab=true
useTicketCache=true

 

You should choose one of them.

If you use "useTicketCache=true", the content is a little different and you will have to kinit the ticket before submiting the job.

 

But I think you should check the point raised around the encryption.

Re: Solr Indexing using Morphline with Kerberos

Explorer

It was a typo while posting my code here, so the useKeyTab=false and useTicketCache=true is what i used.