Reply
Contributor
Posts: 25
Registered: ‎12-27-2016

Solr Indexing using Morphline with Kerberos

Hi, 

I am trying to index some data using SolrCtl. 

I have create the Solr index "testindex" using the solrctl commands (instancedir & collection). I could see the solr index in both Hue Search->Indexes and also in Solr Admin page.

 

I am trying to index the solr using Morphline. For that I created Morphline conf file "testindex.conf". Below is the content of the conf file.

 

SOLR_LOCATOR {
zkHost : "ZookperHost1:2181,ZookperHost2:2181,ZookperHost3:2181/solr"
collection : testindex
}

morphlines : [
{
id: morphline_0
importCommands: ["org.kitesdk.**", "org.apache.solr.**"]
commands : [
{
readAvroParquetFile {
projectionSchemaString: """{
"type" : "record",
"name" : "cloudera_solr_avro",
"namespace" : "batch1",
"fields" : [ {
"name" : "name",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "age",
"type" : [ "null", "string" ],
"default" : null
}
]
}"""
}
}
{
extractAvroPaths {
flatten : true
paths : {

Data_Element : /name
Supplier_Channel : /age
}
}
}
{
generateUUID {
field: id
}
}
{
sanitizeUnknownSolrFields {
# Location from which to fetch Solr schema
solrLocator : ${SOLR_LOCATOR}
}
}
{
loadSolr {
solrLocator : ${SOLR_LOCATOR}
}
}
]
}
]

 

 Since my cluster is Kerberosed with SSL & TSL enabled, I also using the jaas-client.conf file. Below is the content of the jaas-client.conf file

 

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="PATH of the keytab file"
storeKey=true
useTicketCache=false
debug=true
principal="fully qualified principal name";
};

 

Below is the command on how I am executing the whole script

 

HADOOP_OPTS="-Djava.security.auth.login.config= LocalPath/jaas-client.conf" hadoop jar $SearchMrjar $className -D "mapreduce.job.maps=16" -D "mapred.child.java.opts=-Xmx8192m" -D "mapreduce.map.memory.mb=8192" -D "mapreduce.reduce.memory.mb=8192" --morphline-file testindex.conf --output-dir $outputDirPath --zk-host $zookeeperHosts --collection $CollectionName --go-live $ParquetPath

 

When I run the above script, I am getting the below error

 

16/12/27 05:09:56 INFO hadoop.MapReduceIndexerTool: Done. Indexing 1 files using 1 real mappers into 1 reducers took 32.0959 secs
16/12/27 05:09:56 INFO hadoop.GoLive: Live merging of output shards into Solr cluster...
16/12/27 05:09:56 INFO hadoop.GoLive: Live merge hdfs:<HDFSPATH>/results/part-00000 into https:<SOLRHTTPSPORT>/solr
16/12/27 05:09:56 INFO impl.HttpClientUtil: Setting up SPNego auth with config: <LOCALJAASPATH>/jaas-client.conf
16/12/27 05:09:56 WARN client.TargetAuthenticationStrategy: Authentication scheme Basic not supported
Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is <KEYTABPATH>/<KEYTABNAME> refreshKrb5Config is false principal is <PRINCIPALNAME> tryFirstPass is false useFirstPass is false storePass is false clearPass is false
principal is <PRINCIPALNAME>
Will use keytab
Commit Succeeded

16/12/27 05:09:57 ERROR hadoop.GoLive: Error sending live merge command
java.util.concurrent.ExecutionException: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: User:solr not allowed to do 'DECRYPT_EEK' on 'testKey'
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.hadoop.GoLive.goLive(GoLive.java:126)
at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:954)
at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:681)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.solr.hadoop.MapReduceIndexerTool.main(MapReduceIndexerTool.java:668)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: User:solr not allowed to do 'DECRYPT_EEK' on 'sesameKey'
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:620)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:228)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:224)
at org.apache.solr.client.solrj.request.CoreAdminRequest.process(CoreAdminRequest.java:510)
at org.apache.solr.hadoop.GoLive$1.call(GoLive.java:100)
at org.apache.solr.hadoop.GoLive$1.call(GoLive.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

 

I am not sure what's wrong in my approach and what I am missing. Any help is greatly appreciated.

 

 

Posts: 177
Topics: 8
Kudos: 28
Solutions: 19
Registered: ‎07-16-2015

Re: Solr Indexing using Morphline with Kerberos

[ Edited ]

Does the "solr" user has enough permission on the keytab for this to work ?

 

Since we are using the jaas configuration with "useTicketCache=true", I can't really help you more than that.

Cloudera Employee
Posts: 273
Registered: ‎01-09-2014

Re: Solr Indexing using Morphline with Kerberos

It looks like you are using hdfs encryption zones. If so, does your solr user have permissions to DECRYPT_EEK in the kms-acls.xml safety valve? (look in the KMS Service)

-pd
Contributor
Posts: 25
Registered: ‎12-27-2016

Re: Solr Indexing using Morphline with Kerberos

Thanks for the guidance, I also tried with the use useTicketCache=true as below, but that didn't change the error.

 

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
useTicketCache=true
debug=true
principal="fully.qualified.domain.name@<YOUR-REALM";
};

Contributor
Posts: 25
Registered: ‎12-27-2016

Re: Solr Indexing using Morphline with Kerberos

I will check and get back to you.
Posts: 177
Topics: 8
Kudos: 28
Solutions: 19
Registered: ‎07-16-2015

Re: Solr Indexing using Morphline with Kerberos

[ Edited ]

Hey Ravi,

 

You can't use these two properties at the same time :

useKeyTab=true
useTicketCache=true

 

You should choose one of them.

If you use "useTicketCache=true", the content is a little different and you will have to kinit the ticket before submiting the job.

 

But I think you should check the point raised around the encryption.

Highlighted
Contributor
Posts: 25
Registered: ‎12-27-2016

Re: Solr Indexing using Morphline with Kerberos

It was a typo while posting my code here, so the useKeyTab=false and useTicketCache=true is what i used.