Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

MapReduceIndexerTool with Kerberos

Solved Go to solution
Highlighted

MapReduceIndexerTool with Kerberos

Contributor

Hi,

I'm trying to index some logs with solr.
I defined the schema.xml and the morphline.conf file. I tried to run this on a Cluster without Kerberos and it works fine (CDH 5.4.0).

When I launch this on a Kerberized Cluster (CDH 5.4.0), I cannot index the logs.

 

I followed the Cloudera documentation 

 

I defined the jaas.conf equal to:

Client {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 keyTab="/tmp/krb5cc_0"
 storeKey=true
 useTicketCache=false
 principal="user/fully.qualified.domain.name@REALM";
};

or (if I already used 'kinit')

 

Client {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=false
 useTicketCache=true
 principal="user/fully.qualified.domain.name@REALM";
 };

but when I launch the following command (from Cloudera documentation)

 

hadoop  jar /opt/cloudera/parcels/CDH/lib/solr/contrib/mr/search-mr-*-job.jar org.apache.solr.hadoop.MapReduceIndexerTool -D 'mapred.child.java.opts=-Xmx2048m' -Djava.security.auth.login.config=/home/user/clouderaSearch/test_collection/jaas.conf --log4j /opt/cloudera/parcels/CDH/share/doc/search-1.0.0+cdh5.4.0+0/examples/solr-nrt/log4j.properties --morphline-file /home/user/clouderaSearch/test_collection/test_morphline.conf --output-dir hdfs://nameservice1:8020/data/test/output_solr --verbose --go-live --zk-host  zookeeper_host:2181/solr --collection test_collection hdfs://nameservice1:8020/data/test/incoming

I get this log:

 

0    [main] INFO  org.apache.solr.common.cloud.SolrZkClient  - Using default ZkCredentialsProvider
32   [main] INFO  org.apache.solr.common.cloud.ConnectionManager  - Waiting for client to connect to ZooKeeper
57   [main-EventThread] INFO  org.apache.solr.common.cloud.ConnectionManager  - Watcher org.apache.solr.common.cloud.ConnectionManager@17fca9f9 name:ZooKeeperConnection Watcher:zookeeper_host:2181/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
57   [main] INFO  org.apache.solr.common.cloud.ConnectionManager  - Client is connected to ZooKeeper
57   [main] INFO  org.apache.solr.common.cloud.SolrZkClient  - Using default ZkACLProvider
72   [main] INFO  org.apache.solr.common.cloud.ZkStateReader  - Updating cluster state from ZooKeeper...
564  [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Cluster reports 40 mapper slots
1040 [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Cluster reports 8 reduce slots
1040 [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Using these parameters: numFiles: 1, mappers: 320, realMappers: 1, reducers: 2, shards: 2, fanout: 2147483647, maxSegments: 1
1040 [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Randomizing list of 1 input files to spread indexing load more evenly among mappers
1200 [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Done. Randomizing list of 1 input files took 5.3011456E7 secs
1217 [main] INFO  org.apache.solr.common.cloud.SolrZkClient  - Using default ZkCredentialsProvider
1217 [main] INFO  org.apache.solr.common.cloud.ConnectionManager  - Waiting for client to connect to ZooKeeper
1230 [main-EventThread] INFO  org.apache.solr.common.cloud.ConnectionManager  - Watcher org.apache.solr.common.cloud.ConnectionManager@74f6a607 name:ZooKeeperConnection Watcher:zookeeper_host:2181/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
1231 [main] INFO  org.apache.solr.common.cloud.ConnectionManager  - Client is connected to ZooKeeper
1231 [main] INFO  org.apache.solr.common.cloud.SolrZkClient  - Using default ZkACLProvider
1231 [main] INFO  org.apache.solr.hadoop.ZooKeeperInspector  - Load collection config from:/collections/test_collection
1250 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/admin-extra.menu-top.html
1260 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/currency.xml
1261 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/protwords.txt
1262 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/mapping-FoldToASCII.txt
1264 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/solrconfig.xml.secure
1265 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/_schema_analysis_synonyms_english.json
1266 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/_rest_managed.json
1267 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/solrconfig.xml
1269 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/_schema_analysis_stopwords_english.json
1271 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/contractions_ga.txt
1272 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_hi.txt
1273 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_eu.txt
1274 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_es.txt
1275 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/contractions_it.txt
1276 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/userdict_ja.txt
1277 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_sv.txt
1278 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stemdict_nl.txt
1280 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stoptags_ja.txt
1281 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_bg.txt
1282 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_hy.txt
1283 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_el.txt
1284 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/hyphenations_ga.txt
1285 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ar.txt
1286 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_fr.txt
1287 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_pt.txt
1288 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_nl.txt
1289 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_th.txt
1290 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_de.txt
1291 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_lv.txt
1292 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/contractions_ca.txt
1293 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ru.txt
1294 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ca.txt
1295 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_da.txt
1296 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_gl.txt
1297 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_no.txt
1298 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_tr.txt
1299 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_fa.txt
1300 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ckb.txt
1301 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_hu.txt
1302 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ro.txt
1303 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_id.txt
1304 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_cz.txt
1305 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ja.txt
1306 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_fi.txt
1307 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_it.txt
1308 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/contractions_fr.txt
1309 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_ga.txt
1310 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/lang/stopwords_en.txt
1311 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/stopwords.txt
1312 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/spellings.txt
1313 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/mapping-ISOLatin1Accent.txt
1314 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/admin-extra.html
1316 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/xslt/luke.xsl
1317 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/xslt/updateXml.xsl
1318 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/xslt/example_atom.xsl
1319 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/xslt/example.xsl
1320 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/xslt/example_rss.xsl
1321 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/synonyms.txt
1322 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/scripts.conf
1323 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/update-script.js
1326 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/query_spatial.vm
1327 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/mime_type_lists.vm
1328 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/results_list.vm
1329 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/cluster_results.vm
1330 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/hit_grouped.vm
1331 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/layout.vm
1332 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/join_doc.vm
1333 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/debug.vm
1334 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/facet_queries.vm
1335 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/query_group.vm
1336 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/main.css
1337 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/VM_global_library.vm
1338 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/pagination_bottom.vm
1339 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/error.vm
1340 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/facets.vm
1341 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/richtext_doc.vm
1342 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/head.vm
1343 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/query_form.vm
1344 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/cluster.vm
1345 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/hit_plain.vm
1346 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/footer.vm
1347 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/jquery.autocomplete.js
1348 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/browse.vm
1349 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/facet_pivot.vm
1350 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/README.txt
1351 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/did_you_mean.vm
1352 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/jquery.autocomplete.css
1353 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/header.vm
1354 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/hit.vm
1355 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/suggest.vm
1356 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/pagination_top.vm
1358 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/query.vm
1359 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/product_doc.vm
1360 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/tabs.vm
1361 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/facet_ranges.vm
1362 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/velocity/facet_fields.vm
1363 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/elevate.xml
1364 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/admin-extra.menu-bottom.html
1365 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/schema.xml
1368 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/clustering/carrot2/lingo-attributes.xml
1369 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/clustering/carrot2/stc-attributes.xml
1370 [main] INFO  org.apache.solr.cloud.ZkController  - Write file /tmp/1431512151138-0/clustering/carrot2/kmeans-attributes.xml
1565 [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Indexing 1 files using 1 real mappers into 2 reducers
1637 [main] ERROR org.apache.hadoop.hdfs.KeyProviderCache  - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
1700 [main] ERROR org.apache.hadoop.hdfs.KeyProviderCache  - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
1701 [main] ERROR org.apache.hadoop.hdfs.KeyProviderCache  - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
2745 [main] ERROR org.apache.hadoop.hdfs.KeyProviderCache  - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
2747 [main] ERROR org.apache.hadoop.hdfs.KeyProviderCache  - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
55781 [main] INFO  org.apache.solr.hadoop.MapReduceIndexerTool  - Done. Indexing 1 files using 1 real mappers into 2 reducers took 1.80719104E10 secs
55895 [main] INFO  org.apache.solr.hadoop.GoLive  - Live merging of output shards into Solr cluster...
55901 [pool-4-thread-2] INFO  org.apache.solr.hadoop.GoLive  - Live merge hdfs://nameservice1:8020/data/test/output_solr/results/part-00000 into http://SolrServer:8983/solr
55901 [pool-4-thread-1] INFO  org.apache.solr.hadoop.GoLive  - Live merge hdfs://nameservice1:8020/data/test/output_solr/results/part-00000 into http://SolrServer:8983/solr
55901 [pool-4-thread-3] INFO  org.apache.solr.hadoop.GoLive  - Live merge hdfs://nameservice1:8020/data/test/output_solr/results/part-00001 into http://SolrServer:8983/solr
55901 [pool-4-thread-4] INFO  org.apache.solr.hadoop.GoLive  - Live merge hdfs://nameservice1:8020/data/test/output_solr/results/part-00001 into http://solrServer:8983/solr
56208 [main] ERROR org.apache.solr.hadoop.GoLive  - Error sending live merge command
java.util.concurrent.ExecutionException: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <html><head><title>Apache Tomcat/6.0.43 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 401 - Authentication required</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Authentication required</u></p><p><b>description</b> <u>This request requires HTTP authentication.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.43</h3></body></html>
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:188)
        at org.apache.solr.hadoop.GoLive.goLive(GoLive.java:126)
        at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:920)
        at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:662)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.solr.hadoop.MapReduceIndexerTool.main(MapReduceIndexerTool.java:649)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <html><head><title>Apache Tomcat/6.0.43 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 401 - Authentication required</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Authentication required</u></p><p><b>description</b> <u>This request requires HTTP authentication.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.43</h3></body></html>
        at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:581)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:229)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:225)
        at org.apache.solr.client.solrj.request.CoreAdminRequest.process(CoreAdminRequest.java:510)
        at org.apache.solr.hadoop.GoLive$1.call(GoLive.java:100)
        at org.apache.solr.hadoop.GoLive$1.call(GoLive.java:89)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
56212 [main] INFO  org.apache.solr.hadoop.GoLive  - Live merging of index shards into Solr cluster took 1.05398328E8 secs
56212 [main] INFO  org.apache.solr.hadoop.GoLive  - Live merging failed

The merge phase fails (Live merge hdfs://nameservice1:8020/data/test/output_solr/results/part-00000 into http://SolrServer:8983/solr), and I cannot visit the page http://SolrServer:8983/solr because an authentication is required.

 

Any idea of why is not working?

 

Many thanks

Alessio

1 ACCEPTED SOLUTION

Accepted Solutions

Re: MapReduceIndexerTool with Kerberos

Expert Contributor

Did you include:

 

HADOOP_OPTS="-Djava.security.auth.login.config=/path/to/jaas.conf"

 

Before your "hadoop jar" command?  IE:

 

 

HADOOP_OPTS="-Djava.security.auth.login.config=/path/to/jaas.conf" hadoop jar /opt/cloudera/parcels/CDH/lib/solr/contrib/mr/search-mr-*-job.jar org.apache.solr.hadoop.MapReduceIndexerTool -D 'mapred.child.java.opts=-Xmx2048m' -Djava.security.auth.login.config=/home/user/clouderaSearch/test_collection/jaas.conf --log4j /opt/cloudera/parcels/CDH/share/doc/search-1.0.0+cdh5.4.0+0/examples/solr-nrt/log4j.properties --morphline-file /home/user/clouderaSearch/test_collection/test_morphline.conf --output-dir hdfs://nameservice1:8020/data/test/output_solr --verbose --go-live --zk-host  zookeeper_host:2181/solr --collection test_collection hdfs://nameservice1:8020/data/test/incoming

 

Notice how "HADOOP_OPTS" and "hadoop jar" are on the same line, that is how they need to be.

2 REPLIES 2

Re: MapReduceIndexerTool with Kerberos

Expert Contributor

Did you include:

 

HADOOP_OPTS="-Djava.security.auth.login.config=/path/to/jaas.conf"

 

Before your "hadoop jar" command?  IE:

 

 

HADOOP_OPTS="-Djava.security.auth.login.config=/path/to/jaas.conf" hadoop jar /opt/cloudera/parcels/CDH/lib/solr/contrib/mr/search-mr-*-job.jar org.apache.solr.hadoop.MapReduceIndexerTool -D 'mapred.child.java.opts=-Xmx2048m' -Djava.security.auth.login.config=/home/user/clouderaSearch/test_collection/jaas.conf --log4j /opt/cloudera/parcels/CDH/share/doc/search-1.0.0+cdh5.4.0+0/examples/solr-nrt/log4j.properties --morphline-file /home/user/clouderaSearch/test_collection/test_morphline.conf --output-dir hdfs://nameservice1:8020/data/test/output_solr --verbose --go-live --zk-host  zookeeper_host:2181/solr --collection test_collection hdfs://nameservice1:8020/data/test/incoming

 

Notice how "HADOOP_OPTS" and "hadoop jar" are on the same line, that is how they need to be.

Re: MapReduceIndexerTool with Kerberos

Contributor

Great. It works !!!

 

Many thanks for your suggestion.

Alessio

 

Don't have an account?
Coming from Hortonworks? Activate your account here