@Peter Kim I don't think you can control file sizes other than changing the number of mappers as you seem to be doing, with one exception. If you are in direct mode you can use --direct-split-size 570000000 which will split into approximately 570MB files. ... View more

Your log has this error. Diagnostics: File does not exist: hdfs://xxx-hdfs.abc.net:8020/user/yarn/.hiveJars/hive-exec-1.2.1.2.3.4.0-3485-bb59749376792da886f093283cc8bbdb78c69612f13abcbcedbef00717030c90.jar Check if the /user/yarn directory exists and check that owner:group are yarn:hdfs ... View more

+1 - @Misti Mordoch - This seems to have fixed the problem for someone else I was talking to. You can also check for out of sync hosts with the Rest API: http://:8080/api/v1/clusters//stack_versions/1>:8080/api/v1/clusters/<CLUSTER_NAME>/stack_versions/1 -u <USERNAME> and then Check for       "OUT_OF_SYNC" : [ ] to make sure there are no hosts list as out of sync. Reinstall services for out of sync hosts or remove the hosts and re-add will probably fix it. ... View more

I may have been wrong about adding "/solr" to your zookeepers. I know I had to do that somewhere, but I guess it was when starting solr from the commandline without the "bin/solr start" command. So, you can re-upload your config directory to a configset named "lab". It will create or overwrite the current configset (which is just a ZK directory of your conf directory). The default configset name is the same as your collection. T ./zkcli.sh -zkhost localhost:2181 -cmd upconfig -confdir /opt/lucidworks-hdpsearch/solr/server/solr/configsets/data_driven_schema_configs_hdfs/conf -confname lab If your configset is called testcoll, then do this to show the contents of the solrconfig.xml in zookeeper: ./zkcli.sh -zkhost localhost:2181 -cmd get /configs/lab/solrconfig.xml I recommend running the list command which will dump everything in zookeeper, not just listing files but will print the contents of the files. That's a bit too much, so just pipe it to "less" and then search for your collection name as you would with vi (with / and ? to search). Then you'll see the path to your configs. ./zkcli.sh -zkhost localhost:2181 -cmd list |less you will see something like this (my collection is called testcoll in this example): /configs/testcoll/solrconfig.xml (0) DATA: ...supressed... /configs/testcoll/lang (38) /configs/testcoll/lang/contractions_ga.txt (0) DATA: ...supressed... /configs/testcoll/lang/stopwords_hi.txt (0) DATA: ...supressed... /configs/testcoll/lang/stopwords_eu.txt (0) DATA: ...supressed... /configs/testcoll/lang/stopwords_sv.txt (0) DATA: ...supressed... /configs/testcoll/lang/contractions_it.txt (0) I hope that helps. ... View more

This will upload your config directory to a configset named "testcoll". The default configset name is the same as your collection ./zkcli.sh -zkhost localhost:2181 -cmd upconfig -confdir ../../solr/configsets/data_driven_schema_configs/conf -confname testcoll. If your configset is called testcoll, then do this to show the contents of the solrconfig.xml in zookeeper: ./zkcli.sh -zkhost localhost:2181 -cmd get /configs/testcoll/solrconfig.xml I recommend running the list command which will dump everything in zookeeper, not just listing files but will print the contents of the files. That's a bit too much, so just pipe it to "less" and then search for your collection name as you would with vi (with / and ? to search). Then you'll see the path to your configs. ... View more

@Saurabh Kumar You need to add "/solr" to the end of your zookeeper host:port like this (you probably only need to list one of the zookeepers for the command) ./zkcli.sh -cmd upconfig -confdir /opt/lucidworks-hdpsearch/solr/server/solr/configsets/data_driven_schema_configs_hdfs/conf -confname labs -z m1.hdp22:2181/solr That command will upload the conf directory. I'd also suggest trying the list command ("-cmd list") to see what's in zookeeper. It has been a ehwhile since I have used it and I can't try it at the moment. ... View more