@Developer Developer As @Felix Albani suggested above i'd go with spawning multiple threads to process the dataframes in parallel. This article has a good example : https://hadoopist.wordpress.com/2017/02/03/how-to-use-threads-in-spark-job-to-achieve-parallel-read-and-writes/ ... View more

Thanks . That got me a bit closer.   I discovered that there was no core created on solrserver2, which i just did and restarted  both servers.   now i am getting Node: solrserver2:8983_solr is not live !   but i see it started in the manager , and no logs either. What am I missing !!   Your help probably saved couple days of reading and frustration ! ... View more