https://community.cloudera.com/t5/Support-Questions/Fetch-distinct-values-of-a-column-in-Dataframe-using-Spark/m-p/174638#M136901 <P>Its the same code as above without collect()</P><PRE> preProcessedDataFrame.registerTempTable("tApplicationid") preProcessedDataFrame.select(ApplicationId).distinct().foreach(record =&gt; { val applicationId = record.getAs[String](ApplicationId) val selectedApplicationDataFrame = sqlContext.sql("SELECT * FROM tApplicationId WHERE ApplicationId = " + applicationId) logger.info("selectedApplicationId: " + applicationId) //DO FURTHER PROCESSSING on selectedApplicationDataFrame..... }) </PRE> Tue, 16 Aug 2016 02:05:56 GMT kaz_narasimhan 2016-08-16T02:05:56Z