https://community.cloudera.com/t5/Support-Questions/How-to-get-the-non-group-by-columns-in-spark-structured/m-p/200514#M162535 <P><A rel="user" href="https://community.cloudera.com/users/24259/elangork.html" nodeid="24259">@elango vaithiyanathan</A> </P><P>To get <STRONG>non group by columns</STRONG> after <STRONG>grouped dataframe</STRONG>, we need to use one of the <STRONG>aggregate(agg)</STRONG> function(<STRONG>max, min, mean and sum..etc) </STRONG>for all the non group by columns.</P><P><STRONG>Example:-</STRONG></P><PRE>val grpbyDF = updatedDf.groupBy("ROW_ID","ODS_WII_VERB").agg(&lt;strong&gt;max&lt;/strong&gt;("STG_LOAD_TS"),&lt;strong&gt;min&lt;/strong&gt;("non groupby column"),&lt;strong&gt;mean&lt;/strong&gt;("non groupby column"),&lt;strong&gt;sum&lt;/strong&gt;("non groupby column"))</PRE><P>In the above grpbydf we are grouping by <STRONG>ROW_ID,ODS_WII_VERB an</STRONG>d all non group by columns are in agg function with one of the function(max, min, mean and sum).</P><P>Please Refer to below link for more details about groupBy<BR /><A href="http://spark.apache.org/docs/2.1.0/api/python/pyspark.sql.html#pyspark.sql.DataFrame.groupBy" target="_blank">http://spark.apache.org/docs/2.1.0/api/python/pyspark.sql.html#pyspark.sql.DataFrame.groupBy</A></P> Sun, 04 Feb 2018 13:59:43 GMT Shu_ashu 2018-02-04T13:59:43Z