It is not recommended to downgrade Spark from 1.6 ot 1.5 as 1.5 is not tested with HDP 2.4.2 and might lead to issues.
Thanks and Regards,
View solution in original post
it seems No, HDP didn't talk about Spark 1.6.1 downgrade compatibility with HDP2.4.2
ref link: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark...
you can refer to the discussion under this thread,