I have one doubt that somehow push down predicates are not working have quick read into this article https://engineering.dynatrace.com/blog/optimizing-spark-queries-with-filter-pushdown/   ... View more