Support Questions
Find answers, ask questions, and share your expertise

Pyspark SQL pivot error


Hi All,


i am trying to do pivot on a table registered from a dataframe. sample dataframe is as below.

Fan 2000 USA
Fan 2000 USA
TV 400 India

when i register it as temp view like this df.createOrReplaceTempView("pivot_test")

and use query

spark.sql("SELECT * FROM pivot_test \
where Country IN ('India','USA')").show() works fine.


but below pivot query is giving me error. i think, i am doing some mistake in indentation or the code itself. can someone please help?


spark.sql("SELECT * FROM pivot_test \
pivot(sum(Amount)FOR Country IN ('India','USA'))").show()

ERROR : pyspark.sql.utils.ParseException: u"\nmismatched input 'FROM' expecting <EOF>(line 1, pos 9)\n\n== SQL ==\nSELECT * FROM pivot_test pivot(sum(Amount)FOR Country IN ('India','USA'))\n---------^^^\n"


Thanks in advance.


Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.