Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

spark exception when reading a parquet file

avatar
New Member

when I try to read parquet file from Azure datalake container from databricks, I am getting spark exemption. Below is my query

 

 


import pyarrow.parquet as pq
from pyspark.sql.functions import *
from datetime import datetime
data = spark.read.parquet(f"/mnt/data/country/abb/countrydata.parquet")

 

 

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 14.0 failed 4 times, most recent failure: Lost task 0.3 in stage 14.0 (TID 35) (10.135.39.71 executor 0): org.apache.spark.SparkException: Exception thrown in awaitResult:

 

what does this mean? What I need to do for this?

1 REPLY 1

avatar
Expert Contributor

@shamly  Can you share full stack trace?