HELP PLEASE!!!, Failed to write at least 1000 rows to Kudu; Sample errors: Timed out: cannot complete before timeout:

[#1]

I have an exception thrown when trying to write a data frame to kudu of size 524GB.

After calling this writing part: 

df.write.format("org.apache.kudu.spark.kudu").option(
            "kudu.master",
            "master1:port,master2:port,master3:port",
).option("kudu.table", f"impala::{schema}.{table}").mode("append").save()

this exception is thrown:

java.lang.RuntimeException: PendingErrors overflowed. Failed to write at least 1000 rows to Kudu; Sample errors: Timed out: cannot complete before timeout: Batch{operations=1000, tablet="0bc1e2a497ab4306b6861f81dc678d9f" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=0bc1e2a497ab4306b6861f81dc678d9f, attempt=26, TimeoutTracker(timeout=30000, elapsed=29585), Trace Summary(29585 ms): Sent(26), Received(26), Delayed(26), MasterRefresh(0), AuthRefresh(0), Truncated: false

here is the yarn.log for this spark job that throws the error

• Yarn log for a spark job 

Here is the error that is thrown when inserting data to kudu.

I really appreciate any help you can provide.

Thanks in advance!

[#2]

Hi @as30 ,

Can you share if you found the solution to the problem?

Thanks.