Support Questions

Find answers, ask questions, and share your expertise

Remote Rpc client disassociated

avatar
Contributor

hi there, I see these errors in my spark streaming application.Dont see anymore logs, I have enabled write ahead logs, and check pointing.So when I have the below errors, I loose an executor and an other comes right back up, and the streaming application does not stop, so what happened to the data ?.I hope I did not loose the data, and I am assuming the data was re processed some how.Please advice.

16/06/30 16:38:23 ERROR cluster.YarnScheduler: Lost executor 9 on sxn19.dcc.localdomain: remote Rpc client disassociated

Thanks

Pradeep

2 REPLIES 2

avatar
Master Mentor
@pradeep arumalla

See this solution mentioned in this link. You may have to play around with the different number to find the optimal setting.

avatar
Contributor

Thanks Neeraj, saw the link before, increased memory like they said, but still see errors like above.And also they dont answer about data loss, reprocessing the data etc .