- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Remote Rpc client disassociated
- Labels:
-
Apache Spark
Created ‎07-05-2016 04:54 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi there, I see these errors in my spark streaming application.Dont see anymore logs, I have enabled write ahead logs, and check pointing.So when I have the below errors, I loose an executor and an other comes right back up, and the streaming application does not stop, so what happened to the data ?.I hope I did not loose the data, and I am assuming the data was re processed some how.Please advice.
16/06/30 16:38:23 ERROR cluster.YarnScheduler: Lost executor 9 on sxn19.dcc.localdomain: remote Rpc client disassociated
Thanks
Pradeep
Created ‎07-05-2016 05:04 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
See this solution mentioned in this link. You may have to play around with the different number to find the optimal setting.
Created ‎07-05-2016 05:22 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Neeraj, saw the link before, increased memory like they said, but still see errors like above.And also they dont answer about data loss, reprocessing the data etc .
