Member since
10-16-2016
46
Posts
9
Kudos Received
0
Solutions
02-28-2017
07:02 PM
Thanks for your answer @Dan Zaratsian ..i implemented a solution similar to what you suggested but still facing performance issue could you please take a look at https://community.hortonworks.com/questions/85745/spark-writing-data-to-amazondynamodb-is-slow.html
... View more
02-06-2017
02:42 PM
You can check if the values you configured are actually in use with this command: hdfs getconf -confKey yarn.nodemanager.resource.memory-mb or from the ResourceManager UI: <rm.host>:8088/conf If the values are in sync, you can also check the ResourceManager logs for further information.
... View more
03-29-2017
09:55 PM
5 Kudos
Hi @Aditya Mamidala, You also need to set yarn.nodemanager.localizer.cache.target-size-mb along with yarn.nodemanager.localizer.cache.cleanup.interval-ms. Also the resources are not cleaned up if they are being used by running containers. More details given in below blog https://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/
... View more
12-26-2016
09:44 PM
As you question states it and how this exception block shows it: Caused by: java.lang.NullPointerException at DevMain$anonfun$5.apply(DevMain.scala:2)
at (DevMain.scala:2)
at scala.collection.Iterator$anon$11.next(Iterator.scala:328) Hard to guess without debugging the code. My no-brainer advice :): debug your code line by line to detect when you pass a null value where a non-value is expected.
... View more