https://community.cloudera.com/t5/Support-Questions/Spark-How-to-make-calls-to-database-using-foreachPartition/m-p/123339#M86083 <P>We have spark streaming job ..writing data to AmazonDynamoDB using <STRONG>foreachRDD</STRONG> but it is very slow with our consumption rate at 10,000/sec and writing 10,000 takes 35min ...this is the code piece</P><P>From research learnt that using <STRONG>foreachpartition</STRONG> and creating a connection per partition will help ..but not sure how to go about writing code for it ..will greatly appreciate if someone can help with this ...Also any other suggestion to speed up writing is greatly appreciated</P><PRE>tempRequestsWithState.foreachRDD { rdd =&gt; if ((rdd != null) &amp;&amp; (rdd.count() &gt; 0) &amp;&amp; (!rdd.isEmpty()) ) { rdd.foreachPartition { case (topicsTableName, hashKeyTemp, attributeValueUpdate) =&gt; { val client = new AmazonDynamoDBClient() val request = new UpdateItemRequest(topicsTableName, hashKeyTemp, attributeValueUpdate) try client.updateItem(request) catch { case se: Exception =&gt; println("Error executing updateItem!\nTable ", se) } } case null =&gt; } } } </PRE> Fri, 24 Feb 2017 12:37:24 GMT chmamidala 2017-02-24T12:37:24Z