<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question JoltTransformJSON performance issue when using default operation in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/JoltTransformJSON-performance-issue-when-using-default/m-p/226827#M188687</link>
    <description>&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="85730-flowfilesin5min.jpg" style="width: 600px;"&gt;&lt;img src="https://community.cloudera.com/t5/image/serverpage/image-id/15335iEA8FBCA13B2038B2/image-size/medium?v=v2&amp;amp;px=400" role="button" title="85730-flowfilesin5min.jpg" alt="85730-flowfilesin5min.jpg" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;JoltTransformJSON&lt;/STRONG&gt; processor is used in our data pipeline.The Jolt Specification in the data pipeline contains two operations (&lt;STRONG&gt;shift &lt;/STRONG&gt;and &lt;STRONG&gt;default). shift&lt;/STRONG&gt; operation to translates Json fields from input message into database fields and &lt;STRONG&gt;default&lt;/STRONG&gt; operation to read from flow file attribute to database field. The performance was good when we just had jolt_shift operation but the jolt_default operation decreases the performance. The &lt;STRONG&gt;Transform Cache Size &lt;/STRONG&gt;is set 10000 but still we see the performance issue.&lt;/P&gt;&lt;P&gt;consumeKafka -&amp;gt; JoltTransformJSON -&amp;gt; putDatabaseRecord&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Jolt specification&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;[{&lt;/P&gt;&lt;P&gt; "operation": "shift",&lt;/P&gt;&lt;P&gt; "spec": {&lt;/P&gt;&lt;P&gt;   "studentName":"STUDENT_NAME",&lt;/P&gt;&lt;P&gt; "Age":"AGE",&lt;/P&gt;&lt;P&gt; "address_city":"CITY",&lt;/P&gt;&lt;P&gt; "address1":"ADDRESS1",&lt;/P&gt;&lt;P&gt; "zipcode":"POSTLCODE",&lt;/P&gt;&lt;P&gt; "id":"ID"&lt;/P&gt;&lt;P&gt; }&lt;/P&gt;&lt;P&gt;},{&lt;/P&gt;&lt;P&gt;"operation": "default",&lt;/P&gt;&lt;P&gt;"spec":{&lt;/P&gt;&lt;P&gt;      "PRTN_NBR" : "${kafka.partition}"&lt;/P&gt;&lt;P&gt; }&lt;/P&gt;&lt;P&gt;}]&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Input message&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;[{"studentName":"Foo2","Age":"12","address_city":"newyork","address1":"North avenue","zipcode":"123213","id":"103"}]&lt;/P&gt;&lt;P&gt;Please find attached summary of Total Task Duration and FlowFiles in 5 min. Any suggestions or any other alternatives? Thanks in advance.&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="85731-totaltaskdurationin5min.jpg" style="width: 640px;"&gt;&lt;img src="https://community.cloudera.com/t5/image/serverpage/image-id/15336i215F474BB71F52CE/image-size/medium?v=v2&amp;amp;px=400" role="button" title="85731-totaltaskdurationin5min.jpg" alt="85731-totaltaskdurationin5min.jpg" /&gt;&lt;/span&gt;&lt;/P&gt;</description>
    <pubDate>Sun, 18 Aug 2019 01:24:30 GMT</pubDate>
    <dc:creator>subramp_</dc:creator>
    <dc:date>2019-08-18T01:24:30Z</dc:date>
  </channel>
</rss>

