Hello There,   Am seeing the same error here:   2020-08-13 19:39:01 WARN FileTxnLog:338 - fsync-ing the write ahead log in SyncThread:2 took 4963ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide 2020-08-13 19:39:01 WARN SendAckRequestProcessor:64 - Closing connection to leader, exception during packet send.   Thanks, Rajat. Would you please update what the solution was... ... View more

@Jacob Paul Try to increase the kryoserializer buffer value after you initialized spark context/spark session. change the property name spark.kryoserializer.buffer.max to spark.kryoserializer.buffer.max.mb conf.set("spark.kryoserializer.buffer.max.mb", "512") Refer to this and this link for more details regards to this issue. ... View more

@Jacob Paul I believe your flowfiles having source-date1 attribute with value 20181119112100. Then change your update attribute property values as source-date as ${source-date1:substring(0,8)} source-time as ${source-date1:substring(8,13)} Then update attribute adds these flowfile attributes for all outgoing flowfiles from UpdateAttribute processor. In addition you can also perform same kind of operation without extracting as attributes using QueryRecord processor. Configure/Enable Record Reader/Writer controller services and use apache-calcite's Substring function to create source-date,source-time columns in the flowfile. ... View more

@Jacob Paul - The following article may help you understand what is going in your NiFi: https://community.hortonworks.com/articles/82308/understanding-how-nifis-content-repository-archivi.html - It is important to understand that NiFi cannot be configured to automatically delete Content repo claims that have not been archived. Claims that still have an active FlowFile on the canvas somewhere referencing them will remain in the content repository. - Since the content repo can fill to 100%, it is strongly recommended that this specific repository is isolated from all other NiFi repositories. - Thank you, Matt ... View more