Support Questions

Find answers, ask questions, and share your expertise

NiFi: Convert Avro to XML and replace first two lines of XML

avatar
Contributor
  • I have a requirement where I need to execute a SQL query to a DB and transform the returned avro file to XML.
    • First I need to replace a few of the records in the file. So I used a split record processor with Avro reader and XML writer to split the records
    • Two replace text processors to replace the required texts in each record
    • Then merge the files into a single file

      Once the file is merged, I need to replace the XML declaration and root recrod of XML with custom values. 
  • The resultant file is about 2GB in size. Can I use sed 1,2d command in execute stream command processor to remove the files and replace text processor to prepend the required text. 
  • If I use this method, will the large file occupy system memory and affect performance of other pipelines?

Flow is attached

Jagapriyan_0-1679920323196.png

 

0 REPLIES 0