question Re: How to Convert XML file into CSV file using NIFI in Support Questions

How to Convert XML file into CSV file using NIFI

nitindamle_123 — Thu, 14 Feb 2019 13:13:56 GMT

i have XML files about 1-2 GB i want it to convert into CSV format using apache nifi.

so that i can process CSV for joining the two csv.

Re: How to Convert XML file into CSV file using NIFI

alim — Thu, 14 Feb 2019 22:53:02 GMT

You can use the ConvertRecord processor (with XMLReader/CSVRecordSetWriter controller services). I have an article here that shows CSV to XML (with CSVReader/XMLRecordSetWriter controller services), so you just need to do the inverse of that record transformation:

https://community.hortonworks.com/content/kbentry/199310/xml-record-writer-in-apache-nifi-170.html

-Andrew

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Fri, 15 Feb 2019 13:57:11 GMT

@Andrew Lim

can you please upload the template here i have tried to upload template given in link but it throwing error

Re: How to Convert XML file into CSV file using NIFI

alim — Fri, 15 Feb 2019 23:15:08 GMT

@Nitin Damle

convert-cvs-to-xml.xml

If you are still having problems uploading the template, can you confirm what version of NiFi you are on? I just did a test in NiFi 1.8.0 and the upload was successful.

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Sun, 18 Aug 2019 00:06:23 GMT

@Andrew Lim After importing it giving me this error

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Wed, 20 Feb 2019 16:11:04 GMT

FYI

https://community.hortonworks.com/questions/226390/nifi-18-error-updateattribute-is-not-known-to-this.html

Re: How to Convert XML file into CSV file using NIFI

alim — Thu, 21 Feb 2019 00:13:10 GMT

Hi @Nitin Damle

It would be helpful if you selected "Reply" to each of my comments instead of adding new Answers, which makes the readability of this question more difficult. Understandably, the "Reply" link is very small and hard to see.

Are you modifying the template prior to uploading it into NiFi? That was the issue in the other HCC article you referenced (https://community.hortonworks.com/questions/226390/nifi-18-error-updateattribute-is-not-known-to-this.html). Because the template was malformed, it gave that error.

Re: How to Convert XML file into CSV file using NIFI

max_boreback — Sat, 23 Feb 2019 18:53:36 GMT

You can use this NIFI Groovy processor that converts XML to CSV or AVRO

https://github.com/maxbback/nifi-xml

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Mon, 25 Feb 2019 14:48:30 GMT

@Andrew Lim can you upload the template for XML to CSV with sample data .

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Mon, 25 Feb 2019 14:54:09 GMT

@Max Börebäck can you upload the template here

Re: How to Convert XML file into CSV file using NIFI

alim — Tue, 26 Feb 2019 04:40:56 GMT

Hi @Nitin Damle

Here is the XML to CSV template: convert-xml-to-csv.xml

Here is the sample XML file to use: users.xml

If you find these helpful, please "Accept" my original answer from February 14th as this template is the implementation of what I suggested to do.

Re: How to Convert XML file into CSV file using NIFI

alim — Tue, 26 Feb 2019 04:53:36 GMT

One thing, I wanted to note. For the XMLReader controller service (https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/index.html), I needed to set the Expect Records as Array property to "true".

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Wed, 27 Feb 2019 13:57:03 GMT

Thanks @Andrew Lim

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Wed, 27 Feb 2019 18:21:57 GMT

@Andrew Lim , i am getting header only not data

cm_samples.xml

schema.txt

i attached my schema and xml pls let me know where i am going wrong

Re: How to Convert XML file into CSV file using NIFI

alim — Thu, 28 Feb 2019 05:16:45 GMT

Hi @Nitin Damle

I'm assuming you got the flow working with the sample data and template I provided.

Can you provide an example of the desired csv output you are hoping to achieve from the cm_samples.xml file you provided? That would help me understand your end goal for the flow.

Thanks!

Re: How to Convert XML file into CSV file using NIFI

nitindamle_123 — Thu, 28 Feb 2019 17:21:19 GMT

xmlsampleop.png

@Andrew Lim you can refercm_sample.txt

this image and file

Re: How to Convert XML file into CSV file using NIFI

alim — Fri, 01 Mar 2019 05:14:29 GMT

Thanks for the information Nitin.

It seems like it would make sense to "clean up" your incoming XML using the TransformXML processor (https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.TransformXml/) prior to ConvertRecord since there is data there like the header info that you don't care about for the CSV file. Additionally, could change the tag from `p` to its `name` attribute.

I haven't used the TransformXML processor before (or worked with XSLT files before). Take a look yourself and see if you think it could be a solution for you. I will try it out myself if I get the opportunity.

Re: How to Convert XML file into CSV file using NIFI

debananda_sahoo — Fri, 05 Apr 2019 16:13:10 GMT

Hi Andrew,

I am not able to convert xml to csv as suggested by you to do the inverse of this. Could you please provide the nifi xml file so that I can upload the template and check.

Re: How to Convert XML file into CSV file using NIFI

alim — Tue, 09 Apr 2019 23:16:16 GMT

convert-xml-to-csv.xml

Re: How to Convert XML file into CSV file using NIFI

debananda_sahoo — Tue, 23 Apr 2019 18:20:02 GMT

Hi Andrew , this works for single xml but I have nested xml. How to convert nested xml to multiple csvs ?