question Re: XSL to CSV using NiFi in Archives of Support Questions (Read Only)

XSL to CSV using NiFi

regie_canada — Thu, 15 Dec 2016 13:41:57 GMT

Hi guys,

Is there any process where i can make my XSL File to CSV File?

Thanks.

Re: XSL to CSV using NiFi

sunile_manjee — Thu, 15 Dec 2016 13:46:03 GMT

Convert xsl to json using this

https://community.hortonworks.com/articles/29474/nifi-converting-xml-to-json.html

and then convert json to csv using this

https://github.com/sunileman/NiFi-Json2Csv

Re: XSL to CSV using NiFi

knarayanan — Thu, 15 Dec 2016 14:00:24 GMT

@regie canada you want to convert XSL to CSV? just wanting to confirm you didn't mean XLS.

Re: XSL to CSV using NiFi

regie_canada — Thu, 15 Dec 2016 14:11:30 GMT

@Karthik Narayanan

Hi sir, yes it is XSL. We made a program that will turn our XML to XSL.

This is the current sample output:

1000002746|CRl Campbell|905-899-3495|905-899-3495|lrobert@crlcampbell.com|11675 Burnaby Rd ||Wainfleet|province-ONTA|L0S1V0|Active|HostedIPV_1000003883_SA

We use PIPE for delimiter.

and now I need to make it csv so i can convert it to json.

Thank.

Re: XSL to CSV using NiFi

knarayanan — Thu, 15 Dec 2016 14:16:54 GMT

i guess based on your comment, use replacetext processor to replace all occurrence of | with ,.

Re: XSL to CSV using NiFi

knarayanan — Thu, 15 Dec 2016 14:36:17 GMT

you may need header information to convert this to json, technically even the xsl output you have is CSV, just delimited with pipe. I think you can directly use the executescript processor, to call a python script, and go from xsl to json..

getfile (read the file with xsl data ) -> splittext (split the data into lines )-->execute script (with script below to convert to json) --> merge content (merge contents based on fragment.identifier attribute of split text) --> put file (gives you json files )

--- example scrip for converting to json..

import json
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback


class PyStreamCallback(StreamCallback):
  def __init__(self):
        pass
  def process(self, inputStream, outputStream):
    header=["column1","column2","column3"...] # the header for the xsl, this will become the name for json nodes
    text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
    output={}
    for column in text.split("|"):
	index=0 # counter to keep track of column, so we can assing a name to the value.
	output[header(index)]=column
	outputStream.write(bytearray(json.dumps(output, indent=4).encode('utf-8')))	
flowFile = session.get()
if (flowFile != None):
  flowFile = session.write(flowFile,PyStreamCallback())
  flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename').split('.')[0]+'_translated.json')
  session.transfer(flowFile, REL_SUCCESS)

Re: XSL to CSV using NiFi

regie_canada — Thu, 15 Dec 2016 14:56:39 GMT

@Karthik Narayanan

Thank you so much for answering.

Do I need to install anything to run a python script?

Re: XSL to CSV using NiFi

knarayanan — Thu, 15 Dec 2016 23:42:17 GMT

@regie canada no you dont have to, it should work out of the box.

Re: XSL to CSV using NiFi

max_boreback — Sat, 23 Feb 2019 18:58:53 GMT

You can also try this NIFI groovy processor that converts XML to CSV

https://github.com/maxbback/nifi-xml