question Re: How to aggregate csv or json data in Nifi? in Archives of Support Questions (Read Only)

How to aggregate csv or json data in Nifi?

regie_canada — Wed, 16 Aug 2017 16:53:26 GMT

Hi,

How can i aggregate my csv data in nifi? Can you please give me options.

I have csv coming every hour.

For example my csv data is:

Name, Subject

stud1, math

stud2, english

stud3, math

stud4, math

stud5, english

stud6, science

My needed output is:

math, 3

english, 2

science, 1

Can i make this without using any execute scripts?

Thank you.

Re: How to aggregate csv or json data in Nifi?

quentin_toulou — Wed, 16 Aug 2017 20:59:41 GMT

Hi,

I think you can do that with the QueryRecord processor. Add a new property, whatever the name and the content may be something like "select Subject, count(Name) as nb from flowfile group by Subject".

Set correctly the schemas for the reader and the writer and you will find the result of the query as a new relationship of your processor. (A new relationship with the name of your property will appear)

Re: How to aggregate csv or json data in Nifi?

regie_canada — Thu, 17 Aug 2017 10:09:12 GMT

It worked! thank you so much.
can this query record make aggregation in real time manner?
like ill accept data every second then go save it in hbase, but i always should update the data in hbase.