Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to aggregate csv or json data in Nifi?

Solved Go to solution
Highlighted

How to aggregate csv or json data in Nifi?

Hi,

How can i aggregate my csv data in nifi? Can you please give me options.

I have csv coming every hour.

For example my csv data is:

Name, Subject

stud1, math

stud2, english

stud3, math

stud4, math

stud5, english

stud6, science

My needed output is:

math, 3

english, 2

science, 1

Can i make this without using any execute scripts?

Thank you.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to aggregate csv or json data in Nifi?

Hi,

I think you can do that with the QueryRecord processor. Add a new property, whatever the name and the content may be something like "select Subject, count(Name) as nb from flowfile group by Subject".

Set correctly the schemas for the reader and the writer and you will find the result of the query as a new relationship of your processor. (A new relationship with the name of your property will appear)

View solution in original post

2 REPLIES 2
Highlighted

Re: How to aggregate csv or json data in Nifi?

Hi,

I think you can do that with the QueryRecord processor. Add a new property, whatever the name and the content may be something like "select Subject, count(Name) as nb from flowfile group by Subject".

Set correctly the schemas for the reader and the writer and you will find the result of the query as a new relationship of your processor. (A new relationship with the name of your property will appear)

View solution in original post

Re: How to aggregate csv or json data in Nifi?

It worked! thank you so much.
can this query record make aggregation in real time manner?
like ill accept data every second then go save it in hbase, but i always should update the data in hbase.

Don't have an account?
Coming from Hortonworks? Activate your account here