Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

IS THERE A PROCESSOR IN NIFI THAT CAN CREATE MANY DIRECTORY IN HDFS

Contributor

Hello everyone, I've start working with nifi last week so sorry if my question is an obvious one.

I'm currentlty taking data in a Kafka Topic, those data are in Json format. I'm scaning every Json document and depending of the value of some fields, i would like to sort them in a directory with the name of the value.

for example if i have :

{"timestamp_da":"2016-07-06T15:04:33.332+00:00","type":"question","subject":"nifi"}

{"timestamp_da":"2016-07-06T15:04:33.332+00:00","type":"question","subject":"Kafka"}

==> I would like to extract the timestamp value and convert it into "20160706" and then create a directory with it.

After that every Json with the same timestamp will be route there. Is it possible ? or I'm I dreaming ?

I initially was doing this with spark but i think it's a waste to use spark just for this. If my question wasn't clear enought i can give more details.

Thanks in advance

1 ACCEPTED SOLUTION
2 REPLIES 2

Contributor

Thank you very much for your Help sir and have a good day or night depend on where you are 🙂

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.