Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

IS THERE A PROCESSOR IN NIFI THAT CAN CREATE MANY DIRECTORY IN HDFS

Solved Go to solution

IS THERE A PROCESSOR IN NIFI THAT CAN CREATE MANY DIRECTORY IN HDFS

Contributor

Hello everyone, I've start working with nifi last week so sorry if my question is an obvious one.

I'm currentlty taking data in a Kafka Topic, those data are in Json format. I'm scaning every Json document and depending of the value of some fields, i would like to sort them in a directory with the name of the value.

for example if i have :

{"timestamp_da":"2016-07-06T15:04:33.332+00:00","type":"question","subject":"nifi"}

{"timestamp_da":"2016-07-06T15:04:33.332+00:00","type":"question","subject":"Kafka"}

==> I would like to extract the timestamp value and convert it into "20160706" and then create a directory with it.

After that every Json with the same timestamp will be route there. Is it possible ? or I'm I dreaming ?

I initially was doing this with spark but i think it's a waste to use spark just for this. If my question wasn't clear enought i can give more details.

Thanks in advance

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: IS THERE A PROCESSOR IN NIFI THAT CAN CREATE MANY DIRECTORY IN HDFS

Super Guru
2 REPLIES 2
Highlighted

Re: IS THERE A PROCESSOR IN NIFI THAT CAN CREATE MANY DIRECTORY IN HDFS

Super Guru
Highlighted

Re: IS THERE A PROCESSOR IN NIFI THAT CAN CREATE MANY DIRECTORY IN HDFS

Contributor

Thank you very much for your Help sir and have a good day or night depend on where you are :)

Don't have an account?
Coming from Hortonworks? Activate your account here