Nifi putHDFS processor not working although putFIle does

[#1]

I have a small workflow with a listenProcesser that listens to the webhook and that is connected to a put Hdfs processor.

Now the problem is that the mesages keep getting queued rather than writing to HDFS. Where do I check the logs and how do I fix it?

I have tried connecting it to putFile and that works. Not sure why putHDFS is not working. In putHDFS, I am writing it in /user/centos directory and so am through putFile, which means user and group is same for the directories used in both putFile and putHDFS.

Attached the snapshot:

[#2 · Accepted solution]

@Simran Kaur

I see from your screenshot that your putHDFS processor is producing bulletins (Red square in upper right corner). If you float your cursor over this red square you will see the bulletin displayed. You can also look for this same error in the nifi-app.log. In many cases the error will be followed by a full stack trace in the nifi-app.log. That stack trace and the error log line may explain what you issue is here.

If this does not help, please share your putHDFS processor configuration.

Thanks,

Matt

[#3]

Thanks. Where exactly can I find nifi-app.log?Turns out I did not add core-site xml and hdfs-site xml in the configuration . I have and seems to work. BUt have a quick question. I have a file each for a JSON object streamed. That sounds like a bad thing to me to have too many small files. Is there a work around for this?

[#4]

@Simran Kaur

I had a feeling your issue was related to a missing config. Glad to hear you got it working. If this answer addressed your original question, please mark it as accepted.

As far as your other question goes, I see you already started a new question (https://community.hortonworks.com/questions/105720/nifi-stream-using-listenhttp-processor-creates-to.... That is the correct approach in this forum, we want to avoid asking unrelated questions in the same post.

I will have a look at that post as well.

Thank you,

Matt