Support Questions
Find answers, ask questions, and share your expertise

NiFi PutHDFS not working from another cluster

Explorer

Hi,

I have 2 clusters, 1 HDP 2.6.5 (with HA enabled) and another HDF 3.2. I am trying to tail a file from NiFi server and put it in HDFSusing PutHDFS which is not working. I have hdfs-site.xml & core-site.xml in place on NiFi and mentioned the same path in the processor. Below are the flow & properties

I noticed below error in nifi logs:

2018-11-19 17:08:33,547 INFO [NiFi Web Server-63525] o.a.n.c.s.StandardProcessScheduler Starting PutHDFS[id=cfe24ec0-8402-157d-ffff-ffff9f510fd7]
2018-11-19 17:08:34,070 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@6c4b3fee // Another save pending = false
2018-11-19 17:08:34,444 WARN [Timer-Driven Process Thread-11] o.a.h.h.shortcircuit.DomainSocketFactory The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2018-11-19 17:08:34,462 INFO [Timer-Driven Process Thread-11] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHDFS[id=cfe24ec0-8402-157d-ffff-ffff9f510fd7] to run with 1 threads

Do I need to check on any specific port whether it's able to communicate from HDF to HDP for HDFS?

or any Hadoop library that needs to be looked in nifi?

Is there any particular reason it's unable to load Hadoop library in NiFi? Can anyone help here?


puthdfs-properties.pngputhdfs-flow.png
0 REPLIES 0