Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to use NIFI to connect to the CDP cluster in Azure

avatar
Contributor

Hi Experts,

I am trying to use the nifi processor getHDFS from my CDP cluster in Azure and then use a PutFile processor to download that file to my local filesystem.

My Nifi is a standalone server, separate from my CDP cluster.

 

Currently I am seeing this error in the nifi-app.log

2022-10-21 18:18:29,631 ERROR [Timer-Driven Process Thread-5] o.apache.nifi.processors.hadoop.GetHDFS GetHDFS[id=fab61e35-0183-1000-2eb0-4d511c15db51] Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to Unable to load custom token provider class.
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException: Unable to load custom token provider class.

 

Any help is much appreciated.


Thanks,

1 REPLY 1

avatar

@ryu CDP Public Cloud Azure or CDP Private Cloud on Azure VMs?
To link a NiFi outside of the cluster, you will need to provide that nifi with the files from the CDP Cluster. For example core-site.xml, hdfs-site.xml.   Outside of that configuration, you will need to do some networking to allow access between systems, and then last but not least deal with access/auth and kerberos.   

If you are already working on some of these areas, be sure to include screen shots of processors, controller services, configs, etc.