- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to use NIFI to connect to the CDP cluster in Azure
Created ‎10-21-2022 04:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Experts,
I am trying to use the nifi processor getHDFS from my CDP cluster in Azure and then use a PutFile processor to download that file to my local filesystem.
My Nifi is a standalone server, separate from my CDP cluster.
Currently I am seeing this error in the nifi-app.log
2022-10-21 18:18:29,631 ERROR [Timer-Driven Process Thread-5] o.apache.nifi.processors.hadoop.GetHDFS GetHDFS[id=fab61e35-0183-1000-2eb0-4d511c15db51] Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to Unable to load custom token provider class.
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException: Unable to load custom token provider class.
Any help is much appreciated.
Thanks,
Created ‎10-25-2022 12:53 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@ryu CDP Public Cloud Azure or CDP Private Cloud on Azure VMs?
To link a NiFi outside of the cluster, you will need to provide that nifi with the files from the CDP Cluster. For example core-site.xml, hdfs-site.xml. Outside of that configuration, you will need to do some networking to allow access between systems, and then last but not least deal with access/auth and kerberos.
If you are already working on some of these areas, be sure to include screen shots of processors, controller services, configs, etc.
