Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Best practice for updating Nifi's 'Hadoop Configuration Resources'

Solved Go to solution

Best practice for updating Nifi's 'Hadoop Configuration Resources'

New Contributor

Hey everyone,

I have a HDF cluster storing data into a HDP cluster using Nifi. To use the putHDFS processor I needed to store copies of the core-site.xml and hdfs-site.xml in each of my HDF worker nodes. What is the suggested method for automating the updating of those files? For example, if I were to change the configurations of either of those files would I need to manually copy them over to the worker nodes again? Or is there a better, more automated, solution?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Best practice for updating Nifi's 'Hadoop Configuration Resources'

You can use ListSFTP -> FetchSFTP -> PutFile flow on NiFi to grab the files from wherever you're storing them for the master copy of the configs. This will have NiFi keep itself up to date and you can have your hadoop resources point at the location where you do the PutFile.

View solution in original post

2 REPLIES 2
Highlighted

Re: Best practice for updating Nifi's 'Hadoop Configuration Resources'

You can use ListSFTP -> FetchSFTP -> PutFile flow on NiFi to grab the files from wherever you're storing them for the master copy of the configs. This will have NiFi keep itself up to date and you can have your hadoop resources point at the location where you do the PutFile.

View solution in original post

Highlighted

Re: Best practice for updating Nifi's 'Hadoop Configuration Resources'

New Contributor

Clever, thank you for the suggestion!

Don't have an account?
Coming from Hortonworks? Activate your account here