Member since
07-30-2019
944
Posts
197
Kudos Received
91
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1503 | 10-05-2021 01:53 PM | |
| 16316 | 09-23-2019 06:03 AM | |
| 6814 | 05-04-2019 08:42 PM | |
| 1502 | 06-11-2018 12:45 PM | |
| 12617 | 06-04-2018 01:11 PM |
09-29-2017
07:26 PM
@Benjamin Hopp Did you copy over the core-site.xml and hdfs-site.xml files from the HDP cluster after enabling kerberos?
... View more
09-28-2017
02:57 PM
1 Kudo
@John Carter Did you get chance to look at my flow? Also, will you post a snapshot of your DistributedMapCacheClientService configuration? Your flow should work like you expect.
... View more
09-28-2017
02:30 PM
1 Kudo
@Anagha Khanolkar There is no preferred OS for HDF, use the one that you have the most knowledge. I will say that LINUX is the most tested OS used. There is documentation that cover OS specific tuning and best practices. The only required software is JAVA 8. The minimum system resources will be driven by the volume of data, size of files and how much processing will be done on the data. Here is a link to documentation that provides a good starting point and hardware sizing recommendations : Planning your deployment
... View more
09-28-2017
01:39 PM
@sally sally To process only one file per day, just configure your processor to run once a day using a Scheduling Strategy of CRON driven Here is an example of a processor scheduled to run everyday at 1PM
... View more
09-26-2017
07:11 PM
1 Kudo
@John Carter I am using a similar setup for my testing and it works just like you think it should. For your test, make sure and clear any residual file counts, by stopping and starting the DistributedMapCacheServer and the DistributedMapCacheClientService controller services. Then restart the test. UpdateAttribute Notify Wait Required controller services
... View more
09-25-2017
03:01 PM
@David Williamson Yes, the ListFile writes the same attributes. But, make sure the user running NiFi has the permissions to create directories in HDFS.
... View more
09-25-2017
02:42 PM
1 Kudo
@David Williamson
If you are using GetFile and PutFile processors to copy individual files, they can also be used for recursive folder structure. One of the attributes written by GetFile is the folder path of the file, so you can use PutFile to create the folder structure in the new location.
... View more
09-24-2017
08:04 PM
@John Carter Are you not seeing the behavior that you expect? You don't need to stop and start the flow, that has no effect on the count of the flow files the Wait processor is waiting for. Would you be willing to provide a snapshot of your flow?
... View more
09-22-2017
07:11 PM
@Albert Mc Gregor Have you tried pulling the same file from the command line of the NiFi system using wget?
... View more