Member since
01-07-2019
220
Posts
23
Kudos Received
30
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 11999 | 08-19-2021 05:45 AM | |
| 3152 | 08-04-2021 05:59 AM | |
| 1540 | 07-22-2021 08:09 AM | |
| 6026 | 07-22-2021 08:01 AM | |
| 5490 | 07-22-2021 07:32 AM |
07-23-2021
12:38 AM
1 Kudo
@DennisJaheruddi , Thanks for your reply. I a able to resolve this issue with your suggestion. Thanks!
... View more
07-22-2021
08:09 AM
I am not entirely sure what you are trying to achieve. If the set of data lives somewhere in a file you should be able to read it (e.g. list and fetchfile). If the data is generated by a script, is that just for testing? You could look at executescript or ExecuteGroovyScript, or perhaps just generate fake data directly with GenerateFlowFile (though the exact variety that you mention would be tough with the latter)
... View more
02-12-2021
01:44 AM
1 Kudo
Looks like the files are available here: https://repo.hortonworks.com/content/repositories/releases/org/apache/nifi/nifi-hive-nar/
... View more
02-04-2021
01:57 AM
Hello Dennis, thank you for the reply. It really helped a lot! Q1: That worked very well with the updateAttrbute processor. Q2: This also worked. I had the the settings of the csvWriter service (UpdateRecord) messed up. But it works fine. Q3: That is a bummer, I hoped it will would be a piece of cake to implement that. But i will look into one of the mentioned tools and figure it out. Q4: True that. The file is going to explode with data.
... View more
02-03-2021
08:22 AM
Apologies to disturb this old thread, but it seems people are still landing on this via search: The discussion here is outdated, especially the exclusions around gateway nodes. Please contact your Cloudera Representative for the latest terms an conditions.
... View more
02-02-2021
06:41 AM
1 Kudo
@Arash In your 4 node NiFi cluster, what value do you have set in the "nifi.remote.input.host" property in the nifi.properties file for each of the 4 nodes? It should be the FQDN for each node and not be the same value on all 4 nodes. Form the host where MiNiFi is running, can all 4 of those FQDNs be resolved and reachable over the network? If not, MiNiFI RPG is only going to be able to send successfully to one FQDN it can reach. When the RPG is started it reaches out to the URL configured in the RPG to obtain S2S details from the target host. That target host collects the host details for all currently connected nodes in the cluster and communicates that back to the client (MiNiFi). If all 4 nodes report the same configured FQDN in the "nifi.remote.input.host" property, then client only knows of one FQDN to which it can send FlowFiles over Site-To-Site (S2S). To improve redundancy in the RPG, you can provide a comma separated list of URLS in the RPG configuration so if any one node is down, the RPG can try fetch S2S details from the next host in the comma separated list. Hope this helps, Matt
... View more
02-01-2021
06:27 AM
9 out of 10 times this message is caused because you run the GetHDFS on multiple nodes. Both nodes see it, perhaps even try to pick it up, but clearly not both of these can delete it. In old versions of NiFi you can fix this by setting the GetHDFS to run only on the primary node. However, that will ofcourse burden the primary node more than it should. So in recent versions (and likely yours) you will find the ListHDFS and FetchHDFS processors (and similar sets for different data sources). The lightweight List processor can then run on the primary node, and loadbalance to all nodes which will then Fetch.
... View more
01-31-2021
01:50 PM
Unfortunately most sources are in Dutch but for good measure I will explain the most important data points: 1. Total weekly new covid infections come from the RIVM (dutch official body): https://www.rivm.nl/coronavirus-covid-19/archief-corona-updates 2. For the week of 26 jan, it is mentioned in an article by the RIVM that over one third of the current new infections is of the British Variant: https://www.rivm.nl/nieuws/Britse-variant-wint-terrein-in-Nederland 3. The same article mentions the rate was 8.6% in the period of 4-10 jan These are the most important points, and already show the trend. However, here are the additional sources: 4. In 'early december' the rate was about 1% according to this article on the largest news site of the country: https://www.nu.nl/coronavirus/6101869/wat-weten-we-nu-van-de-britse-coronavariant-ja-die-is-echt-besmettelijker.html 5. In a national press conference on 12 Jan, the minister of health indicated that in the past period the rate was between 2-5%. More frequent updates confirmed that there was steady growth in this period, so for the period between 9 and 29 december, there was likely growth from about 2% to about 5%. This could be off by one or two percentage points. 6. Though there was no clear source, several news sites recently referred to a current rate of 20%, this was likely observed between 13 and 19 jan.
... View more
08-13-2020
08:11 AM
@JonnyL I would highly recommend that you back up and create a small 3 node nifi cluster to test this feature. Putting 2 nifi on single node, does not satisfy the test cases you really want to be experimenting with. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven @ DFHZ
... View more
08-13-2020
06:51 AM
@stevenmatison Thanks .I used QueryRecord ,it helped to get count .
... View more