Member since
04-05-2016
139
Posts
143
Kudos Received
16
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
31760 | 02-14-2019 02:53 PM | |
2449 | 01-04-2019 08:39 PM | |
10547 | 11-05-2018 03:38 PM | |
4877 | 09-27-2018 04:21 PM | |
2651 | 07-05-2018 02:56 PM |
02-01-2020
05:40 AM
I have uploaded a NiFi example template on https://github.com/maxbback/nifi-xml
... View more
12-17-2019
07:30 PM
Hi @alim Is there any way to work with CaptureChangeMySQL and EnforceOrder in cluster environment for better performance?
... View more
12-16-2019
06:32 PM
Hi Alim, I just started to repeat our work and found the following error. My testing environment is : Ubuntu 16.04; MySQL and NiFi are installed in the system, I could access to MySQL via the terminal in a remote Windows PC. Does anyone know how to address the issue? Thanks. Kind regards, Raymond
... View more
10-16-2019
01:57 PM
The issue seems to be due to the error documented in the Registry Admin Guide (https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#system-requirements) if you don't have a JDK installed: When running Registry with only a JRE you may encounter the following error as Flyway (database migration tool) attempts to utilize a resource from the JDK: java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1.3 (DropBucketItemNameUniqueness) Try installing the JDK , deleting the `database` directory that was created prior and then start the Registry.
... View more
10-03-2019
08:02 AM
You can change the variables with REST API, but I wouldn't suggest this. 1. In this case within NIFI Cluster version numbers could be different between Nodes 2. After variable update a restart is required for controller and processor too (it doesn't matter if you changing it with REST API, or with other methods), you can check the process, when you manually changing it. It's better to create a flowfile with attributes, or write-back data to sources (if it's a database), and compare it from there if possible. If you have an another solution, please share it :).
... View more
10-24-2018
03:15 PM
2 Kudos
Objective
In a separate HCC article, I detailed how to offload nodes using the NiFi UI. This article covers how to perform the same operation using the NiFi Toolkit CLI.
Note: This tutorial assumes you have already setup a running NiFi cluster. The examples included are for a 2 node NiFi cluster. Environment
This tutorial was tested using the following environment and components:
Mac OS X 10.11.6 Apache NiFi 1.8.0 Apace NiFi Toolkit 1.8.0 Offload Nodes via CLI Tutorial Queue FlowFiles
First, generate some queued flowfiles:
The queued flowfiles are distributed between both nodes in the cluster:
Disconnect Node
From a terminal window, navigate to the directory where the NiFi Toolkit was installed. From the /nifi-toolkit-1.8.0/bin directory, run:
./cli.sh
Enter the command
nifi get-nodes to retrieve the node IDs:
#> nifi get-nodes
# Node ID Node Address API Port Node Status
- ------------------------------------ ------------ -------- -----------
0 dfa62636-479b-4456-aa16-3bfc19f35cb5 localhost 9443 CONNECTED
1 20a68479-e8dc-46c9-8612-872200e2fdab localhost 9444 CONNECTED
Note: If you see the error "ERROR: Error executing command 'get-nodes' : Error retrieving node status: Unable to view the controller. Contact the system administrator." You need to add "CN=localhost, OU=NIFI" to the "access the controller" view policy.
Use the command
nifi disconnect-node to disconnect the node on Port 9444:
#> nifi disconnect-node -nnid 20a68479-e8dc-46c9-8612-872200e2fdab
Node ID: 20a68479-e8dc-46c9-8612-872200e2fdab
Node Address: localhost
API Port: 9444
Node Status:DISCONNECTED~
The node is disconnected:
Note: If you see the error "ERROR: Error executing command 'disconnect-node' : Error disconnecting node: Unable to modify the controller. Contact the system administrator." You need to add "CN=localhost, OU=NIFI" to both the "access the controller" modify policy.
Offload Node
Use the command
nifi offload-node to offload the flowfiles on port 9444:
#> nifi offload-node -nnid 20a68479-e8dc-46c9-8612-872200e2fdab
Node ID: 20a68479-e8dc-46c9-8612-872200e2fdab
Node Address: localhost
API Port: 9444
Node Status:OFFLOADING~
Repeating the command returns the current status of the node:
#> nifi offload-node -nnid 20a68479-e8dc-46c9-8612-872200e2fdab
Node ID: 20a68479-e8dc-46c9-8612-872200e2fdab
Node Address: localhost
API Port: 9444
Node Status:OFFLOADED~
In the UI, it is confirmed that all the flowfiles have been offloaded to the active node:
Delete & Decommission Node
An offloaded node can be either connected back to the cluster (
nifi connect-node ) or deleted. If the goal is to to decommission the node, use the
nifi delete-node command:
#> nifi delete-node -nnid 20a68479-e8dc-46c9-8612-872200e2fdab
OK
Then after the node is deleted, stop/remove the NiFi service on the host.
... View more
Labels:
08-31-2018
03:09 PM
2 Kudos
@bangalore SubbU You might be interested in the available documentation on using custom properties (https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Using_Custom_Properties) via:
Variables window in the NiFi UI (https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables_Window) nifi.properties as suggested earlier (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#custom_properties) Note that NiFi automatically picks up new or modified variables created in the UI (no restart required).
... View more
08-09-2018
04:01 PM
2 Kudos
Hi @Bharat
Mupparthi
Make sure you are looking for Controller Services at the proper level. There are “Reporting Task Controller Services” which are defined via the Global menu (“hamburger” menu) at the top right of the UI: There are Controller services defined at the Root Process group level: There are Controller Services defined at the process group level (in your instance, “Membership Processing” process group or any nested process groups within it). To check, make sure the process group is selected and then choose Configure from the Operate Palette (gear icon) or right-click and select “Configure”:
... View more
07-06-2018
01:47 PM
@Mateusz Koszutowski Glad your issue was resolved. To help others who run into the same problem, could you please select the Accept link for the answer that I provided? Thanks!
... View more
06-21-2018
03:51 PM
@Sami Ahmad Users who are not authorized for specific NiFi components (process groups, processors, input/output ports, etc...) will only see "ghost" processors. Those ghost processors will not contain any information on the face of them the reveals their type or configuration. - - Unless authorized, user also will not be able to list queues, perform provenance queries, etc on FlowFiles that traverse these components. - There are good reasons why NiFi does not hide these ghost processor completely: 1. All components operate within the same resource constraints of a single JVM. It is important that all user can identify areas of any flow that may be backlogging or have large queues, as these things may impact their own dataflows. While the can not see the actual data or processor details, they can observe FlowFile stat info as you can see in above screenshot. 2. If User1 built a dataflows only user 1 could see, User2 may end up building a dataflow directly on top of it. While functionally NiFi does not care about overlapping components, user3 who logs in and has access to both user1 and user2 flows would. It would make it very hard for user 3 to follow the canvas in such a case. - Thanks, Matt
... View more