Member since
07-30-2019
920
Posts
195
Kudos Received
91
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1036 | 10-05-2021 01:53 PM | |
14461 | 09-23-2019 06:03 AM | |
5599 | 05-04-2019 08:42 PM | |
1142 | 06-11-2018 12:45 PM | |
10436 | 06-04-2018 01:11 PM |
07-18-2019
06:21 PM
@luan ha If you want to be able to access the NiFi UI via all of those URLs, then you will have to clear the nifi.web.http.host property.
... View more
05-04-2019
08:42 PM
1 Kudo
@Mario Tigua You can use the REST api to clear state on the processor. You can get the curl command to do this using the Developer tools available in the Chrome browser. Then pass that command, in a script, to a ExecuteStreamCommand processor that is triggered after the files are fetched and clear the state of the ListSFTP processor. For example: This simple flow shows how you could list the files, fetch them and then after they're fetched, use the results flow files to trigger the state being cleared in the ListSFTP processor: The configuration of the ExecuteStreamCommand processor looks like: The contents of the curl-script.sh file, of course you would fill in the appropriate host and port for your environment as well as the uuid of the processor. #!/bin/bash
# curl command to stop processor
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c' -X PUT -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --data-binary '{"revision":{"clientId":"8440f58c-016a-1000-f56a-4b8ca84fb669","version":30},"component":{"id":"84415a50-016a-1000-0000-00001257b02c","state":"STOPPED"}}' --compressed
# curl command to clear state
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c/state/clear-requests' -X POST -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Content-Length: 0' --compressed The curl command can be obtained by Then using the Developer tool, get the curl command when you manually clear the state of the processor This example was done with an insecure instance of NiFi, if NiFi is secure, you'll have to get a token to run the curl commands.
... View more
12-20-2018
10:34 PM
2 Kudos
Summary: This article provides the steps for using ConsumeJMS and
PublishJMS processors with Weblogic Topics and Queues. Prerequisites: In this article a Weblogic 12c server was used with HDF-3.3.0.
Also, for this article the JMS Server, Connection Factory, Queue and Topic have
already been added to the Weblogic server. These blogs were used as references
for this article, Install Weblogic 12.1.3 server, Basic JMS Demo using WebLogic Queue. Much of the credit for the information here
goes to @emattos and @Jay Kumar SenSharma, both provided a great deal of assistance in
understanding the Weblogic server and sample java programs for testing. Also, for this you will need the wlfullclient.jar, this can be
created by following this guide: Using the WebLogic JarBuilder Tool Note: The JndiJmsConnectionFactoryProvider
controller service was introduced in HDF-3.3.0. So, this controller service
isn’t available in any earlier versions of HDF. NiFi Setup: In this article we'll
use a simple flow to demonstrate the functionality of the processors. This is not meant to be an exhaustive article
on the PublishJMS and ConsumeJMS processors.
Just the information required to get the processors working. The configuration of
the controller service is based on the configuration of a very simple queue and
topic in a Weblogic 12c server. A full explanation of the creation of the JMS
server, Connection Factory, Queue and Topic can be found in the blogs listed in
the beginning of the article. For
reference, here is the configuration of the Weblogic server used for this
article. JMS Server JMS Module The Connection Factory,
Queue and Topic. Based on the above information, this is the configuration of the
JndiJmsConnectionFactoryProvider controller service: The Initial Naming Factory Class property is used to identify the factory that actually creates the
Context. To use WebLogic JNDI, you use weblogic.jndi.WLInitialContextFactory. The Naming Provider URL, specifies the URL of the WebLogic Server that provides the
name serviceused, for this
tutorial it is just the default value. The controller service
uses the Java Naming and Directory Interface (JNDI) to lookup a JMS connection
factory. So, the JNDI name of the
Connection Factory, MCF1, is used in the Connection
Factory Name property of the controller service. The Naming Factory
Libraries property just identifies the jar with the Initial Naming Factory Class. Now that the controller service is configured,
the processors can be configured. The PublishJMS processor
flow. The flow simply tails the
local nifi-app.log file, splits the flow files into single log lines and then
writes the log lines to queue Queue1 and topic Topic1. The configuration of the
PublishJMS processor for Queue1 The Destination Name property can be seen in the Weblogic console. The configuration of the
PublishJMS processor for Topic1 is the same except for the Destination Type and Destination Name properties. Here is the simple flow demonstrating the ConsumeJMS processors
for Queue1 and Topic1. The configuration of the ConsumeJMS processors are the same for the Connection Factory Service, Destination
Name and Destination Type
properties.
... View more
Labels:
11-01-2018
02:40 PM
@nifi_is_awesome FYI, It is better to start a new question, referencing this question, rather than adding a new question via an answer here.
... View more
10-03-2018
01:07 PM
Sharath, Another way to resolve the issue is to make sure the symbolic link to the shared object points to the 64-bit library in the /usr/lib directory. I added the below symbolic link to my system running NiFi and the error was resolved: ln -s /opt/mqm/java/lib64/libmqjbnd.so /ust/lib/libmqjbnd.so Here is a snapshot of my JMSConnectionFactoryProvider controller service configuration
... View more
10-01-2018
06:44 PM
@leah wang The NiFi java process is actually two processes. The actual NiFi java process and a bootstrap java process. The bootstrap process has a very small memory footprint, 12MB to 24MB, that looks for the NiFi java process. If it doesn't find that process it will try to start it. The NiFi java process will reserve the memory allocated, in this case 8GB, and hold it. It doesn't actually use the memory, but reserves it so it is available as NiFi needs it. There won't be any garbage clean up until the memory usage is around 80% utilization.
... View more
09-25-2018
06:07 PM
1 Kudo
@leah wang How are you measuring the memory being utilized by NiFi? FYI, the content of a flow file is not stored in memory, only the properties and attributes associated with the physical file in the content repository are held in the JVM heap memory. The only time the content is in memory is when a processor that deals with the content of a file is in the flow.
... View more
06-11-2018
12:45 PM
1 Kudo
@dhieru singh You are correct. When running NiFi in an Ambari managed cluster, you need to restart NiFi so the changes made in the Ambari UI are pushed out the the logback.xml file. If you make the changes directly to the logback,xml, then NiFi will pick up those changes the next time it does the refresh. But, the next time you restart NiFi via Ambari those changes would be over written if you didn't also make the changes via the Ambari UI.
... View more
06-07-2018
07:04 PM
@Jason Sphar What about the windows firewall? Try turning it off completely and see if it works then.
... View more
06-06-2018
11:11 PM
@dhieru singh You should use a demarcator that keeps your messages the way you want them.
... View more