Member since
09-04-2019
62
Posts
17
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
761 | 10-30-2023 06:50 AM | |
10161 | 02-27-2023 09:25 AM | |
1807 | 07-07-2022 09:17 AM | |
1713 | 01-26-2022 06:25 AM | |
2479 | 01-25-2022 06:19 AM |
01-24-2022
11:56 AM
ExecuteSQL will use Database Connection Pooling Service controller service. From there you configure a JDBC connection string. The connection string will depend on the specific jar you will use so in this case Presto Here is reference on JDBC connection for Presto and that is an external database function and not so much NiFi issue: https://prestodb.io/docs/current/installation/jdbc.html
... View more
01-24-2022
11:52 AM
1 Kudo
First confirm if the NiFi JVM is running search your logs for the word "UI is available" If it is then I would look at your systems firewall windows defender? If you do not see that log entry I would focus on why NiFi JVM is not coming up
... View more
01-24-2022
11:44 AM
Old post but for awareness: As reported here: UUID Logical type is now included under Apache Parquet 1.12 https://issues.apache.org/jira/browse/PARQUET-1827 Apache NiFi will use apache Parquet 1.12 starting from Apache NiFi 1.14
... View more
01-21-2022
09:46 AM
1 Kudo
Without knowing how the FTP directory is populated, you might have an issue with the ListFTP state and might consider changing its its Listing Strategy, However assuming that is no issue and you want to only perform an action based on any file with a naming structure like this: xxxx-xxxx-xxxx-xxxx-20220121110000-xxxx-xxxx.csv and only if it is older than two hours from "now" I would run the output of your ListFTP to an UpdateAttribute and add these 2 properties: property name fileTime value ${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()} property name timeNow value ${now():toNumber():minus(7200000)} Then route that to a RouteOnAttribute and add a new property: property name value 2 hours old ${fileTime:le(${timeNow})} Then you can drag that connection to follow on processing and the unmatched connection to other processing or terminate it. Explanation: filelName ${filename:getDelimitedField(5,'-'):trim():toDate('yyyyMMddHHmmss'):toNumber()} This grabs the time out of your filename and converts it to a Date object so it can be converted to its epoch representation timeNow ${now():toNumber():minus(7200000)} Sets a value 2 hours ago from current time ${fileTime:le(${timeNow})} If attribute fileName is less than or equal to timeNow it means that it is 2 hours old.
... View more
01-20-2022
12:05 PM
1 Kudo
I understand the issue EvaluateXPath does not take in an attribute syntax and fails at validation since it is validating for a valid Xpath path. I wonder if there is a way to use QueryRecord instead.
... View more
01-20-2022
10:58 AM
Have you considered looking at the ListFtp property: Minimum File Age and set it to 2 hours
... View more
01-20-2022
09:43 AM
Deleting that state directory should not be a normal maintenance function. What you initially described is a very odd case. The fact that your node went down 003 and if it was the primary node or the coordinator node, internally there would have been a election to nominate a new cluster member node to perform those functions. In your case node 003 is a member of the cluster but it is not connected. Why it is not connected could be the cause of n reasons typically node is down or it was manually disconnected. When you see that message how many member nodes do you have? I expect the UI to show 2/3 because node 3 is not connected. The solution is to connect it by fixing the issue of why node is down or connect it through the UI
... View more
01-19-2022
10:03 AM
Reading your sample log messages closer I can see that the coordinator received a heartbeat from node 3 "2022-01-19 16:04:52,573 INFO [Process Cluster Protocol Request-30] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 35f2ed1a-ca6f-4cc6-ab4a-6c0774fc9c6d (type=HEARTBEAT, length=2837 bytes) from nifi-hatest-03:9091 in 19 millis" So I also wonder if we have a caching rendering issue, can you see what the UI shows using incognito mode? And finally if this is a DEV environment you can also try and delete your local state directory, that value is set in file "state-management.xml" Deleting state will clear out any local state your processors might depend on if configured as such, so remove with caution. It will also clear out cluster node ID's it local knows of.
... View more
01-19-2022
09:35 AM
This to me sounds like a hostname issue. could you confirm the value on nifi.properites for: nifi.cluster.node.address= nifi.web.https.host= Those values should match for the name of the host. If nothing there stands out check log entries for latest messages contain below strings: "org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator: Status" "org.apache.nifi.controller.StandardFlowService: Setting Flow Controller's Node ID:" "org.apache.nifi.web.server.HostHeaderHandler"
... View more
01-19-2022
09:01 AM
Can I clarify that what you want to do is dealing with XML and XPath? Or based on this "I'm trying to pass an output value of a processor dynamically into another and use it as a new property" Output values in to flowfile attributes depend on the previous processor. If you are not dealing with XML then the processor you most likely need is ExtractText followed by an RouteOnAttribute
... View more