Support Questions
Find answers, ask questions, and share your expertise
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- Cloudera Community
- Support
- Support Questions
- Re: How do we find the time difference between con...
Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How do we find the time difference between contents of a flowfile and assign them to a attribute?
Labels:
- Labels:
-
Apache NiFi
Explorer
Created 11-11-2019 08:28 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have the source timestamps from some simulation software, and passing them to Nifi - I need to find the time difference between 2 time stamps. Both of the timestamps are historical in nature and not related to the current time. Please help me with an example
6 REPLIES 6
Master Mentor
Created 11-12-2019 05:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Your question states "contents of a FlowFile". Each FlowFile only references one piece of content.
Are you saying the content of your FlowFile contains two different timestamps in it and you would like to calculate the time difference between those two timestamps?
If so, you would need to extract those two timestamps from the content in to unique FlowFile attributes on that FlowFile. You could then use an UpdateAttribute processor that creates another new FlowFile attribute that contains the time difference derived using the NiFi Expression Language (EL) to do the timestamp conversion to numbers and then subtract one from the other.
With out an example of your timestamps, it would be difficult to provide and example of a NiFi EL statement to accomplish this task.
Matt
Explorer
Created 11-12-2019 05:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the response Matt.
I recieve a continuous stream from one of the machines.
Let's say the timestamp of 1 flowfile is 10 am
And the next is 10 02 am, 10 05 am etc..
I am merging 20 such records using the merge content processor and getting a json response as a single flow file.
Now I need to write a loop to check for the number of records and compare the timestamps vs the previous timestamp and calculate it took 2 mins for first iteration. .. ur. 10 to 10 02. Then 3 mins for next. Etc.
The idea is to calculate cycle time between the first flow file and the next flow file content. Since only one flow file is generated I have to merge contents of the flow file before this.
Let me know if there is a better way to handle this scenario.
Master Mentor
Created 11-12-2019 07:33 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Raghu_daredev
Just to me clear, the timestamps you want to compare are part of the json content of each FlowFile before the mergeContent or are you trying to compare the timestamp NiFi assigns to each FlowFile when it is created in NiFi.
If it is in the JSON content, you could use the EvaluateJsonPath processor to extract the timestamps in to NiFi attributes and then compare those individual timestamps in a series of NiFi EL statements in a single UpdateAttribute processor (assuming the exact same number of merged FlowFiles in each new Merged FlowFile).
If it based in timestamp created by NiFi when they became FlowFiles, this becomes more challenging.
While it is possible to persist these individual timestamp to a unique attribute on each FlowFile allowing them to persist as those unique attributes in the new merged FlowFile, the uniqueness is going to make it impossible to reference them in NiFi EL statements.
You may need to add the timestamp value to the source JSON before the merge so that it can be extracted numerically via EvaluateJsonPath after merge.
Matt
Explorer
Created 11-12-2019 08:25 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you Matt.
Yes, the content of the flowfile contains the time stamps.
But this is after merge content and not before. Let me send a sample
tomorrow of what I intend to do so that you could help me out.
Yes, the content of the flowfile contains the time stamps.
But this is after merge content and not before. Let me send a sample
tomorrow of what I intend to do so that you could help me out.
Explorer
Created on 11-13-2019 03:16 AM - edited 11-13-2019 03:18 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Attached ny sample template.
1. I get the OPC UA protocol data from the industrial machines - in my case currently, it is a simulation server called prosys.
2. Once the stream is setup - for every second - I am interested in one counter's value and am configuring the nodelist for the same to be pulled.
3. I split out the values and end up with value list which contains counter vd value and timestamp.
4. I need to aggregate 20 -30 of these records - i am using mergecontent for the same
5. Now i will need to calculate the time difference between each of those records and figure out cycle time that has elapsed between first and second record, second and third etc.... only if the time stamps are different.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description></description>
<groupId>71903a7e-016d-1000-1fd6-51588b10dcb7</groupId>
<name>TestOPCUA</name>
<snippet>
<connections>
<id>1ecfbd0d-a9f9-3ae1-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>faf6db52-86e2-3104-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<prioritizers>org.apache.nifi.prioritizer.OldestFlowFileFirstPrioritizer</prioritizers>
<selectedRelationships>Failure</selectedRelationships>
<selectedRelationships>Success</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>28951f8c-7304-303e-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>59936215-6b84-3e65-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>7aef6e2a-5188-30b3-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>95e6be48-1b47-317b-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>6c914493-5654-3e5e-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>7aef6e2a-5188-30b3-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<selectedRelationships>failure</selectedRelationships>
<selectedRelationships>original</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>7eabf75f-d2b9-3f33-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>a896bb2f-16e6-3608-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>95e6be48-1b47-317b-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<selectedRelationships>merged</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>7eabf75f-d2b9-3f33-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>b11ff6b9-c0fd-3c3e-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>77c4b1b5-e3f4-369f-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<prioritizers>org.apache.nifi.prioritizer.OldestFlowFileFirstPrioritizer</prioritizers>
<selectedRelationships>Success</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>faf6db52-86e2-3104-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>bcdb816b-b393-3156-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>485e873f-4cac-3233-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<selectedRelationships>failure</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>77c4b1b5-e3f4-369f-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>c0feca03-ce22-38f4-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>485e873f-4cac-3233-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<selectedRelationships>failure</selectedRelationships>
<selectedRelationships>unmatched</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>bce48e93-c7c3-3765-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>e8179182-d60d-3395-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>7eabf75f-d2b9-3f33-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<prioritizers>org.apache.nifi.prioritizer.OldestFlowFileFirstPrioritizer</prioritizers>
<selectedRelationships>matched</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>bce48e93-c7c3-3765-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>f42837a9-e582-3b64-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>485e873f-4cac-3233-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<selectedRelationships>Failure</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>faf6db52-86e2-3104-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>f8f50476-af5d-3526-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>bce48e93-c7c3-3765-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
<loadBalancePartitionAttribute></loadBalancePartitionAttribute>
<loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
<loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
<name></name>
<prioritizers>org.apache.nifi.prioritizer.OldestFlowFileFirstPrioritizer</prioritizers>
<selectedRelationships>split</selectedRelationships>
<source>
<groupId>d3b2fbd2-6ae7-3172-0000-000000000000</groupId>
<id>77c4b1b5-e3f4-369f-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<controllerServices>
<id>6c04e050-dd18-322f-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<bundle>
<artifact>nifi-opcua-bundle-nar</artifact>
<group>default</group>
<version>unversioned</version>
</bundle>
<comments></comments>
<descriptors>
<entry>
<key>Endpoint URL</key>
<value>
<name>Endpoint URL</name>
</value>
</entry>
<entry>
<key>Security Policy</key>
<value>
<name>Security Policy</name>
</value>
</entry>
<entry>
<key>Certificate for Server application</key>
<value>
<name>Certificate for Server application</name>
</value>
</entry>
<entry>
<key>Authentication Policy</key>
<value>
<name>Authentication Policy</name>
</value>
</entry>
<entry>
<key>User Name</key>
<value>
<name>User Name</name>
</value>
</entry>
<entry>
<key>Password</key>
<value>
<name>Password</name>
</value>
</entry>
<entry>
<key>Application Name</key>
<value>
<name>Application Name</name>
</value>
</entry>
</descriptors>
<name>StandardOPCUAService</name>
<persistsState>false</persistsState>
<properties>
<entry>
<key>Endpoint URL</key>
<value>opc.tcp://TSSHMRL256.corp.trelleborg.com:53530/OPCUA/SimulationServer</value>
</entry>
<entry>
<key>Security Policy</key>
<value>None</value>
</entry>
<entry>
<key>Certificate for Server application</key>
</entry>
<entry>
<key>Authentication Policy</key>
<value>Anon</value>
</entry>
<entry>
<key>User Name</key>
</entry>
<entry>
<key>Password</key>
</entry>
<entry>
<key>Application Name</key>
<value>NiFi</value>
</entry>
</properties>
<state>ENABLED</state>
<type>com.hashmapinc.tempus.processors.StandardOPCUAService</type>
</controllerServices>
<processors>
<id>28951f8c-7304-303e-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>360.6978525643334</x>
<y>0.0</y>
</position>
<bundle>
<artifact>nifi-opcua-bundle-nar</artifact>
<group>default</group>
<version>unversioned</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>OPC UA Service</key>
<value>
<identifiesControllerService>com.hashmapinc.tempus.processors.OPCUAService</identifiesControllerService>
<name>OPC UA Service</name>
</value>
</entry>
<entry>
<key>Recursive Depth</key>
<value>
<name>Recursive Depth</name>
</value>
</entry>
<entry>
<key>Node Filter</key>
<value>
<name>Node Filter</name>
</value>
</entry>
<entry>
<key>Print Indentation</key>
<value>
<name>Print Indentation</name>
</value>
</entry>
<entry>
<key>Remove Expanded Node ID</key>
<value>
<name>Remove Expanded Node ID</name>
</value>
</entry>
<entry>
<key>Max References Per Node</key>
<value>
<name>Max References Per Node</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>OPC UA Service</key>
<value>6c04e050-dd18-322f-0000-000000000000</value>
</entry>
<entry>
<key>Recursive Depth</key>
<value>3</value>
</entry>
<entry>
<key>Node Filter</key>
<value>ns=5;s=Counter1</value>
</entry>
<entry>
<key>Print Indentation</key>
<value>No</value>
</entry>
<entry>
<key>Remove Expanded Node ID</key>
<value>Yes</value>
</entry>
<entry>
<key>Max References Per Node</key>
<value>1000</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>2 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>GetOPCNodeList</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>Failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>Success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>com.hashmapinc.tempus.processors.GetOPCNodeList</type>
</processors>
<processors>
<id>485e873f-4cac-3233-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>1319.8058237557395</x>
<y>399.8453826904297</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.9.2</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Directory</key>
<value>
<name>Directory</name>
</value>
</entry>
<entry>
<key>Conflict Resolution Strategy</key>
<value>
<name>Conflict Resolution Strategy</name>
</value>
</entry>
<entry>
<key>Create Missing Directories</key>
<value>
<name>Create Missing Directories</name>
</value>
</entry>
<entry>
<key>Maximum File Count</key>
<value>
<name>Maximum File Count</name>
</value>
</entry>
<entry>
<key>Last Modified Time</key>
<value>
<name>Last Modified Time</name>
</value>
</entry>
<entry>
<key>Permissions</key>
<value>
<name>Permissions</name>
</value>
</entry>
<entry>
<key>Owner</key>
<value>
<name>Owner</name>
</value>
</entry>
<entry>
<key>Group</key>
<value>
<name>Group</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Directory</key>
<value>C:\Iot\test</value>
</entry>
<entry>
<key>Conflict Resolution Strategy</key>
<value>fail</value>
</entry>
<entry>
<key>Create Missing Directories</key>
<value>true</value>
</entry>
<entry>
<key>Maximum File Count</key>
</entry>
<entry>
<key>Last Modified Time</key>
</entry>
<entry>
<key>Permissions</key>
</entry>
<entry>
<key>Owner</key>
</entry>
<entry>
<key>Group</key>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>10 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>PutFile</name>
<relationships>
<autoTerminate>true</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>true</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.PutFile</type>
</processors>
<processors>
<id>77c4b1b5-e3f4-369f-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>79.36002249917351</x>
<y>205.2951328973337</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.9.2</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>JsonPath Expression</key>
<value>
<name>JsonPath Expression</name>
</value>
</entry>
<entry>
<key>Null Value Representation</key>
<value>
<name>Null Value Representation</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>JsonPath Expression</key>
<value>$.values</value>
</entry>
<entry>
<key>Null Value Representation</key>
<value>empty string</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>SplitJson</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>true</autoTerminate>
<name>original</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>split</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.SplitJson</type>
</processors>
<processors>
<id>7aef6e2a-5188-30b3-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>967.9091745854496</x>
<y>688.3724595011644</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.9.2</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Directory</key>
<value>
<name>Directory</name>
</value>
</entry>
<entry>
<key>Conflict Resolution Strategy</key>
<value>
<name>Conflict Resolution Strategy</name>
</value>
</entry>
<entry>
<key>Create Missing Directories</key>
<value>
<name>Create Missing Directories</name>
</value>
</entry>
<entry>
<key>Maximum File Count</key>
<value>
<name>Maximum File Count</name>
</value>
</entry>
<entry>
<key>Last Modified Time</key>
<value>
<name>Last Modified Time</name>
</value>
</entry>
<entry>
<key>Permissions</key>
<value>
<name>Permissions</name>
</value>
</entry>
<entry>
<key>Owner</key>
<value>
<name>Owner</name>
</value>
</entry>
<entry>
<key>Group</key>
<value>
<name>Group</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Directory</key>
<value>C:\Iot\matched</value>
</entry>
<entry>
<key>Conflict Resolution Strategy</key>
<value>fail</value>
</entry>
<entry>
<key>Create Missing Directories</key>
<value>true</value>
</entry>
<entry>
<key>Maximum File Count</key>
</entry>
<entry>
<key>Last Modified Time</key>
</entry>
<entry>
<key>Permissions</key>
</entry>
<entry>
<key>Owner</key>
</entry>
<entry>
<key>Group</key>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>10 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>PutFile</name>
<relationships>
<autoTerminate>true</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>true</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.PutFile</type>
</processors>
<processors>
<id>7eabf75f-d2b9-3f33-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>0.0</x>
<y>726.632736869651</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.9.2</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Merge Strategy</key>
<value>
<name>Merge Strategy</name>
</value>
</entry>
<entry>
<key>Merge Format</key>
<value>
<name>Merge Format</name>
</value>
</entry>
<entry>
<key>Attribute Strategy</key>
<value>
<name>Attribute Strategy</name>
</value>
</entry>
<entry>
<key>Correlation Attribute Name</key>
<value>
<name>Correlation Attribute Name</name>
</value>
</entry>
<entry>
<key>mergecontent-metadata-strategy</key>
<value>
<name>mergecontent-metadata-strategy</name>
</value>
</entry>
<entry>
<key>Minimum Number of Entries</key>
<value>
<name>Minimum Number of Entries</name>
</value>
</entry>
<entry>
<key>Maximum Number of Entries</key>
<value>
<name>Maximum Number of Entries</name>
</value>
</entry>
<entry>
<key>Minimum Group Size</key>
<value>
<name>Minimum Group Size</name>
</value>
</entry>
<entry>
<key>Maximum Group Size</key>
<value>
<name>Maximum Group Size</name>
</value>
</entry>
<entry>
<key>Max Bin Age</key>
<value>
<name>Max Bin Age</name>
</value>
</entry>
<entry>
<key>Maximum number of Bins</key>
<value>
<name>Maximum number of Bins</name>
</value>
</entry>
<entry>
<key>Delimiter Strategy</key>
<value>
<name>Delimiter Strategy</name>
</value>
</entry>
<entry>
<key>Header File</key>
<value>
<name>Header File</name>
</value>
</entry>
<entry>
<key>Footer File</key>
<value>
<name>Footer File</name>
</value>
</entry>
<entry>
<key>Demarcator File</key>
<value>
<name>Demarcator File</name>
</value>
</entry>
<entry>
<key>Compression Level</key>
<value>
<name>Compression Level</name>
</value>
</entry>
<entry>
<key>Keep Path</key>
<value>
<name>Keep Path</name>
</value>
</entry>
<entry>
<key>Tar Modified Time</key>
<value>
<name>Tar Modified Time</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Merge Strategy</key>
<value>Bin-Packing Algorithm</value>
</entry>
<entry>
<key>Merge Format</key>
<value>Binary Concatenation</value>
</entry>
<entry>
<key>Attribute Strategy</key>
<value>Keep Only Common Attributes</value>
</entry>
<entry>
<key>Correlation Attribute Name</key>
</entry>
<entry>
<key>mergecontent-metadata-strategy</key>
<value>Use First Metadata</value>
</entry>
<entry>
<key>Minimum Number of Entries</key>
<value>30</value>
</entry>
<entry>
<key>Maximum Number of Entries</key>
<value>1000</value>
</entry>
<entry>
<key>Minimum Group Size</key>
<value>0 B</value>
</entry>
<entry>
<key>Maximum Group Size</key>
</entry>
<entry>
<key>Max Bin Age</key>
<value>2 min</value>
</entry>
<entry>
<key>Maximum number of Bins</key>
<value>1</value>
</entry>
<entry>
<key>Delimiter Strategy</key>
<value>Filename</value>
</entry>
<entry>
<key>Header File</key>
</entry>
<entry>
<key>Footer File</key>
</entry>
<entry>
<key>Demarcator File</key>
</entry>
<entry>
<key>Compression Level</key>
<value>1</value>
</entry>
<entry>
<key>Keep Path</key>
<value>false</value>
</entry>
<entry>
<key>Tar Modified Time</key>
<value>${file.lastModifiedTime}</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>60 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>MergeContent</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>merged</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>original</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.MergeContent</type>
</processors>
<processors>
<id>95e6be48-1b47-317b-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>255.08104534885285</x>
<y>990.0772998138473</y>
</position>
<bundle>
<artifact>nifi-update-attribute-nar</artifact>
<group>org.apache.nifi</group>
<version>1.9.2</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Delete Attributes Expression</key>
<value>
<name>Delete Attributes Expression</name>
</value>
</entry>
<entry>
<key>Store State</key>
<value>
<name>Store State</name>
</value>
</entry>
<entry>
<key>Stateful Variables Initial Value</key>
<value>
<name>Stateful Variables Initial Value</name>
</value>
</entry>
<entry>
<key>canonical-value-lookup-cache-size</key>
<value>
<name>canonical-value-lookup-cache-size</name>
</value>
</entry>
<entry>
<key>NewValue</key>
<value>
<name>NewValue</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Delete Attributes Expression</key>
</entry>
<entry>
<key>Store State</key>
<value>Do not store state</value>
</entry>
<entry>
<key>Stateful Variables Initial Value</key>
</entry>
<entry>
<key>canonical-value-lookup-cache-size</key>
<value>100</value>
</entry>
<entry>
<key>NewValue</key>
<value>${tokenvalue:plus(20)}</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>UpdateAttribute</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.attributes.UpdateAttribute</type>
</processors>
<processors>
<id>bce48e93-c7c3-3765-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>465.5156136614967</x>
<y>507.3796516914497</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.9.2</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Destination</key>
<value>
<name>Destination</name>
</value>
</entry>
<entry>
<key>Return Type</key>
<value>
<name>Return Type</name>
</value>
</entry>
<entry>
<key>Path Not Found Behavior</key>
<value>
<name>Path Not Found Behavior</name>
</value>
</entry>
<entry>
<key>Null Value Representation</key>
<value>
<name>Null Value Representation</name>
</value>
</entry>
<entry>
<key>timestamp</key>
<value>
<name>timestamp</name>
</value>
</entry>
<entry>
<key>tokenvalue</key>
<value>
<name>tokenvalue</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Destination</key>
<value>flowfile-attribute</value>
</entry>
<entry>
<key>Return Type</key>
<value>auto-detect</value>
</entry>
<entry>
<key>Path Not Found Behavior</key>
<value>ignore</value>
</entry>
<entry>
<key>Null Value Representation</key>
<value>empty string</value>
</entry>
<entry>
<key>timestamp</key>
<value>$.ts</value>
</entry>
<entry>
<key>tokenvalue</key>
<value>$.vd</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>EvaluateJsonPath</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>matched</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>unmatched</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.EvaluateJsonPath</type>
</processors>
<processors>
<id>faf6db52-86e2-3104-0000-000000000000</id>
<parentGroupId>d3b2fbd2-6ae7-3172-0000-000000000000</parentGroupId>
<position>
<x>1031.8422007088645</x>
<y>18.125730514526367</y>
</position>
<bundle>
<artifact>nifi-opcua-bundle-nar</artifact>
<group>default</group>
<version>unversioned</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>OPC UA Service</key>
<value>
<identifiesControllerService>com.hashmapinc.tempus.processors.OPCUAService</identifiesControllerService>
<name>OPC UA Service</name>
</value>
</entry>
<entry>
<key>Return Timestamp</key>
<value>
<name>Return Timestamp</name>
</value>
</entry>
<entry>
<key>Exclude Null Value</key>
<value>
<name>Exclude Null Value</name>
</value>
</entry>
<entry>
<key>Null Value String</key>
<value>
<name>Null Value String</name>
</value>
</entry>
<entry>
<key>DATA_FORMAT</key>
<value>
<name>DATA_FORMAT</name>
</value>
</entry>
<entry>
<key>LONG_TIMESTAMP</key>
<value>
<name>LONG_TIMESTAMP</name>
</value>
</entry>
<entry>
<key>TEMPUS_DEVICE_TYPE</key>
<value>
<name>TEMPUS_DEVICE_TYPE</name>
</value>
</entry>
<entry>
<key>TEMPUS_DEVICE_NAME</key>
<value>
<name>TEMPUS_DEVICE_NAME</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>OPC UA Service</key>
<value>6c04e050-dd18-322f-0000-000000000000</value>
</entry>
<entry>
<key>Return Timestamp</key>
<value>SourceTimestamp</value>
</entry>
<entry>
<key>Exclude Null Value</key>
<value>false</value>
</entry>
<entry>
<key>Null Value String</key>
</entry>
<entry>
<key>DATA_FORMAT</key>
<value>JSON</value>
</entry>
<entry>
<key>LONG_TIMESTAMP</key>
<value>false</value>
</entry>
<entry>
<key>TEMPUS_DEVICE_TYPE</key>
<value>Device</value>
</entry>
<entry>
<key>TEMPUS_DEVICE_NAME</key>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>2 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<executionNodeRestricted>false</executionNodeRestricted>
<name>GetOPCData</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>Failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>Success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>com.hashmapinc.tempus.processors.GetOPCData</type>
</processors>
</snippet>
<timestamp>11/13/2019 16:42:20 IST</timestamp>
</template>
Explorer
Created 11-19-2019 09:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Any updates to this one, Matt ?
Announcements
What's New @ Cloudera
What's New @ Cloudera
What's New @ Cloudera
What's New @ Cloudera
What's New @ Cloudera
