Member since
02-25-2020
35
Posts
1
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1293 | 04-05-2020 01:51 PM | |
1576 | 03-15-2020 02:04 PM | |
4583 | 03-04-2020 12:15 PM |
09-11-2020
12:24 PM
@Gubbi use this: ListFile -> FetchFile -> ConvertRecord
... View more
08-14-2020
06:11 AM
@Gubbi Since you comment that NiFi starts fine when you remove the flow.xml.gz, this points at an issue with loading /reading the flow.xml.gz file. I would suggest opening you flow.xml.gz in an xml editor and make sure it is valid. When NiFi starts it loads the flow.xml in to heap memory. From then on all changes made within the UI are made in memory and a flow.xml.gz is written to disk. I would be looking for XML special characters (http://xml.silmaril.ie/specials.html) that were not escaped in the XML that written out to the flow.xml.gz. Manually remove or correct the invalid flow.xml.gz may work. You may get lucky going back and trying multiple archived copies of flow.xml.gz files. Perhaps one still exist prior to the config change that was made that introduced the issue? Note: Be aware that starting NiFi without the flow.xml.gz will result in loss of all queued FlowFiles. Hope this helps, Matt
... View more
04-05-2020
01:51 PM
Issue resolved. Had to re-install NiFi and services are up and running.
... View more
04-02-2020
11:45 AM
@Gubbi I think your ListFile proc is still executing 0 sec. Reference our private message.
... View more
04-02-2020
06:32 AM
1 Kudo
@Gubbi The next solution here is to just add each route for today, yesterday, day before yesterday. Then route all 3 to the next proc. Anything not matching won't be routed.
... View more
04-02-2020
06:29 AM
1 Kudo
@Gubbi The solution here is a now minute 24 hours: Yesterday: ${now():minus(86400000):format('MM-dd-yyyy hh:mm:ss') } Day Before Yesterday: ${now():minus(86400000):minus(86400000):format('MM-dd-yyyy hh:mm:ss') }
... View more
03-26-2020
10:22 PM
@shuhei_shogen : I had similar used case but when i tried to reproduce the same approach its not working.My file name is like equity_asia2.dif.gz.20200324 and i want these files to be in target folder 20200324. In Update Attribute i have used ${filename:matches(${today:prepend('[^\.].*gz.'):append('\d{8}')})}. But it doesnt seem to be working. Could you please check and assist me on this.
... View more
03-16-2020
05:09 PM
1 Kudo
@Gubbi Depending on which processor is being used to create your FlowFile from you source linux directory, you will likely have an "absolute.path" FlowFile attribute created on the FlowFile. absolute.path = /users/abc/20200312/gtry/ You can pass that FlowFile to an UpdateAttribute processor which can use NiFi Expression Language (EL) to extract the date from that absolute path in to a new FlowFile attribute Add new property (property name becomes new FlowFile attribute): Property: Value: pathDate ${absolute.path:getDelimitedField('4','/')} The resulting FlowFile will have a new attribute: pathDate = 20200312 Now you can use that FlowFile attribute later when writing to your target directory in S3. I assume you would use the putS3Object processor for this? If so, you can configure the "Object Key" property with the following: /Users/datastore/${pathDate}/${filename} NiFi EL will replace ${pathDate} with "20200312" and ${filename} will be replaced with "gyyy.csv". Hope this helps you, Matt
... View more
03-15-2020
02:04 PM
Issue has been resolved.
... View more
03-04-2020
12:15 PM
1 Kudo
I got solution for this. Had to use expression language in Object Key to fetch date from file and it worked. Below is expression :${filename:substringAfter('.gz.')}/${filename}
... View more