Member since
02-05-2021
18
Posts
1
Kudos Received
0
Solutions
01-23-2022
10:55 PM
Hi, I am currently trying to count the number of files in the directory. The command that I keyed to count in the windows command prompt is "dir /a:-d /s /b "D:\Testing\UploadFile" | find /c ":"" I am trying to use NiFi to apply the above command using the ExecuteStreamCommand processor. Currently I am facing the "Access Denied error" as shown in the picture. Execute Stream Command Processor Error Execute Stream Command Processor Properties Appreciate if anyone could help with an example and explanation. Thanks. Note: the fileslocation set in the properties is the path to the file.
... View more
Labels:
- Labels:
-
NiFi Registry
01-06-2022
10:52 PM
Scenario: I have a List File Processor looking out for incoming file ("file 1"). I scheduled it to start picking up the "file 1" 20s after the "file 1" is downloaded. Let assume that the List File Processor noticed the incoming "file 1" and started the delay 20 sec before picking the "file 1". In the middle of 20 sec delay, there is an new incoming file ("file 2") noticed by the List File Processor. List File Processor will be interrupt and reset the 20 sec schedule time when ("file 2") appear within the previous 20 sec delay Could the List File Processor in the middle of 20 sec delay be interrupt and restart the initial delay? Appreciate help with explanation and example. Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
08-25-2021
03:39 AM
Hi, Would like to seek an example of caching the 2 singe flow file attribute (filename attribute) and store it for later part processing. Scenario: 1) Folder containing (file1.xml and file2.txt) 2) Using NiFi GetFile processor to get both file 3) Get the attribute of both files (filename) 4) store it for future storage into database Looking forward for the help with example and explanation. Appreciate the helps. Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
06-17-2021
08:06 PM
Hi @MattWho Thanks for your help. Appreciate.
... View more
06-17-2021
06:58 PM
Hi @MattWho The following is my use case: There are 2 computer A & B. Computer A has a folder named "LocalFolder" (example: C:\User\LocalFolder\) Computer B has a GetSFTP NiFi processor that listen to and extract files inside the Computer A "LocalFolder" Folder In odd cases, Computer A will generate a new folder named "newFolder" located in the "C:\User\LocalFolder" (example: C:\User\LocalFolder\newFolder) which is not known to Computer B (sudden appear). Is there any way computer B will 1) Get known of the new generated folder by Computer A and Computer B will generate the folder that is same at Computer A size 2) Extract all the file in computer A folder to Computer B newly generated folder Thank & it will be a good help if there is Example with internal NiFi configuration and explanation. Appreciate
... View more
06-16-2021
07:09 PM
Hi @MattWho, Apologies as I made a mistake on the example. The example should be as follow: Input: <particular> <name>Nicholas</name> <height unit = "meter">183</height> <weight unit = "kilogram" unit = "lb">75</weight> </particular> <particular> <name>Debbie</name> <height unit = "meter">163</height> <weight unit = "kilogram" unit = "lb">45</weight> </particular> Output: <particular> <name>Nicholas</name> <height>183</height> <weight>75</weight> </particular> <particular> <name>Debbie</name> <height>163</height> <weight>45</weight> </particular> Are there some ways that NiFi processor could remove the attributes (unit or etc..) inside the XML tag name as mention in the example above? It will be a great help if there is an example with internal processor configuration shown and explanation. Appreciate your help. Thanks.
... View more
06-15-2021
09:17 PM
Hi I currently have to extract folder from other computer via NiFi SFTP. I been finding for solution and notice that the NiFi processor (GetFile, GetSFTP) only allow to extract files instead of folder. Is there any nifi processor that is able to extract the folder? Example: Computer 1 directory: C:\User\localFolder Computer 2 Using GetSFTP processor (remote path: C:\User) to extract the folder Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
06-15-2021
06:36 PM
Hi, is there some way that nifi could remove the attributes inside the XML tag name? Please advise with some example to help me. Appreciate. Thanks Example: <particular> <name>Nicholas</name> <height>183</height> <weight>75</weight> </particular> <particular> <name>Debbie</name> <height163</height> <weight>45</weight> </particular> to something like: <particular> <name>Nicholas</name> <height unit = "meter">183</height> <weight unit = "kilogram" unit = "lb">75</weight> </particular> <particular> <name>Debbie</name> <height unit = "meter">163</height> <weight unit = "kilogram" unit = "lb">45</weight> </particular>
... View more
- Tags:
- NiFi
- xml
- xml tag name
Labels:
- Labels:
-
Apache NiFi
05-14-2021
08:54 AM
@MattWho Hi Matt, Thanks for the reply. I am sending a 1kb file over the rest endpoint of a different IP machine. What I am asking here is how do I configure the invokeHTTP request using IP address? Example: Machine 1 ( 192.168.4.1) which contain NiFi (GetFile processor --> invokeHTTP processor) to Machine 2 (192.168.3.1) which contain the Wildfly restful webservice. Thanks.
... View more
05-13-2021
11:57 PM
Current setup, there are 2 computer machine 1) First machine (IP of 192.168.4.1) It contain the NiFi setup as show in the image below: Inside the NiFi setup, I get a file from a local directory and I invoke a HTTP POST to send over to the second machine which hit the Wildfly Restful Webservice endpoint. Below image show the configuration I have done inside the Invoke HTTP: (Note: the Remote URL: http://192.168.3.1:8081/JaxRS-Nifi/restapi/multipart/upload ) 2) Second machine (IP of 192.168.3.1) It contain a Wildfly Restful Webservice which allow my first machine to hit the webservice endpoint. Error that I am facing: I have a problem which the first machines doesn't send a HTTP POST over to the Second machines. I am currently looking for solutions on what are the different configurations should I do to make a NiFi HTTP over to hit the Restful Webservice endpoint? Thanks. Appreciate if there is anyone who can help me.
... View more
- Tags:
- NiFi
Labels:
- Labels:
-
Apache NiFi
04-20-2021
12:41 AM
Hi, I got a problem while sending a Zero Byte flow file through Nifi PutSFTP processor. Wondering if there is some configuration I have to set? thanks. Use case: get a file from a directory (GetFile Processor) send the file through SFTP (PutSFTP Processor) if success, change the filename to new filename (UpdateAttribute Processor) Remove the all of the content (ReplaceText Processor). Zero Byte flow file send the new filename through SFTP (PutSFTP Processor) I gotten this error while sending Zero Byte flow file over using PutSFTP Processor PutSFTP[id=1a091db4-1f9e-14cd-5206-1514485febc8] Rejecting StandardFlowFileRecord[uuid=247457a8-6608-4271-a626-3c016a1bbe7c,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1618805512561-40, container=default, section=40], offset=15043, length=0],offset=0,name=AS_MPS_20200927_010010.xml.done,size=0] because it is zero bytes It will be great if someone could help me. Thanks in advance.
... View more
Labels:
02-07-2021
06:45 PM
Hi, Thanks for the good explanation. What would be the jolt specification for the following input/output. There are two input Json: 1st Json input: { "Name" : "Alvin", "Status" : "Single", "Life" : [ { "Sport" : "Swimming", "Singing" : "K-box", "Food" : "Burger", "Alcohol" : "Rum" }, { "Sport" : "Boxing", "Singing" : "party world", "Food" : "Chicken Wing", "Alcohol" : "Whisky" }, { "Sport" : "Running", "Singing" : "KTV", "Food" : "Muffin", "Alcohol" : "Martel" }] } 2nd Json input: { "Name" : "Alvin", "Status" : "Single", "Life" : { "Sport" : "Swimming", "Singing" : "K-box", "Food" : "Burger", "Alcohol" : "Rum" } } This two Json message input should go to a same JoltTransformJson processor and come out with the following output: 1st Json output: { "Name" : "Alvin", "Status" : "Single", "Sport" : [ "Swimming", "Boxing", "Running"], "Singing" : [ "K-box", "party world" , "KTV"], "Food" : [ "Burger", "Chicken Wing" , "Muffin"], "Alcohol" : [ "Rum", "Whisky", "Martel"] } 2nd Json output: { "Name" : "Alvin", "Status" : "Single", "Sport" : [ "Swimming"], "Singing" : [ "K-box"], "Food" : [ "Burger"], "Alcohol" : [ "Rum"] } How can I configure the JoltTransformJson processor to get the above output? Or is there any other ways to do it? Please advise with step and example. appreciate
... View more
02-05-2021
06:11 AM
1 Kudo
There are two input Json: 1st Json input: { "Name" : "Alex", "Status" : "Single", "Life" : [ { "Sport" : "Swimming", "Singing" : "K-box", "Food" : "Burger", "Alcohol" : "Rum" }, { "Sport" : "Boxing", "Singing" : "party world", "Food" : "Chicken Wing", "Alcohol" : "Whisky" }, { "Sport" : "Running", "Singing" : "KTV", "Food" : "Muffin", "Alcohol" : "Martel" }] } 2nd Json input: { "Name" : "Alex", "Status" : "Single", "Life" : { "Sport" : "Swimming", "Singing" : "K-box", "Food" : "Burger", "Alcohol" : "Rum" } } This two Json message input should go to a same JoltTransformJson processor and come out with the following output: 1st Json output: { "Name" : "Alex", "Status" : "Single", "Sport" : [ "Swimming", "Boxing", "Running"], "Singing" : [ "K-box", "party world" , "KTV"], "Food" : [ "Burger", "Chicken Wing" , "Muffin"], "Alcohol" : [ "Rum", "Whisky", "Martel"] } 2nd Json output: { "Name" : "Alex", "Status" : "Single", "Sport" : [ "Swimming"], "Singing" : [ "K-box"], "Food" : [ "Burger"], "Alcohol" : [ "Rum"] } How can I configure the JoltTransformJson processor to get the above output? Or is there any other ways to do it? Please advise with step and example. appreciate.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)