Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Nifi : Is it possible to create multiple flow file based on attributes ?

avatar
Explorer

Hello,

I receive a json from a HTTP Request. It looks like this : 

 

{
  "accessToken" : "token",
  "expiresIn" : "12600",
  "refreshToken" : "token",
  "tokenType" : "Bearer",
  "usagePointsId" : "22516914714270, 22516914714270, 22516914714270"
}

 

I have the usagePointsId key containing a string with multiple values. 
I will need each values to include them in different API calls.
I would therefore like to be able to separate the string at the comma level to extract each value.
Then the idea is to create a new unique flow file for each value. This value would be associated with a "usagePointId" attribute which would be used in my API calls. Also, the others json key would be added as Attributes for each flow files

I don't know if it's possible to create multiple differents flow file based on a split attributes. 
I tried to use the ExecuteScript (with ECMASCRIPT) to split the string and create Flow Files with something like this :

 

flowFile = session.get()

if (!flowFile) return
usagePointsId = flowFile.getAttribute('usagePointsId')
splitStr = usagePointsId.split(",");
let attrs = []

for (let i = 0; i < spltStr.length, i+) {
	newFlowFile = session.create()
	newFlowFile = session.write(newFlowFile, splitStr[i] )
	session.transfer(newFlowFile, REL_SUCCESS)
}

 

Someone could help me with that kind of issue ?

Thanks

1 ACCEPTED SOLUTION

avatar
Explorer

I finally used a Jolt transformation on the last ouput content :
input :

22516914714270

jolt spec :

[
  {
     "operation": "shift",
    	"spec": {
          "*": {
                  "$": "usagePointsId"
          }
    }
  }
]

ouput :
{"usagePointsId": "22516914714270"}

Then i used the EvaluateJsonPath to save the value as attributes

What a day 😅


 

View solution in original post

2 REPLIES 2

avatar
Explorer

I tried to transform the usagePointsId String to an array of element using a jolt transformation :

    {
    "operation": "modify-overwrite-beta",
    "spec": {
        "usagePointsId": "=split(',',@(1,usagePointsId))"
    }
  }

output : 

"usagePointsId" : [ "22516914714270", " 22516914714271", " 22516914714272" ]


Then i split this array with SplitJson on the usagePointsId path.

azg_0-1649948491867.png
It results multiple files with a simple string in content : 

azg_1-1649948555431.png

How can i save that kind of data as Attributes ? I try to use EvaluateJsonPath but unfortunatly, there is no path to precise. Is there a way to handle that ? 

 

avatar
Explorer

I finally used a Jolt transformation on the last ouput content :
input :

22516914714270

jolt spec :

[
  {
     "operation": "shift",
    	"spec": {
          "*": {
                  "$": "usagePointsId"
          }
    }
  }
]

ouput :
{"usagePointsId": "22516914714270"}

Then i used the EvaluateJsonPath to save the value as attributes

What a day 😅