About Anderosn

Anderosn · ‎01-11-2024

I have a API which I need to make a call to get the response using InvokeHttpProcessor. Incase if response status code is not 200's then I need to do a retry for 3 times ( incase of 400's and 500's). I see that I can set a retry on Failure, NO retry and Retry relationships but that is not working as expected. How to configure the relationships such that there I can achieve retry of 3 times on Failure, Retry and NoRetry relationships. Also is there a way to see how many retries are performed and is nifi logging this information some where?

Anderosn · ‎12-12-2023

Hi @SAMSAL when I try in my local with the schema text that you provided. I am seeing this error: UpdateRecord[id=5e779eb4-018c-1000-b9ea-047535bdcfb5] Failed to write MapRecord[{SEARCH_RESULT=unescapeJson(/SEARCH_RESULT, 'true')}] with reader schema ["SEARCH_RESULT" : "STRING"] and writer schema {"type":"record","name":"nifi","namespace":"nifi.com","fields":[{"name":"SEARCH_RESULT","type":{"type":"record","name":"SEARCH_RESULT","fields":[{"name":"requestId","type":"string"},{"name":"responseData","type":{"type":"array","items":{"type":"record","name":"responseData","fields":[{"name":"searchKey","type":"string"},{"name":"data","type":{"type":"array","items":{"type":"record","name":"data","fields":[{"name":"firstName","type":"string"},{"name":"lastName","type":"string"}]}}}]}}}]}}]} as a JSON Object due to org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [unescapeJson(/SEARCH_RESULT, 'true')] of type class java.lang.String to Record for field SEARCH_RESULT org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [unescapeJson(/SEARCH_RESULT, 'true')] of type class java.lang.String to Record for field SEARCH_RESULT at Input that I am using is: [{ "SEARCH_RESULT": "{ \"requestId\": \"203680\", \"responseData\": [ { \"searchKey\": \"cardNumber\", \"data\": [ { \"firstName\": \"Martin\", \"lastName\": \"Garry\" }, { \"firstName\": \"Martin\", \"lastName\": \"Garry\"}, { \"firstName\": \"Martin\", \"lastName\": \"Garry\", } ] } ] }" }] also tried with { "SEARCH_RESULT": "{ \"requestId\": \"203680\", \"responseData\": [ { \"searchKey\": \"cardNumber\", \"data\": [ { \"firstName\": \"Martin\", \"lastName\": \"Garry\" }, { \"firstName\": \"Martin\", \"lastName\": \"Garry\"}, { \"firstName\": \"Martin\", \"lastName\": \"Garry\", } ] } ] }" } org.apache.nifi.serialization.record.util.DataTypeUtils.toRecord(DataTypeUtils.java:444)

Anderosn · ‎12-11-2023

one thing that worked for me use combination of EvaluateJsonPath -> UpdateAttribute -> ReplaceText I have added the entire json to attribute and in UpdateAttribute I have used "${searchValue:unescapeJson():replace('"{','{'):replace('"}','}')}" which does the trick and then used ReplaceText to replace the Entire Json content.

Anderosn · ‎12-11-2023

@joseomjr tried you suggestion, still seeing the same { "SEARCH_RESULT": "{ \"requestId\": \"203680\", \"responseData\": [ { \"searchKey\": \"cardNumber\", \"data\": [ { \"firstName\": \"Martin\", \"lastName\": \"Garry\" }, { \"firstName\": \"Martin\", \"lastName\": \"Garry\"}, { \"firstName\": \"Martin\", \"lastName\": \"Garry\", } ] } ] }" }

Anderosn · ‎12-11-2023

@joseomjr Thank you for you response. I have thought of execute script. but Nifi team in my company doesn't allow us to use ExecuteScript ( unless you are able to get desired solution with existing processors). Can you pls share how we can get it working using evaluateJsonPath? I have tried EvaluteJsonPath using $.[0].SEARCH_RESULT and Return Type as Json it is still picking up entire key value pair instead of value itself. Output of EvaluteJsonPath { "SEARCH_RESULT": "{ \"requestId\": \"203680\", \"responseData\": [ { \"searchKey\": \"cardNumber\", \"data\": [ { \"firstName\": \"Martin\", \"lastName\": \"Garry\" }, { \"firstName\": \"Martin\", \"lastName\": \"Garry\"}, { \"firstName\": \"Martin\", \"lastName\": \"Garry\", } ] } ] }" } Another question if you have array of objects how to convert each of them using EvaluteJsonPath?

Anderosn · ‎12-11-2023

@SAMSAL in the provided output it is a single record, the column that I am trying to pull is a CLOB on SQL and the query is pulling only one column and the output contains only one record.

Anderosn · ‎12-11-2023

Hi @SAMSAL yes I have tried it now, it is also returning the same [ { "SEARCH_RESULT": "{ \"requestId\": \"203680\", \"responseData\": [ { \"searchKey\": \"cardNumber\", \"data\": [ { \"firstName\": \"Martin\", \"lastName\": \"Garry\" }, { \"firstName\": \"Martin\", \"lastName\": \"Garry\"}, { \"firstName\": \"Martin\", \"lastName\": \"Garry\", } ] } ] }" } ]

Anderosn · ‎12-11-2023

I am using ExecuteSQL to pull a colb column from DB. ExecuteSQL provides the result in the avro format. Then I am converting that to Json using AvroToJsonProcessor but it is adding escape characters. I want the value that is being pulled from DB as a Json but I am getting it as a string with escape characters. [ { "SEARCH_RESULT": "{ \"requestId\": \"203680\", \"responseData\": [ { \"searchKey\": \"cardNumber\", \"data\": [ { \"firstName\": \"Martin\", \"lastName\": \"Garry\" }, { \"firstName\": \"Martin\", \"lastName\": \"Garry\"}, { \"firstName\": \"Martin\", \"lastName\": \"Garry\", } ] } ] }" } ] I want that value of Search result to be a Json instead of a String so that I can use JOLT to transform it as needed. Can you someone pls suggest how to do it.

Anderosn · ‎10-12-2023

I have a flow file which needs to be saved to Database. Size of flow file content can be <= 2 MB. I have tried the PutSQL but it throws ORA-01704 as the insert statement with CLOB in it becomes too large. I have tried using PutDatabaseRecord, but it doesn't store complete flowfile in teh clob column, Information that it stores is this " apRecord[{responseMetadata=MapRecord[{responseType=ACTIVITY, activityDataExist=true, partNo=1, noOfParts=6}], responseData=MapRecord[{responseList=[Ljava.lang.Object;@676af33}]}]" @cotopaul @SAMSAL @MattWho @steven-matison @TimothySpann Can you pls look into this looks like it is wrapping the Json with MapRecord. Can someone pls suggest how to insert a flowfile content into a CLOB column in DB. I cannot use ExecuteScript to use PL/SQL ( ExecuteScript is not approved to use by Nifi team in my company).

Anderosn · ‎08-22-2023

I have flowfile which needs to be sent to two different processors which does transformation on the Json. Flow will be split into two parts which are independent of each other so they can be performed parallel but the input for these two parts is the same flow file. how can I send a flowfile in a queue to two different processors at same time.

Online	Offline
Last Visited	‎02-15-2024 09:28 AM

Member Since	‎07-27-2023 05:39 PM
Last Visited	‎02-15-2024 09:28 AM
Posts	18
Kudos received	1

Cloudera Community

Re: Avro to Json adding extra delemeters

InvokeHttpProcessor - Need to retry 3 times in cas...

Re: Avro to Json adding extra delemeters

Re: Avro to Json adding extra delemeters

Re: Avro to Json adding extra delemeters

Re: Avro to Json adding extra delemeters

Re: Avro to Json adding extra delemeters

Re: Avro to Json adding extra delemeters

Avro to Json adding extra delemeters

Inserting CLOB into Datbase throws ORA-01704

Send flowfile to two different processors in paral...