SplitJSON doesnt split anything

[#1]

Hello,

I'm new to nifi so my problem may be dumb. Sorry if it seems really basic.

I'm trying to use nifi to :

1/ Read a JSON response from a REST GET endpoint.
2/ Parse the JSON and split some JSON array into JSON objects

3/ insert objects in SQL.

I'm stuck at Step 2 because my SplitJSON doesnt return anything. No error as well.

Here is a sample of my JSON :

{
   "context":"q%3D%2523all%26q.restriction%3Dclass%253Acr_hashtag%26of%3Djson",
   "nhits":1000000,
   "nmatches":1000000,
   "start":0,
   "ellql":"#query{nbdocs=26966678, text_relevance.expr=\"@term.score*@proximity+@b\", noproximity, term.score=NO_RANKING}(#and(#true() #category(categories,\"Top/datamodelclass/cr_hashtag\") ))",
   "executor":null,
   "estimated":false,
   "autocorrected":false,
   "spellCheckSuggestions":{
      "origStr":null,
      "suggestions":[
         
      ]
   },
   "hits":[
      {
         "did":1,
         "url":"ID=10055&",
         "buildGroup":"bg0",
         "source":null,
         "slice":3,
         "score":0,
         "sort":0,
         "groups":[
            
         ],
         "metas":[
            {
               "name":"created_by",
               "type":2,
               "value":"L37"
            },
            {
               "name":"created_dttm",
               "type":2,
               "value":"2022/07/13 11:29:30"
            },
            {
               "name":"last_modified_dttm",
               "type":2,
               "value":"2022/07/13 11:29:30"
            },
            {
               "name":"legal_entity_id",
               "type":2,
               "value":"100000000000313"
            },
            {
               "name":"source",
               "type":2,
               "value":"consolidation-cbx_bg0"
            },
            {
               "name":"tags",
               "type":2,
               "value":"test10055"
            },
            {
               "name":"id",
               "type":2,
               "value":"10055"
            },
            {
               "name":"url",
               "type":2,
               "value":"ID=10055&"
            }
         ]
      },
      {
         "did":2,
         "url":"ID=10119&",
         "buildGroup":"bg0",
         "source":null,
         "slice":4,
         "score":0,
         "sort":0,
         "groups":[
            
         ],
         "metas":[
            {
               "name":"created_by",
               "type":2,
               "value":"L37"
            },
            {
               "name":"created_dttm",
               "type":2,
               "value":"2022/07/13 11:29:30"
            },
            {
               "name":"last_modified_dttm",
               "type":2,
               "value":"2022/07/13 11:29:30"
            },
            {
               "name":"legal_entity_id",
               "type":2,
               "value":"100000000000313"
            },
            {
               "name":"source",
               "type":2,
               "value":"consolidation-cbx_bg0"
            },
            {
               "name":"tags",
               "type":2,
               "value":"test10119"
            },
            {
               "name":"id",
               "type":2,
               "value":"10119"
            },
            {
               "name":"url",
               "type":2,
               "value":"ID=10119&"
            }
         ]
      }
   ]
}

I want to split the "hits" array.
This is my nifi config so far. I'm able to get the JSON from the invokeHTTP

Still, the output of the relation "split" is empty... The one of "failure" as well. I have no idea what's going on.

What am i doing wrong?

Thanks for your help

[#2]

@totokogure ,

I tested this on the same version of NiFi that you're using (1.17.0) and it worked fine for me.

The sample content that you provided was split in 2 flowfiles.

Cheers,

André

[#3]

@araujo 

Thanks for testing it. I also tried it with a different flow from raw JSON file read through GetFile Processor and managed to split the  JSON.

So as we can say it's not because of the SplitJSON processor, doest it mean i have some issue with the input format received from the SplitJSON aka the output of the InvokedHTTP?

Is there any chance that SplitJSON doesnt recognise the http response as a JSON and skips it?

[#4]

I managed to make it work by introducing a evaluateJsonPath between invokehttp and splitjson : 

Config of evaluateJSONPath:

Config of SplitJSON:

Is it supposed to work like this?

[#5 · Accepted solution]

@totokogure ,

The EvaluateJsonPath that you added is extracting the $.hits array and storing it as an attribute. The next processor (SplitJson), does not even use that attribute, though. It extracts $.hits again from the flowfile content. The EvaluateJsonPath in this flow should be unnecessary. If you connected InvokeHTTP to SplitJson, things should work correctly.

Cheers,

André