Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

RecordLookup cannot parse incoming data

RecordLookup cannot parse incoming data

Explorer

I am really confused by how the RecordLookup processor reads in the data. I have two separate processes flows - one gets data from an API endpoint and the other reads a file (see both below). At the end of each of the flows I have a RecordLookup that searches HBase for a match. The transformed flowfiles I send to the RecordLookup processor look exactly the same as you can see below, but RecordLookup cannot parse the fllowfile from the API flow, and throws this error. It can ONLY parse the input from the file ingestion process.

 

Any ideas what I might be doing wrong here? Any help is greatly appreciated..

 

vatodorov19_1-1604067135054.png

 

AvroSchema used by the ConvertRecord and the RecordLookup processors.

 

 

 

{
  "name": "netflowSchema",
  "namespace": "nifi.examples",
  "type": "record",
  "fields": [
    { "name": "indicator", "type": "string" },
    { "name": "port", "type": "int" },
    { "name": "protocol", "type": "string" },
    { "name": "timestamp", "type": "string" },
    { "name": "threatq_status", "type": ["null","string"], "default": "null" }
  ]
}

 

 

 

 

ListenHTTP -> SplitJson -> EvaluateJsonPath -> JoltTransformJSON (jolt transform adds the threatq_status attribute)

vatodorov19_4-1604067773178.png

 

GetFile -> SplitText -> UpdateAttribute -> ConvertRecord (adds the threatq_status attribute) -> SplitJSON

vatodorov19_5-1604067782529.png

 

Valentin

 

Don't have an account?
Coming from Hortonworks? Activate your account here