Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI Infer Schema trying to convert string(alpha) to date

avatar
Rising Star

While converting the input flow file JSON to any other format using query record (CSVwriter or AVRO Writer) using inferschema strategy the NIFI processor is trying to convert to Date based on first few characters of the incoming string.

NIFI Error Output

Successfully parsed a JSON object from input but failed to convert into a Record object with the given schema\n- Caused by: org.apache.nifi.serialization.MalformedRecordException: Successfully parsed a JSON object from input but failed to convert into a Record object with the given schema\n- Caused by: org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Failed Conversion of Field [sharepoint_documents_name__c] from String [10-7-20 2x10GE LAN PHY XXXXXXXXXXXX] to LocalDate\n- Caused by: java.time.format.DateTimeParseException: Text '10-7-20 2x10GE LAN PHY XXXXXXXXXXX' could not be parsed at index 0:  

 

Any workaround/fixes available in the newer version. Iam using NIFI 16.1

5 REPLIES 5

avatar
Super Guru

@Althotta ,

 

I tested this on 1.16.2 and the behaviour you described doesn't happen to me.

Would you be able to share you flow and processor/controller services configuration?

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
New Contributor

@araujo i'm also facing the similar issue with 1.16.3 and 1.16.2, i'm attaching the referenced images

Running NIFI Using docker in Mac M1

JSON input data

 

[ {
  "full_name" : "Anatolij Finkbeiner",
  "gender" : "male",
  "email" : "anatolij.finkbeiner@example.com",
  "phone" : "0166-2058423",
  "dob" : "2000-09-11T15:40:10.890Z",
  "uuid" : "747f27a8-88c0-452e-b360-3ddf5ee67211",
  "address" : "Germany, Schleswig-Holstein, Guben",
  "createdAt" : "2022-11-15T11:09:35.912Z"
}, {
  "full_name" : "Lucia Soto",
  "gender" : "female",
  "email" : "lucia.soto@example.com",
  "phone" : "946-359-112",
  "dob" : "1987-09-15T09:58:34.821Z",
  "uuid" : "8edb62cd-0bdc-41ee-8806-fecfd6958e33",
  "address" : "Spain, Navarra, Santiago de Compostela",
  "createdAt" : "2022-11-15T11:09:35.911Z"
} ]

 

Error

Screen Shot 2022-11-15 at 3.11.10 PM.png

JSON Tree READER

Screen Shot 2022-11-15 at 3.17.41 PM.png

Avro Schema

 

{
  "type": "record",
  "namespace": "nifi",
  "name": "userSchema",
  "fields": [
    {
      "name": "dob",
      "type": {
        "type": "string",
        "logicalType": "timestamp"
      }
    },
    {
      "name": "createdAt",
      "type": {
        "type": "long",
		"logicalType": "timestamp"
      }
    },
    {
      "name": "full_name",
      "type": "string"
    },
    {
      "name": "gender",
      "type": "string"
    },
    {
      "name": "email",
      "type": "string"
    },
    {
      "name": "address",
      "type": "string"
    },
    {
      "name": "uuid",
      "type": "string"
    },
    {
      "name": "phone",
      "type": "string"
    }
  ]
}

 

Avro Record set writer

Screen Shot 2022-11-15 at 3.15.52 PM.png

Avro reader

Screen Shot 2022-11-15 at 3.14.04 PM.png

avatar
New Contributor

@araujo I have faced the similar issue with Apache/Nifi 1.16.3 and 1.16.2, attaching

 

Avro Schema

{
  "type": "record",
  "namespace": "nifi",
  "name": "userSchema",
  "fields": [
    {
      "name": "dob",
      "type": {
        "type": "string",
        "logicalType": "timestamp"
      }
    },
    {
      "name": "createdAt",
      "type": {
        "type": "long",
		"logicalType": "timestamp"
      }
    },
    {
      "name": "full_name",
      "type": "string"
    },
    {
      "name": "gender",
      "type": "string"
    },
    {
      "name": "email",
      "type": "string"
    },
    {
      "name": "address",
      "type": "string"
    },
    {
      "name": "uuid",
      "type": "string"
    },
    {
      "name": "phone",
      "type": "string"
    }
  ]
}

 

the processor/controller services configuration Convert Record ProcessorConvert Record ProcessorError ImageError ImageAvro Reader ConfigurationAvro Reader ConfigurationAvro Record set writerAvro Record set writerJSON Tree READERJSON Tree READER

avatar
New Contributor

I'm able to fix this .. In JSON Tree READER the configuration for `Schema Access Strategy` should be "Infer Schema"
Input date format will be "yyyy-MM-dd HH:mm:ss.SSS'Z'"
avro schema for the same will be

{
      "name": "createdAt",
      "type": {
        "type": "long",
		"logicalType": "timestamp-millis"
      }
    }

 

avatar
Community Manager

@Althotta,   If you are still experiencing the issue, can you provide the information @araujo  has requested?



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: