question Re: NIFI Infer Schema trying to convert string(alpha) to date in Support Questions

NIFI Infer Schema trying to convert string(alpha) to date

Althotta — Fri, 23 Sep 2022 04:04:00 GMT

While converting the input flow file JSON to any other format using query record (CSVwriter or AVRO Writer) using inferschema strategy the NIFI processor is trying to convert to Date based on first few characters of the incoming string.

NIFI Error Output

Successfully parsed a JSON object from input but failed to convert into a Record object with the given schema\n- Caused by: org.apache.nifi.serialization.MalformedRecordException: Successfully parsed a JSON object from input but failed to convert into a Record object with the given schema\n- Caused by: org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Failed Conversion of Field [sharepoint_documents_name__c] from String [10-7-20 2x10GE LAN PHY XXXXXXXXXXXX] to LocalDate\n- Caused by: java.time.format.DateTimeParseException: Text '10-7-20 2x10GE LAN PHY XXXXXXXXXXX' could not be parsed at index 0:

Any workaround/fixes available in the newer version. Iam using NIFI 16.1

Re: NIFI Infer Schema trying to convert string(alpha) to date

araujo — Sun, 09 Oct 2022 23:01:26 GMT

@Althotta ,

I tested this on 1.16.2 and the behaviour you described doesn't happen to me.

Would you be able to share you flow and processor/controller services configuration?

Cheers,

André

Re: NIFI Infer Schema trying to convert string(alpha) to date

VidyaSargur — Tue, 18 Oct 2022 14:44:34 GMT

@Althotta, If you are still experiencing the issue, can you provide the information @araujo has requested?

Re: NIFI Infer Schema trying to convert string(alpha) to date

Yashar — Tue, 15 Nov 2022 11:21:16 GMT

@araujo i'm also facing the similar issue with 1.16.3 and 1.16.2, i'm attaching the referenced images

Running NIFI Using docker in Mac M1

JSON input data

[ { "full_name" : "Anatolij Finkbeiner", "gender" : "male", "email" : "anatolij.finkbeiner@example.com", "phone" : "0166-2058423", "dob" : "2000-09-11T15:40:10.890Z", "uuid" : "747f27a8-88c0-452e-b360-3ddf5ee67211", "address" : "Germany, Schleswig-Holstein, Guben", "createdAt" : "2022-11-15T11:09:35.912Z" }, { "full_name" : "Lucia Soto", "gender" : "female", "email" : "lucia.soto@example.com", "phone" : "946-359-112", "dob" : "1987-09-15T09:58:34.821Z", "uuid" : "8edb62cd-0bdc-41ee-8806-fecfd6958e33", "address" : "Spain, Navarra, Santiago de Compostela", "createdAt" : "2022-11-15T11:09:35.911Z" } ]

Error

JSON Tree READER

Avro Schema

{ "type": "record", "namespace": "nifi", "name": "userSchema", "fields": [ { "name": "dob", "type": { "type": "string", "logicalType": "timestamp" } }, { "name": "createdAt", "type": { "type": "long", "logicalType": "timestamp" } }, { "name": "full_name", "type": "string" }, { "name": "gender", "type": "string" }, { "name": "email", "type": "string" }, { "name": "address", "type": "string" }, { "name": "uuid", "type": "string" }, { "name": "phone", "type": "string" } ] }

Avro Record set writer

Avro reader

Re: NIFI Infer Schema trying to convert string(alpha) to date

Yashar — Tue, 15 Nov 2022 11:32:01 GMT

@araujo I have faced the similar issue with Apache/Nifi 1.16.3 and 1.16.2, attaching

Avro Schema

the processor/controller services configuration Convert Record ProcessorError ImageAvro Reader ConfigurationAvro Record set writerJSON Tree READER

Re: NIFI Infer Schema trying to convert string(alpha) to date

Yashar — Wed, 16 Nov 2022 07:19:47 GMT

I'm able to fix this .. In JSON Tree READER the configuration for `Schema Access Strategy` should be "Infer Schema"
Input date format will be "yyyy-MM-dd HH:mm:ss.SSS'Z'"
avro schema for the same will be

{ "name": "createdAt", "type": { "type": "long", "logicalType": "timestamp-millis" } }