Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI Malformed Data.Length is negative

Solved Go to solution

NIFI Malformed Data.Length is negative

Explorer

I am new to NIFI learning how each processor works....

I am trying to generate CSV file extracting some from Database, following are sequence of processors i am using to achive this

ExecuteSQL -> Execute a simple query (select empno,name,deptno,age,sal from emp limit 1 ) -- Working perfectly fine

UpdateAttribute -> Added a new property "schema.name"= emp -- Working perfectly fine

Convert Record -> Getting Error here

RecordReader - AVROReader

RecordWriter - CSVRecordSetWriter

AVROSchemaRegistry:

{ "name":"emp", "namespace":"nifi", "type":"record", "fields": [ {"name": "empno" , "type":"int"}, {"name": "name", "type":"string"}, {"name": "deptno", "type":"int"}, {"name": "age", "type":"int"}, {"name": "sal", "type":"float"} ] }

Data In Queue before failure processor:

[ { "empno" : 10, "name" : "abcadefshnz", "deptno" : 100, "age" : 20, "sal" : 12700.5 } ]

ErrorMessage :

failed to process session due to org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51; Processor Administratively Yielded for 1 sec: org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51 org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51

Thanks in advance...

1 ACCEPTED SOLUTION

Accepted Solutions

Re: NIFI Malformed Data.Length is negative

Super Guru
@Bal P

Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and

Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services

Avro Reader:

78468-schema-access.png

CsvSetwriter:

78469-csvsetwriter.png

With the above configs i'm able to get the output flowfile in csv format as shown below.

empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5
5 REPLIES 5

Re: NIFI Malformed Data.Length is negative

Please show the configuration of your AvroReader, specifically how the schema access strategy is configured.

Re: NIFI Malformed Data.Length is negative

Explorer

Here are the screenshots avroreader.jpgcsvrecordsetwriter.jpg

Re: NIFI Malformed Data.Length is negative

Super Guru
@Bal P

Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and

Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services

Avro Reader:

78468-schema-access.png

CsvSetwriter:

78469-csvsetwriter.png

With the above configs i'm able to get the output flowfile in csv format as shown below.

empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5

Re: NIFI Malformed Data.Length is negative

Explorer

Thanks Shu, it is working fine, does this mean SchemaRegistry setup is not required ??

Re: NIFI Malformed Data.Length is negative

Super Guru
@Bal P

For avro data file schema will be embedded and you want to write all the fields in CSV format so we don't need to setup the registry. if you are writing only specific columns not all and for other formats JSON.. then schema registry is required

Don't have an account?
Coming from Hortonworks? Activate your account here