Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI Malformed Data.Length is negative

avatar
Explorer

I am new to NIFI learning how each processor works....

I am trying to generate CSV file extracting some from Database, following are sequence of processors i am using to achive this

ExecuteSQL -> Execute a simple query (select empno,name,deptno,age,sal from emp limit 1 ) -- Working perfectly fine

UpdateAttribute -> Added a new property "schema.name"= emp -- Working perfectly fine

Convert Record -> Getting Error here

RecordReader - AVROReader

RecordWriter - CSVRecordSetWriter

AVROSchemaRegistry:

{ "name":"emp", "namespace":"nifi", "type":"record", "fields": [ {"name": "empno" , "type":"int"}, {"name": "name", "type":"string"}, {"name": "deptno", "type":"int"}, {"name": "age", "type":"int"}, {"name": "sal", "type":"float"} ] }

Data In Queue before failure processor:

[ { "empno" : 10, "name" : "abcadefshnz", "deptno" : 100, "age" : 20, "sal" : 12700.5 } ]

ErrorMessage :

failed to process session due to org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51; Processor Administratively Yielded for 1 sec: org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51 org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51

Thanks in advance...

1 ACCEPTED SOLUTION

avatar
Master Guru
@Bal P

Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and

Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services

Avro Reader:

78468-schema-access.png

CsvSetwriter:

78469-csvsetwriter.png

With the above configs i'm able to get the output flowfile in csv format as shown below.

empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5

View solution in original post

5 REPLIES 5

avatar
Master Guru

Please show the configuration of your AvroReader, specifically how the schema access strategy is configured.

avatar
Explorer

Here are the screenshots avroreader.jpgcsvrecordsetwriter.jpg

avatar
Master Guru
@Bal P

Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and

Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services

Avro Reader:

78468-schema-access.png

CsvSetwriter:

78469-csvsetwriter.png

With the above configs i'm able to get the output flowfile in csv format as shown below.

empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5

avatar
Explorer

Thanks Shu, it is working fine, does this mean SchemaRegistry setup is not required ??

avatar
Master Guru
@Bal P

For avro data file schema will be embedded and you want to write all the fields in CSV format so we don't need to setup the registry. if you are writing only specific columns not all and for other formats JSON.. then schema registry is required