Support Questions
Find answers, ask questions, and share your expertise

NIFI Malformed Data.Length is negative

Solved Go to solution
Highlighted

NIFI Malformed Data.Length is negative

Explorer

I am new to NIFI learning how each processor works....

I am trying to generate CSV file extracting some from Database, following are sequence of processors i am using to achive this

ExecuteSQL -> Execute a simple query (select empno,name,deptno,age,sal from emp limit 1 ) -- Working perfectly fine

UpdateAttribute -> Added a new property "schema.name"= emp -- Working perfectly fine

Convert Record -> Getting Error here

RecordReader - AVROReader

RecordWriter - CSVRecordSetWriter

AVROSchemaRegistry:

{ "name":"emp", "namespace":"nifi", "type":"record", "fields": [ {"name": "empno" , "type":"int"}, {"name": "name", "type":"string"}, {"name": "deptno", "type":"int"}, {"name": "age", "type":"int"}, {"name": "sal", "type":"float"} ] }

Data In Queue before failure processor:

[ { "empno" : 10, "name" : "abcadefshnz", "deptno" : 100, "age" : 20, "sal" : 12700.5 } ]

ErrorMessage :

failed to process session due to org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51; Processor Administratively Yielded for 1 sec: org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51 org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -51

Thanks in advance...

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NIFI Malformed Data.Length is negative

Super Guru
@Bal P

Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and

Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services

Avro Reader:

78468-schema-access.png

CsvSetwriter:

78469-csvsetwriter.png

With the above configs i'm able to get the output flowfile in csv format as shown below.

empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5

View solution in original post

5 REPLIES 5
Highlighted

Re: NIFI Malformed Data.Length is negative

Please show the configuration of your AvroReader, specifically how the schema access strategy is configured.

Highlighted

Re: NIFI Malformed Data.Length is negative

Explorer

Here are the screenshots avroreader.jpgcsvrecordsetwriter.jpg

Highlighted

Re: NIFI Malformed Data.Length is negative

Super Guru
@Bal P

Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and

Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services

Avro Reader:

78468-schema-access.png

CsvSetwriter:

78469-csvsetwriter.png

With the above configs i'm able to get the output flowfile in csv format as shown below.

empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5

View solution in original post

Highlighted

Re: NIFI Malformed Data.Length is negative

Explorer

Thanks Shu, it is working fine, does this mean SchemaRegistry setup is not required ??

Highlighted

Re: NIFI Malformed Data.Length is negative

Super Guru
@Bal P

For avro data file schema will be embedded and you want to write all the fields in CSV format so we don't need to setup the registry. if you are writing only specific columns not all and for other formats JSON.. then schema registry is required