Hello, I have the following setup: a ConsumeKafka processor that receives a protobuf parsed file, then sends to a ExecuteScript in python that parses the protobuf and finally sends to a putfile processor.
My problem is that I'm receiving the 'Truncated Message' error when the message have a nested Json, I don't receive any error when parsing a flat json.
The error occurs at the command: register.ParseFromString(buf)
Thanks in advance
def parser(buf) : import register_pb2 try : register = register_pb2.Register() register.ParseFromString(buf) ... class PyStreamCallback(StreamCallback): def __init__(self): pass def process(self, inputStream, outputStream): text = IOUtils.toByteArray(inputStream) newObj = parser(inputStream) outputStream.write(bytearray(json.dumps(newObj, indent=4).encode('utf-8'))) flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile,PyStreamCallback()) session.transfer(flowFile, REL_SUCCESS)
I'm not familiar with the register_pb2 module, is that a custom module? You are calling ParseFromString but passing in a byte array, so maybe there's an encoding issue? In any case, that error is coming from protobuf code so it is likely not getting something it is expecting.
Hello @Matt Burgess, first I'd like to thank you for all your posts on your blog. They are really useful.
Yes, register_pb2 module is a python compiled custom proto file, I've tried using IOUtils.toString too but it not works too.