SAM error: com.hortonworks.registries.schemaregistry.serde.SerDesException: Unknown protocol id [123] received while deserializing the payload at com.hortonworks.registries.schemaregistry.serdes.avro.AvroSnapsh

[#1]

Simple SAM flow: Kafka -> (Storm) filter -> Kafka

Fails at Storm, which reports: com.hortonworks.registries.schemaregistry.serde.SerDesException: Unknown protocol id [123] received while deserializing the payload at com.hortonworks.registries.schemaregistry.serdes.avro.AvroSnapsh

Wondering what could cause this. (Schema seems properly configured)

[#2]

I'm curious to know how the Avro data was serialized.

I suspect you're experiencing the same issue as me (see https://community.hortonworks.com/questions/114646/sam-application-unknown-protocol-id-12-received-w...) and possibly @Brad Penelli (see https://community.hortonworks.com/questions/114758/sam-application-kafka-source-fails.html).

[#3]

1. Make sure NiFi and SAM are the same version HDF

2. Make sure kafka message is converted properly to Avro.

3. Restart Kafka, SAM, NiFi

4. Make sure all the settings are proper.

[#4]

Make sure your queue is empty, old bad messages might be sent.

I would create a new empty queue add one checked valid message (check your avro message with avro tools) and send that.. No nulls!

[#5 · Accepted solution]

@Timothy Spann @Greg Keys

This is a known issue with HDF-3.0.1 release and we fixed did a release HDF-3.0.1.1. Please find the details here

http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDF3/HDF-3.0.1.1/bk_release-notes/content/c....

1. Take a backup of existing SAM app. You can export the app to your local computer.

2. Upgrade the cluster to 3.0.1.1

3. If you have existing data in a Kafka topic that was ingested with the above bug. Make sure you create a new topic or delete the existing topic.

4. Import the SAM topology, make sure you configure the topic and other cluster details in the app.

[#6]

Followed instructions and it worked. Thanks @Sriharsha Chintalapani

[#7]

@Greg Keys I am having the same issue with hdf 3.3.1. I have checked the schema file and the input file as well.
I have done what was mentioned by @Sriharsha Chintalapani
Schema

{
    "namespace": "hdf.heaptrace.com",
    "type": "record",
    "name": "PatientField",
    "fields": [
    {"name": "Patient_name","type": "string"}
  ]
}

JSON data

{"Patient_name":"john"}

Please help !!!

I have converted data from json to avro and then back again as well using avro tools.

[#8]

@Sriharsha Chintalapani I getting the same error, cluster update didn't solve the issue.

I'm trying to consume plain CSV(roshan,22) to Streamline via NIFI(to convert CSV to AVRO). Kafka > NIFI > Kafka. From Storm I'm getting the error similar to above.

com.hortonworks.registries.schemaregistry.serde.SerDesException: Unknown protocol id [114] received while deserializing the payload at com.hortonworks.registries.schemaregistry.serdes.avro.AvroSnapsh

My Flow is as follows, One branch directly to a Kafka topic and other one to serialize and publish to kafka. After running data through the PublishKafkaRecord the output simply removes the "," (roshan,22 turns to roshan22) and the above mentioned error appears in Storm. I'm really new to this stack any help would be appreciated.

Avro schema

{
 "type": "record",
 "name": "user",
 "fields": [
  {
   "name": "name",
   "type": "string",
   "default": null
  },
  {
   "name": "age",
   "type": "string",
   "default": null
  }
 ]
}

Update Attribute Processor

Publish Kafka Record Processor

CSVReader Controller Service

AvroRecordSetWriter Controller Service

[#9]

Have you resolved your issue ?? How did you do it ??