Validating avro schema and json file

[#1]

There are many ways to validate a json file against a avro schema to verify all is kosher. Sharing a practice I have been using for few years.

Objective - Validate avro schema well bound to the json file

First you must have a avro schema and json file. From there download the latest a avro-tools jar. At the moment 1.8.1 is the latest avro-tools version jar available.

Store the avro schema and json file in the same directory. Issue a wget to fetch the avro-tools jar

wget http://www.us.apache.org/dist/avro/avro-1.8.1/java/avro-tools-1.8.1.jar

Here is what the directory looks like

Objective Details - Validate avro schema student.avsc binds to student.json

How - Issue the following

java -jar ./avro-tools-1.8.1.jar fromjson --schema-file YourSchemaFile.avsc YourJsonFile.json > AnyNameForYourBinaryAvro.avro

Using the student files example:

java -jar ./avro-tools-1.8.1.jar fromjson --schema-file student.avsc student.json > student.avro

Validation passed, a avro binary was created.

Now as a last step lets break something. Another avro schema (student2.avsc) is created which does not conform to student.json. Lets verify the avro-tools jar will fails to build a avro binary

As you can see from above output the avro binary failed to create due to validation errors