Using the Schema Registry API

[#1]

In this article I am going to explain how to use the Schema Registry API to Create and Delete a Schema Entity.  

I am working on a Use Case for creating schemas with over 500 columns on the fly within a NiFi Data Flow.  In order to complete this task I needed to figure out how to create, and update the schemas from outside of the Schema Registry UI.   While testing I also needed to delete the test schemas since there is no way to delete them from within the UI.

First, to get the artifacts I need (API urls, and post formats), I use the Schema Registry UI in Chrome with my Developer Tools open to capture what happens when the UI buttons are clicked.  When I add a test schema, I notice 2 POST calls.  

1. Creates the Schema Entity (without an actual Schema)
2. Posts the Schema to the newly created Entity

Using the information in Developer Tools, I then create 2 Postman api calls to duplicate, debug, and validate they will work externally.  

** Note: I am working in a single node sandbox (hdf.cloudera.com) hdf cluster without any authorization. If your cluster is secured you will need to satisfy access, authorization, and ssl requirements.

Create Schema Entity

POST: http://hdf.cloudera.com:7788/api/v1/schemaregistry/schemas 

BODY:

{"name":"test","type":"avro","schemaGroup":"test","description":"Testing creating a schema from API","evolve":true,"compatibility":"BACKWARD"}

Add Schema To Entity

POST: http://hdf.cloudera.com:7788/api/v1/schemaregistry/schemas/test/versions?branch=MASTER

BODY:

{"schemaText":"{ \"name\" : \"test\", \"type\" : \"record\", \"fields\" : [ { \"name\" : \"column0\", \"type\" : \"string\" }, { \"name\" : \"column1\", \"type\" : \"string\" }, { \"name\" : \"column2\", \"type\" : \"string\" } ]}","description":"Testing creating a schema from API"}

** Note: the string escaped format above is required. If your schema has return lines those should also be replaced with \n.

Delete Schema

DELETE: http://hdf.cloudera.com:7788/api/v1/schemaregistry/schemas/test/

** thanks to @mvalleavila on [ This Thead ] for exposing the simple Delete.

The information above should be enough to get you started using the Schema Registry API via command line, curl, or any other method you prefer.  In my next article, I will go into deeper detail in how I fit these concepts into my NiFi Flow.

[#2]

In working with some advanced features (enable/disable compatibility) I was able to find the Swagger API link for my Schema Registry.  The link should look like:

http://hdf.cloudera.com:7788/swagger/#/

Simply add /swagger/ after your Schema UI port and it will take you there.