Cloudera Labs
Provide feedback on Cloudera Labs
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CLoudera Envelope : sample config for json with avro schema

SOLVED Go to solution

CLoudera Envelope : sample config for json with avro schema

New Contributor

Could you please provide a sample config for specifying schema in avro format for reading json input file for Cloudera Envelope

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: CLoudera Envelope : sample config for json with avro schema

Rising Star

Hi akumar,

 

This is a simple example:

 

steps {
  read_json {
    input {
      type = filesystem
      format = json
      avro-schema.literal = """
        {
          "type": "record",
          "name": "for_json",
          "fields" : [
            {"name": "field1", "type": "string"},
            {"name": "field2", "type": "int"}
          ]
        }"""
      path = test.json
    }
    print.data.enabled = true
  }
}

But this may not be necessary -- if you don't provide a schema the filesystem input will try to infer the schema from the JSON data you provide.

 

Jeremy

1 REPLY 1
Highlighted

Re: CLoudera Envelope : sample config for json with avro schema

Rising Star

Hi akumar,

 

This is a simple example:

 

steps {
  read_json {
    input {
      type = filesystem
      format = json
      avro-schema.literal = """
        {
          "type": "record",
          "name": "for_json",
          "fields" : [
            {"name": "field1", "type": "string"},
            {"name": "field2", "type": "int"}
          ]
        }"""
      path = test.json
    }
    print.data.enabled = true
  }
}

But this may not be necessary -- if you don't provide a schema the filesystem input will try to infer the schema from the JSON data you provide.

 

Jeremy