Support Questions
Find answers, ask questions, and share your expertise

Customize default in .avsc file generated by sqoop import

Highlighted

Customize default in .avsc file generated by sqoop import

New Contributor

Is there anyway to customize the value for the "default" element in the .avsc file generated by sqoop import with--as-avrodatafile?


For e.g.,

I get:

{

"type": "record",

"name": "myTest",

"doc": "Sqoop import of myTest",

"fields": [{

"name": "User_UserId",

"type": ["null", "string"],

"default": null,

"columnName": "User_UserId",

"sqlType": "1"

}, {

"name": "newcolumn",

"type": ["null", "int"],

"default": null,

"columnName": "newcolumn",

"sqlType": "4"

}

],

"tableName": "myTest"

}


I want:

{

"type": "record",

"name": "myTest",

"doc": "Sqoop import of myTest",

"fields": [{

"name": "User_UserId",

"type": ["null", "string"],

"default": null,

"columnName": "User_UserId",

"sqlType": "1"

}, {

"name": "newcolumn",

"type": ["int","null"],

"default": -999,

"columnName": "newcolumn",

"sqlType": "4"

}

],

"tableName": "myTest"

}


I am looking for perhaps, a parameter I can pass. For instance, for "int" fields use default of -999.

I know an alternative is to create your own custom java class. Or, create a script to alter the .avsc file post sqoop import.


Thank you