Reply
Champion Alumni
Posts: 157
Registered: ‎02-11-2014

Avoiding Key Value Literals in Map reduce out put

Hello,

My map reduce program currently outputs  avro data in the following format.Is there a way I can avoid the key and literals in the output?

 

{
"type" : "record",
"name" : "KeyValuePair",
"namespace" : "org.apache.avro.mapreduce",
"doc" : "A key/value pair",
"fields" : [ {
"name" : "key",
"type" : "null",
"doc" : "The key"
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "RecordType",
"namespace" : "model",
"fields" : [ {  ]} }

Champion Alumni
Posts: 157
Registered: ‎02-11-2014

Re: Avoiding Key Value Literals in Map reduce out put

If you want container  files rather than   key value pairs as part of map reduce output you  should use https://avro.apache.org/docs/1.7.6/api/java/org/apache/avro/mapred/AvroOutputFormat.html  than KeyValueOutputFormat

Announcements
The Kite SDK is a collection of docs, sample code, APIs, and tools to make Hadoop application development faster. Learn more at http://kitesdk.org.