Reply
Contributor
Posts: 55
Registered: ‎09-17-2013

Is there a way to prepare HashMap in InputFormat and share it with Reducer

[ Edited ]

Hi, 

 

I have written custom RecordReader and Custom InputFormat. I will have date in file name and i can collect that at RecordReader.

 

Now, i have a requirement to prepare HashMap with all dates, and need to use it while partitioning.

 

Is there any possibility to create HashMap in InputFormat and share it with context.?

 

Thank You.

Highlighted
Posts: 1,896
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: Is there a way to prepare in InputFormat and share it with Reducer

There is nothing as straight-forward, but there are alternative approaches
you can think of - such as if you know the input beforehand, you can
pre-prepare this information and send it as a cache-file to the reducer.

Announcements

Our community is getting a little larger. And a lot better.


Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.