Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Solved Go to solution

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Mentor

@John Smith yes I followed your efforts, can you live with the work around? I honestly don't have cycles to investigate further for you. I think I learned something myself thanks to you :). I'd say open an article on HCC with the proposed workaround and your desired goal, maybe someone else can weigh in. Great job John!

Highlighted

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Expert Contributor

well i cant live with that workaround, thats the problem. what i HCC?

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Mentor

this website is called Hortonworks Community Connection, HCC for short. Again, post this as a separate issue with tags for Avro and Pig. @John Smith

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Expert Contributor

is there any update on this?

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Expert Contributor

one more important observation, when i dump data into avro using

store outputSet into 'avrostorage' using AvroStorage();

the schema inside avro file looks like:

{"type":"record","name":"pig_output","fields":[{"name":"name","type":["null","string"]},{"name":"customerId","type":["null","string"]},{"name":"VIN","type":["null","string"]},{"name":"Birthdate","type":["null","string"]},{"name":"Mileage","type":["null","string"]},{"name":"Fuel_Consumption","type":["null","string"]}]}

Why each field contains null?

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Mentor

@John Smith it means the field can be null if missing, an optional field that is. That way if you don't pass a field it won't complain.

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Expert Contributor

sure but input data contains all the field, so my question is why it generates [null] as part of the datatype.

Also still no luck with

https://issues.apache.org/jira/browse/PIG-4793

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Mentor

@John Smith read avro docs for explanation of optionalvs default fields.

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Expert Contributor

ah i already did ... my question was why its there ... when i use local mode its not there .. anyway there is no reply from anyone behind avrostorage... thats pretty odd.

Re: AvroStorage with mapreduce and java.lang.RuntimeException: could not instantiate

Mentor

@John Smith its a better practice so that if you do happen to get a null at least it won't bomb. As far as jira, that's open source, individual contributors also need earn a living and if there's higher responsibilities then they'll get to it when queue is clear. I wouldn't get your hopes up and identify alternative ways. Shoot an email to the avro mailing list. They may help faster.

Don't have an account?
Coming from Hortonworks? Activate your account here