Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hadop seems to use a wrong xml parser

hadop seems to use a wrong xml parser

New Contributor

I have an existing application which works quite fine. If I add a specific jar (no code changes yet) mappers don't start because there is a probem reading the configuration:

2018-09-06 15:26:12,221 [main] ERROR org.apache.hadoop.conf.Configuration - error parsing conf core-default.xml
javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
	at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2570)
	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2532)
	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2424)
	at org.apache.hadoop.conf.Configuration.get(Configuration.java:1238)
	at org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider.getRecordFactory(RecordFactoryProvider.java:49)
	at org.apache.hadoop.yarn.util.Records.<clinit>(Records.java:32)
	at org.apache.hadoop.yarn.api.records.ApplicationId.newInstance(ApplicationId.java:49)
	at org.apache.hadoop.yarn.api.records.ContainerId.toApplicationAttemptId(ContainerId.java:249)
	at org.apache.hadoop.yarn.api.records.ContainerId.toApplicationAttemptId(ContainerId.java:244)
	at org.apache.hadoop.yarn.api.records.ContainerId.fromString(ContainerId.java:223)
	at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:191)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1463)

It looks as if a wrong xml parser is used here.

Intererstingly, running the application local in eclipse works just fine.

I tried to debug my application. In the driver I didn't see any problems, and the mapper didn't reach my first breakpoint (first line in the main method of YarnChild).

Looking at the stack trace, from where is MRAppMaster called (started)?

How could I get hadoop using the correct xml parser?

I added

javax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

all places I was thinking of:

in my driver class,

in mapreduce.map.java.opts,

in YARN_CLIENT_OPTS,

as parameter when calling "yarn jar"

How could I solve this?

Thanks in advance

Don't have an account?
Coming from Hortonworks? Activate your account here