Support Questions

sai23 · ‎07-05-2018

Hi in my cluster knox gateway was down.i found one issue in knox-gc.log.

Please find the below error

2018-07-05T01:55:29.588+0000: 314055.583: [GC (Allocation Failure) 2018-07-05T01:55:29.887+0000: 314055.882: [ParNew: 2532112K->10894K(2831168K), 0.0440397 secs] 2532112K->10894K(2837312K), 0.3432294 secs] [Times: user=0.03 sys=0.00, real=0.35 secs]

Please help me how to solve

falbani · ‎07-05-2018

@kanna k

Multiple occurrences of GC (Allocation Failure) could lead to OutOfMemory but if you only see single one that isn't probably the problem. If you still think this could be due memory, please review the following HC article on how to calculate and set appropriate memory for Knox:

https://community.hortonworks.com/articles/105860/how-to-calculate-and-change-jvm-heap-memory-settin...

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

View solution in original post

falbani · ‎07-05-2018

@kanna k

Multiple occurrences of GC (Allocation Failure) could lead to OutOfMemory but if you only see single one that isn't probably the problem. If you still think this could be due memory, please review the following HC article on how to calculate and set appropriate memory for Knox:

https://community.hortonworks.com/articles/105860/how-to-calculate-and-change-jvm-heap-memory-settin...

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

sai23 · ‎07-10-2018

@Felix Albani @Sindhu @Jay Kumar SenSharma @Geoffrey Shelton OkotHi,

knox gateway is going down on daily basis.i found only knox-gc.log with toaday's time stamp in /var/log/knox/ directory.

if you think that it is due to allocation failure can you please tell me how much memory i need to allocate for knox gateway.

in my gateway.sh file i found the below line.do i need to change the values here or any other place and please let me know how much memory i should give to resolve this issue.

APP_MEM_OPTS="-Xmx5g -XX:NewSize=3G -XX:MaxNewSize=3G -verbose:gc -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -Xloggc:/var/log/knox/knox-gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps"

Thanks in advance

falbani · ‎07-10-2018

@kanna k

So based on the above logs you added I see

2018-07-10 09:55:07,592 ERROR hadoop.gateway (DefaultTopologyService.java:loadTopologies(252)) - Failed to load topology /usr/hdp/2.6.1.0-129/knox/bin/../conf/topologies/sample.xml: org.xml.sax.SAXParseException; lineNumber: 41; columnNumber: 76; The reference to entity "ServiceAccounts" must end with the ';' delimiter.

This error points to a sample.xml topology file which has errors. Perhaps you can move it out from topologies directory or fix the problem in the before mentioned line.

As far as memory I already provided the link to calculate the memory for Knox in previous post. But as I mentioned, unless you are seeing OOM exceptions I don't think you are having memory problems. I would suggest you check the logs and see what is causing the knox server to go down.

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

sai23 · ‎07-10-2018

i found below information in gateway.log

can any one help me to resolve this...

2018-07-10 09:55:07,535 ERROR hadoop.gateway (DefaultTopologyService.java:loadTopology(111)) - Failed to load topology /usr/hdp/2.6.1.0-129/knox/bin/../conf/topologies/sample.xml, retrying after 50ms: org.xml.sax.SAXParseException; lineNumber: 41; columnNumber: 76; The reference to entity "ServiceAccounts" must end with the ';' delimiter.

2018-07-10 09:55:07,588 ERROR digester3.Digester (Digester.java:fatalError(1541)) - Parse Fatal Error at line 41 column 76: The reference to entity "ServiceAccounts" must end with the ';' delimiter.

org.xml.sax.SAXParseException; lineNumber: 41; columnNumber: 76; The reference to entity "ServiceAccounts" must end with the ';' delimiter.

at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)

at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)

at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1437)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEntityReference(XMLDocumentFragmentScannerImpl.java:1850)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3067)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)

at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)

at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649)

at org.apache.commons.digester3.Digester.parse(Digester.java:1642)

at org.apache.commons.digester3.Digester.parse(Digester.java:1701)

at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.loadTopologyAttempt(DefaultTopologyService.java:124)

at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.loadTopology(DefaultTopologyService.java:100)

at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.loadTopologies(DefaultTopologyService.java:235)

at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.reloadTopologies(DefaultTopologyService.java:320)

at org.apache.hadoop.gateway.GatewayServer.start(GatewayServer.java:422)

at org.apache.hadoop.gateway.GatewayServer.startGateway(GatewayServer.java:295)

at org.apache.hadoop.gateway.GatewayServer.main(GatewayServer.java:148)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)

at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)

at org.apache.hadoop.gateway.launcher.Command.run(Command.java:99)

at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)

at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)

2018-07-10 09:55:07,589 ERROR digester3.Digester (Digester.java:parse(1652)) - An error occurred while parsing XML from '(already loaded from stream)', see nested exceptions

org.xml.sax.SAXParseException; lineNumber: 41; columnNumber: 76; The reference to entity "ServiceAccounts" must end with the ';' delimiter.

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)