Member since
04-21-2016
13
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1902 | 10-12-2016 04:28 PM | |
1451 | 08-17-2016 12:42 PM |
11-29-2016
11:29 AM
Does anyone else have this requirement? We have a set of automated processes that ingest data from all over the place (on windows) and need a way to reliably load that data in to HDFS What is the best way to do this? NFS gateway (lack of ACL is a concern here)? HTTPFS? Cluster is kerberized and CentOS based, if that helps. Any insights/tips would be very helpful. Thanks.
... View more
Labels:
- Labels:
-
HDFS
10-12-2016
04:28 PM
The latter. Active NN will write every edit to all available journal nodes. A majority is confirm is required before the transaction is committed - that is, 2 out of your 3 journal nodes will have to have accepted the write. In the case of the standby NN reading from the journal nodes, it will again look for a majority and discard the discrepancy. I think if one out of three JN are down, and the final two don't agree, it will accept the most up to date JN's answer. But I could be completely wrong on this.
... View more
08-24-2016
10:33 AM
Thank you. I guess I haven't visited that URL in a while so I mis-attributed it to the upgrade.
... View more
08-24-2016
10:16 AM
Using CM 5.8.1 and CDH 5.8.0 The Spark History WebUI link automatically goes to https://HOSTNAME:18088 and chrome immediately reports ERR_SSL_PROTOCOL_ERROR Attempting to manually hit http://HOSTNAME:18088 automatically pushes the connection to https and reports the same error. There is still no configuration for spark history server SSL. Tried the configuration here: http://spark.apache.org/docs/latest/configuration.html#encryption and placing it in the advanced configuration snippet, but that just leads to a set of new errors. Anyone else encounter this? It was working fine on CM/CDH 5.7.1
... View more
Labels:
08-17-2016
12:42 PM
HAProxy is extremely lightweight. You should put it somewhere that is highly available or use two in a HA configuration.
... View more
04-21-2016
06:05 PM
CDH version 5.7 deployed using Cloudera Manager.
Environment is Kerberized with TLS/SSL where applicable.
Solr logs have this as INFO every minute or so:
Could not consume full client request
java.io.IOException: Stream closed
at org.apache.catalina.connector.InputBuffer.readByte(InputBuffer.java:315)
at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:105)
at org.apache.solr.servlet.SolrDispatchFilter.consumeInput(SolrDispatchFilter.java:916)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:216)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.solr.servlet.SolrHadoopAuthenticationFilter$2.doFilter(SolrHadoopAuthenticationFilter.java:394)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
at org.apache.solr.servlet.SolrHadoopAuthenticationFilter.doFilter(SolrHadoopAuthenticationFilter.java:399)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.solr.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
[admin] webapp=null path=/admin/cores params={action=STATUS&wt=json} status=0 QTime=0
... View more