Member since
01-15-2018
93
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
589 | 09-13-2018 02:23 PM |
03-31-2020
05:39 AM
2 Kudos
What happened?
Starting up a ZooKeeper server in a Kerberized CDP-DC 7.0.3 cluster failed with the logs below.
2020-03-30 12:23:10,251 ERROR org.apache.zookeeper.server.quorum.QuorumPeerMain: Unexpected exception, exiting abnormally java.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: Message stream modified (41) at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:243) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:148) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)
The JDK for this environment is OpenJDK 1.8.0_242.
# java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
Solution
Removing the line of renew_lifetime in /etc/krb5.conf.
Removing this line means to use the default value, 0, for renew_lifetime.
Thus, it may also need to specify renew_lifetime when running kinit command.
See also
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201909.mbox/%3CCAKRKJ1O3yrYKDZ=WhU=i6A+zqxFnUidxvwQzNCTW0mnEv2WFPA@mail.gmail.com%3E
On this page, Akira Ajisaka, one of Hadoop PMCs, described the solution.
He also mentioned a related OpenJDK's JIRA ticket.
https://bugs.centos.org/view.php?id=17000
This page also introduced the same solution.
Additionally, this page showed another solution, setting sun.security.krb5.disableReferrals=true in java.security file. But in my case, this solution didn't work.
https://my.cloudera.com/knowledge/Cloudera-Customer-Advisory-Servers-with-Kerberos-enabled-stop?id=292027
This is a related article from the Cloudera Knowledge Base.
It also describes sun.security.krb5.disableReferrals=true as its workaround.
... View more
Labels:
11-08-2018
12:37 AM
I wrote "restarting Ambari Server" but that does not mean "restarting a host that Ambari Server is installed". Just restarting "Ambari Server" takes effect. (= ambari-server restart)
... View more
11-06-2018
12:11 AM
Unfortunately, I haven't got a solution yet.
... View more
10-18-2018
09:49 AM
Thanks, @Jay Kumar SenSharma
... View more
10-17-2018
01:42 AM
How do I restrict access to Ambari WebUI by IP address? Of course, using a firewall included in OS is a solution, but I'd like to know the way which requires only to modify Ambari's configurations. I know Ambari uses Jetty for HTTP server and Jetty provides IP address restriction by IPAccessHandler https://www.eclipse.org/jetty/documentation/9.4.x/ipaccess-handler.html, but I'm not sure how to apply this to Ambari.
... View more
Labels:
- Labels:
-
Apache Ambari
10-09-2018
07:51 AM
Thanks, @Jay Kumar SenSharma!
... View more
10-09-2018
06:11 AM
I'm using Ambari 2.6.2.2 and Ambari 2.6.2.2 has "Service Auto Start Configuration" that enables a component restarting when it went down unexpectedly. However, I could not find an automatic "Restart" operation in the operations history when the automatic restart functionality worked. How can I get to know that the automatic component restart happened?
... View more
- Tags:
- ambari-server
Labels:
- Labels:
-
Apache Ambari
10-05-2018
07:30 AM
Hi, Please try adding <service>
<role>ATLAS-API</role>
<url>http://${atlas_metadata_server_host}:${atlas_metadata_server_port}/</url>
</service> (e.g. <service>
<role>ATLAS-API</role>
<url>http://toide-2.toide.hortonworks.com:21000/</url>
</service> ) to your "Advanced topology". I met "Something went wrong" error on Atlas WebUI before adding it.
... View more
09-28-2018
02:48 AM
Thanks @Sharmadha Sainath
... View more
09-28-2018
02:24 AM
Thanks @akulkarni
... View more
09-27-2018
06:34 AM
How do I delete an Atlas tag which is associated with entities those are already deleted? I found that I couldn't delete an Atlas tag named "mytag" and when I tried to delete that tag I saw "Given type mytag has references". However, "mytag" was not associated with any entities. Then I noticed "Show historical entities" checkbox and I checked the checkbox, then an entity already deleted appeared. I thought I could delete "mytag" after deleting the association between "mytag" and that entity, but I couldn't find the way to remove that association.
... View more
Labels:
- Labels:
-
Apache Atlas
09-25-2018
05:51 AM
How do I reload the file (specified by "FilePath" attribute in enricherOptions) or RangerFileBasedGeolocationProvider? As far as I tried, that file is not automatically reloaded. Restarting HiveServer2 if the component using RangerFileBasedGeolocationProvider is Hive causes reloading the file. Doing "Save" a policy also causes reloading the file, no modification required. This is very inconvenient. (Of course, just "Save" an already-existence policy is very light, but hacky a little bit) I want only to reload the file if possible.
... View more
Labels:
- Labels:
-
Apache Ranger
09-19-2018
10:14 AM
Thanks, @Sandeep Nemuri That reference link is also helpful!
... View more
09-18-2018
03:43 AM
What stores clusterID, namespaceID, and blockpoolID? As far as I know... ${dfs.namenode.name.dir}/current/VERSION stores clusterID, namespaceID, and blockpoolID. ${dfs.journalnode.edits.dir}/${nameservice}/current/VERSION stores clusterID and namespaceID. ${dfs.datanode.data.dir}/current/VERSION stores clusterID. Directory name under ${dfs.datanode.data.dir}/current includes blockpoolID. ${dfs.datanode.data.dir}/current/${blockpoolID}/current/VERSION stores namespaceID and blockpoolID. Is there any other file or file name/directory name storing those IDs?
... View more
- Tags:
- Hadoop Core
- HDFS
Labels:
- Labels:
-
Apache Hadoop
09-14-2018
09:37 AM
Thanks, @Chiran Ravani
... View more
09-13-2018
02:23 PM
This is very easy. I've just found soon before.
You can create an additional interpreter from "+Create" button on "Interpreters" screen. If you want to create one more "livy" interpreter, choose "livy" from "Interpreter group". And then I can enable the additional livy interpreter by clicking the interpreter in "Interpreter binding" menu on a notebook that you want to use the interpreter on.
... View more
09-13-2018
08:22 AM
What should I set to an "entity" access policy? As far as I tried, Atlas requires an "entity" access policy defined in Ranger must be "*". If I set other than "*" for "entity", Atlas says "You are not authorized for READ on [ENTITY] : *". I thought I can set the name of an entity like Hive table name to a value of an "entity" access policy. Is this incorrect?
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache Ranger
09-12-2018
08:29 AM
Thanks, @Jay Kumar SenSharma These days your comments help me a lot! Thank you very much!
... View more
09-12-2018
07:20 AM
What does "Recovery Host" do in Ambari 2.6.2.2? I found "Recovery Host" in "Host Actions". I learned that this functionality is enabled if all the components on the host are stopped. And then, after stopping all the components and click "Recovery Host", the confirmation window appeared and it shows "This action will completely re-install all components on this host." What does "completely re-install" mean? Additionally, as far as I tried, nothing happened when I clicked "Yes" on the confirmation window.
... View more
Labels:
- Labels:
-
Apache Ambari
09-11-2018
12:56 PM
As far as I saw, running ZooKeeper's "Restart All" on Ambari looks rolling-restart style. That is when one ZK is restarting, others are waiting. Is it right?
... View more
Labels:
- Labels:
-
Apache Ambari
09-11-2018
11:40 AM
Thank you very much, @Karthik Palanisamy That link is also very helpful.
... View more
09-11-2018
10:32 AM
"NameNode Service RPC Queue Latency (Hourly)" and "NameNode Service RPC Queue Latency (Hourly)" are "UNKNOWN".
Both alerts show "Service RPC port is not enabled.", but NameNodes are running without problems. (NameNodes are configured HA.)
Do I have to ignore these alerts?
But If possible, I want to turn these alerts to "OK".
... View more
Labels:
- Labels:
-
Apache Ambari
09-11-2018
10:04 AM
Thank you for your detailed comment!
... View more
09-11-2018
06:57 AM
Thanks, @Jay Kumar SenSharma You mean, for example, even if Metrics Collector is dead from 9 am to 11 am, after Metrics Collector recovers, Metrics Collector will receive all metrics from 9 am to 11 am, right?
... View more
09-11-2018
06:37 AM
Do Metrics Monitor and HadoopTimelineMetricsSink retain metrics to emit while MetricsCollector is dead? Or, discard them? If retaining metrics, how long?
... View more
Labels:
- Labels:
-
Apache Ambari
09-11-2018
02:21 AM
Hi @Predrag Minovic, You can set the interval of UserSync by ranger.usersync.sleeptimeinmillisbetweensynccycle, but there's a minimum value for each UserSync source. Please check https://github.com/apache/incubator-ranger/blob/master/ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java and grep this code by "UGSYNC_SLEEP_TIME_IN_MILLIS_BETWEEN_CYCLE_LDAP_DEFAULT_VALUE". In UNIX, 1 minute, but in LDAP, 1 hour. I think this is the reason of "sometimes not even after 15 minutes."
... View more
09-11-2018
12:49 AM
As far as I read some code of Hive View 2.0, once succeeded in authentication, id and password are stored in a ConcurrentHashMap, but even if authentication fails because of changing the password those are not removed from the ConcurrentHashMap. I think that is the reason..
... View more
09-05-2018
03:33 PM
Thanks for responding so promptly!!!
... View more
09-05-2018
03:17 PM
We can run HDFS Rebalance (HDFS Balancer process) from Ambari WebUI and that operation is an asynchronous one, that is just after starting HDFS Rebalance, "HDFS Rebalance" operation finishes but the real Balancer process is running on the NamoNode host. When I want to know whether the Balancer process finished or not, I can confirm by checking whether the Balancer process exists on the host or not. But this way is bothersome. (I know this is not too difficult. But bothersome a little.) I think if I can receive some notification of the completion of the balancer process on Ambari WebUI, that is awesome. Can I detect the completion of HDFS Rebalance invoked from Ambari WebUI on Ambari WebUI? Or, Do I have to check manually whether the balancer process exists or not?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
09-05-2018
12:34 PM
Thank you, @Jay Kumar SenSharma Then, I have two questions below: 1. If specifying a driver class name by "--driver" option every time when running sqoop, it is not necessary to append the driver class name to "jdbc_drivers" in "Advanced sqoop-env". Is it correct?
2. If I want to connect to a database that is not listed in params.py (e.g. IBM DB2), it is useless to append its driver's class name to "jdbc_drivers" in "Advanced sqoop-env" and I have to specify its driver's class name by "--driver" option every time when running sqoop. Is it correct?
... View more