About toide

toide · ‎03-31-2020

What happened? Starting up a ZooKeeper server in a Kerberized CDP-DC 7.0.3 cluster failed with the logs below. 2020-03-30 12:23:10,251 ERROR org.apache.zookeeper.server.quorum.QuorumPeerMain: Unexpected exception, exiting abnormally java.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: Message stream modified (41) at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:243) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:148) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) The JDK for this environment is OpenJDK 1.8.0_242. # java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) Solution Removing the line of renew_lifetime in /etc/krb5.conf. Removing this line means to use the default value, 0, for renew_lifetime. Thus, it may also need to specify renew_lifetime when running kinit command. See also http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201909.mbox/%3CCAKRKJ1O3yrYKDZ=WhU=i6A+zqxFnUidxvwQzNCTW0mnEv2WFPA@mail.gmail.com%3E On this page, Akira Ajisaka, one of Hadoop PMCs, described the solution. He also mentioned a related OpenJDK's JIRA ticket. https://bugs.centos.org/view.php?id=17000 This page also introduced the same solution. Additionally, this page showed another solution, setting sun.security.krb5.disableReferrals=true in java.security file. But in my case, this solution didn't work. https://my.cloudera.com/knowledge/Cloudera-Customer-Advisory-Servers-with-Kerberos-enabled-stop?id=292027 This is a related article from the Cloudera Knowledge Base. It also describes sun.security.krb5.disableReferrals=true as its workaround.

toide · ‎11-08-2018

I wrote "restarting Ambari Server" but that does not mean "restarting a host that Ambari Server is installed". Just restarting "Ambari Server" takes effect. (= ambari-server restart)

toide · ‎11-06-2018

Unfortunately, I haven't got a solution yet.

toide · ‎10-18-2018

Thanks, @Jay Kumar SenSharma

toide · ‎10-17-2018

How do I restrict access to Ambari WebUI by IP address? Of course, using a firewall included in OS is a solution, but I'd like to know the way which requires only to modify Ambari's configurations. I know Ambari uses Jetty for HTTP server and Jetty provides IP address restriction by IPAccessHandler https://www.eclipse.org/jetty/documentation/9.4.x/ipaccess-handler.html, but I'm not sure how to apply this to Ambari.

toide · ‎10-09-2018

Thanks, @Jay Kumar SenSharma!

toide · ‎10-09-2018

I'm using Ambari 2.6.2.2 and Ambari 2.6.2.2 has "Service Auto Start Configuration" that enables a component restarting when it went down unexpectedly. However, I could not find an automatic "Restart" operation in the operations history when the automatic restart functionality worked. How can I get to know that the automatic component restart happened?

toide · ‎10-05-2018

Hi, Please try adding <service> <role>ATLAS-API</role> <url>http://${atlas_metadata_server_host}:${atlas_metadata_server_port}/</url> </service> (e.g. <service> <role>ATLAS-API</role> <url>http://toide-2.toide.hortonworks.com:21000/</url> </service> ) to your "Advanced topology". I met "Something went wrong" error on Atlas WebUI before adding it.

toide · ‎09-28-2018

Thanks @Sharmadha Sainath

toide · ‎09-28-2018

Thanks @akulkarni

toide · ‎09-27-2018

How do I delete an Atlas tag which is associated with entities those are already deleted? I found that I couldn't delete an Atlas tag named "mytag" and when I tried to delete that tag I saw "Given type mytag has references". However, "mytag" was not associated with any entities. Then I noticed "Show historical entities" checkbox and I checked the checkbox, then an entity already deleted appeared. I thought I could delete "mytag" after deleting the association between "mytag" and that entity, but I couldn't find the way to remove that association.

toide · ‎09-25-2018

How do I reload the file (specified by "FilePath" attribute in enricherOptions) or RangerFileBasedGeolocationProvider? As far as I tried, that file is not automatically reloaded. Restarting HiveServer2 if the component using RangerFileBasedGeolocationProvider is Hive causes reloading the file. Doing "Save" a policy also causes reloading the file, no modification required. This is very inconvenient. (Of course, just "Save" an already-existence policy is very light, but hacky a little bit) I want only to reload the file if possible.

toide · ‎09-19-2018

Thanks, @Sandeep Nemuri That reference link is also helpful!

toide · ‎09-18-2018

What stores clusterID, namespaceID, and blockpoolID? As far as I know... ${dfs.namenode.name.dir}/current/VERSION stores clusterID, namespaceID, and blockpoolID. ${dfs.journalnode.edits.dir}/${nameservice}/current/VERSION stores clusterID and namespaceID. ${dfs.datanode.data.dir}/current/VERSION stores clusterID. Directory name under ${dfs.datanode.data.dir}/current includes blockpoolID. ${dfs.datanode.data.dir}/current/${blockpoolID}/current/VERSION stores namespaceID and blockpoolID. Is there any other file or file name/directory name storing those IDs?

toide · ‎09-14-2018

Thanks, @Chiran Ravani

toide · ‎09-13-2018

This is very easy. I've just found soon before. You can create an additional interpreter from "+Create" button on "Interpreters" screen. If you want to create one more "livy" interpreter, choose "livy" from "Interpreter group". And then I can enable the additional livy interpreter by clicking the interpreter in "Interpreter binding" menu on a notebook that you want to use the interpreter on.

toide · ‎09-13-2018

What should I set to an "entity" access policy? As far as I tried, Atlas requires an "entity" access policy defined in Ranger must be "*". If I set other than "*" for "entity", Atlas says "You are not authorized for READ on [ENTITY] : *". I thought I can set the name of an entity like Hive table name to a value of an "entity" access policy. Is this incorrect?

toide · ‎09-12-2018

Thanks, @Jay Kumar SenSharma These days your comments help me a lot! Thank you very much!

toide · ‎09-12-2018

What does "Recovery Host" do in Ambari 2.6.2.2? I found "Recovery Host" in "Host Actions". I learned that this functionality is enabled if all the components on the host are stopped. And then, after stopping all the components and click "Recovery Host", the confirmation window appeared and it shows "This action will completely re-install all components on this host." What does "completely re-install" mean? Additionally, as far as I tried, nothing happened when I clicked "Yes" on the confirmation window.

toide · ‎09-11-2018

As far as I saw, running ZooKeeper's "Restart All" on Ambari looks rolling-restart style. That is when one ZK is restarting, others are waiting. Is it right?

toide · ‎09-11-2018

Thank you very much, @Karthik Palanisamy That link is also very helpful.

toide · ‎09-11-2018

"NameNode Service RPC Queue Latency (Hourly)" and "NameNode Service RPC Queue Latency (Hourly)" are "UNKNOWN". Both alerts show "Service RPC port is not enabled.", but NameNodes are running without problems. (NameNodes are configured HA.) Do I have to ignore these alerts? But If possible, I want to turn these alerts to "OK".

toide · ‎09-11-2018

Thank you for your detailed comment!

toide · ‎09-11-2018

Thanks, @Jay Kumar SenSharma You mean, for example, even if Metrics Collector is dead from 9 am to 11 am, after Metrics Collector recovers, Metrics Collector will receive all metrics from 9 am to 11 am, right?

toide · ‎09-11-2018

Do Metrics Monitor and HadoopTimelineMetricsSink retain metrics to emit while MetricsCollector is dead? Or, discard them? If retaining metrics, how long?

toide · ‎09-11-2018

Hi @Predrag Minovic, You can set the interval of UserSync by ranger.usersync.sleeptimeinmillisbetweensynccycle, but there's a minimum value for each UserSync source. Please check https://github.com/apache/incubator-ranger/blob/master/ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java and grep this code by "UGSYNC_SLEEP_TIME_IN_MILLIS_BETWEEN_CYCLE_LDAP_DEFAULT_VALUE". In UNIX, 1 minute, but in LDAP, 1 hour. I think this is the reason of "sometimes not even after 15 minutes."

toide · ‎09-11-2018

As far as I read some code of Hive View 2.0, once succeeded in authentication, id and password are stored in a ConcurrentHashMap, but even if authentication fails because of changing the password those are not removed from the ConcurrentHashMap. I think that is the reason..

toide · ‎09-05-2018

Thanks for responding so promptly!!!

toide · ‎09-05-2018

We can run HDFS Rebalance (HDFS Balancer process) from Ambari WebUI and that operation is an asynchronous one, that is just after starting HDFS Rebalance, "HDFS Rebalance" operation finishes but the real Balancer process is running on the NamoNode host. When I want to know whether the Balancer process finished or not, I can confirm by checking whether the Balancer process exists on the host or not. But this way is bothersome. (I know this is not too difficult. But bothersome a little.) I think if I can receive some notification of the completion of the balancer process on Ambari WebUI, that is awesome. Can I detect the completion of HDFS Rebalance invoked from Ambari WebUI on Ambari WebUI? Or, Do I have to check manually whether the balancer process exists or not?

toide · ‎09-05-2018

Thank you, @Jay Kumar SenSharma Then, I have two questions below: 1. If specifying a driver class name by "--driver" option every time when running sqoop, it is not necessary to append the driver class name to "jdbc_drivers" in "Advanced sqoop-env". Is it correct? 2. If I want to connect to a database that is not listed in params.py (e.g. IBM DB2), it is useless to append its driver's class name to "jdbc_drivers" in "Advanced sqoop-env" and I have to specify its driver's class name by "--driver" option every time when running sqoop. Is it correct?

Online	Offline
Last Visited	‎03-31-2020 10:24 PM

Date Registered	‎01-15-2018 03:07 AM
Last Visited	‎03-31-2020 10:24 PM
Total Messages Posted	93
Total Kudos Received	2

Re: How do I create one more Zeppelin spark2 inter...

How to solve the `Message stream modified (41)` er...

Re: How can I input the new LDAP password on Hive ...

Re: How can I input the new LDAP password on Hive ...

Re: How do I restrict access to Ambari WebUI by IP...

How do I restrict access to Ambari WebUI by IP add...

Re: How can I get to know that the automatic compo...

How can I get to know that the automatic component...

Re: Configure Knox to access Atlas UI

Re: How do I delete an Atlas tag which are associa...

Re: How do I reload the file for RangerFileBasedG...

How do I delete an Atlas tag which are associated ...

How do I reload the file for RangerFileBasedGeolo...

Re: What stores clusterID, namespaceID, and blockp...

What stores clusterID, namespaceID, and blockpoolI...

Re: What should I set to an "entity" access policy...

Re: How do I create one more Zeppelin spark2 inter...

What should I set to an "entity" access policy?

Re: What does "Recovery Host" do in Ambari 2.6.2....

What does "Recovery Host" do in Ambari 2.6.2.2?

ZooKeeper "Restart All" on Ambari

Re: "NameNode Service RPC Queue Latency (Hourly)"...

"NameNode Service RPC Queue Latency (Hourly)" and...

Re: Do Metrics Monitor and HadoopTimelineMetricsSi...

Re: Do Metrics Monitor and HadoopTimelineMetricsSi...

Do Metrics Monitor and HadoopTimelineMetricsSink r...

Re: Is there a way to force Ranger user-sync to ru...

Re: How can I input the new LDAP password on Hive ...

Re: Can I detect the completion of HDFS Rebalance ...

Can I detect the completion of HDFS Rebalance kick...

Re: What is "jdbc_drivers" in "Advanced sqoop-env"...