What happened?   Starting up a ZooKeeper server in a Kerberized CDP-DC 7.0.3 cluster failed with the logs below.    2020-03-30 12:23:10,251 ERROR org.apache.zookeeper.server.quorum.QuorumPeerMain: Unexpected exception, exiting abnormally java.io.IOException: Could not configure server because SASL configuration did not allow the   ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: Message stream modified (41)         at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:243)         at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646)         at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:148)         at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123)         at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)   The JDK for this environment is OpenJDK 1.8.0_242.   # java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)   Solution   Removing the line of renew_lifetime in /etc/krb5.conf. Removing this line means to use the default value, 0, for renew_lifetime. Thus, it may also need to specify renew_lifetime when running kinit command.   See also   http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201909.mbox/%3CCAKRKJ1O3yrYKDZ=WhU=i6A+zqxFnUidxvwQzNCTW0mnEv2WFPA@mail.gmail.com%3E On this page, Akira Ajisaka, one of Hadoop PMCs, described the solution. He also mentioned a related OpenJDK's JIRA ticket.   https://bugs.centos.org/view.php?id=17000 This page also introduced the same solution. Additionally, this page showed another solution, setting  sun.security.krb5.disableReferrals=true in java.security file. But in my case, this solution didn't work.   https://my.cloudera.com/knowledge/Cloudera-Customer-Advisory-Servers-with-Kerberos-enabled-stop?id=292027 This is a related article from the Cloudera Knowledge Base. It also describes  sun.security.krb5.disableReferrals=true  as its workaround.   ... View more

As far as I know, the HDP 2.6.5 document does not have the migration guide for IOP to HDP, but the HDP 2.6.4 documentation has it. Is that mean upgrading from IOP to HDP 2.6.5 directly is impossible? ... View more

Is there any case to prefer RDD instead of DataFrame/Dataset/SparkSQLin Spark 2.3.0 or later? AFAIK, using DataFrame/Dataset/SparkSQL has a lot of merits. For example, simpler coding and Optimization by Catalyst. If there's some concrete example that should be written with RDD, let me know! Thanks, ... View more

I read https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/data-operating-system/content/installation_modes_hbase_timeline_service.html This page says "System service mode: This mode of installation works on clusters where the capacity on each node is at least 10 GB and the total cluster capacity is at least 50 GB." Does it mean "`yarn.nodemanager.resource.memory-mb` > 10GB and the sum of that property in the cluster > 50GB" ? ... View more

Why does HDP 3.0.0 have both ATSv1.5 and ATSv2? ... View more

As far as I know, when I run `hdfs dfs -put` command on a DataNode server, the first block replica will be created on that DataNode server. Which DataNode does NameNode choose for the first replica when I run `hdfs dfs -put` command on a DataNode server that does not satisfy the Storage Policy of the file to be written? (e.g. The Storage Policy is "One_SSD" but the DataNode server which `hdfs dfs -put` command is invoked does not have any "SSD" labeled directory.) ... View more

What functionality becomes unavailable when App Timeline Server isn't running? I know Ambari Tez View becomes unavailable (showing an error that indicates it is impossible to connect with ATS) when ATS isn't running. Of course, ATS WebUI becomes unavailable. But, as far as I saw, ResourceManager runs as usual and yarn cli like "yarn application -status" also runs as usual. As for Spark application, it looks no damage. Spark History Server WebUI shows as usual because Spark History Server only depends on HDFS. Is there any other problem? ... View more

Does Spark application communicate with App Timeline Server? Hive has some hook points (pre, post, and failure hooks) and ATSHook is set to those properties in default. How about Spark? As far as I checked, I couldn't find... ... View more

How do I delete an Atlas tag which is associated with entities those are already deleted? I found that I couldn't delete an Atlas tag named "mytag" and when I tried to delete that tag I saw "Given type mytag has references". However, "mytag" was not associated with any entities. Then I noticed "Show historical entities" checkbox and I checked the checkbox, then an entity already deleted appeared. I thought I could delete "mytag" after deleting the association between "mytag" and that entity, but I couldn't find the way to remove that association. ... View more

How do I reload the file (specified by "FilePath" attribute in enricherOptions) or RangerFileBasedGeolocationProvider? As far as I tried, that file is not automatically reloaded. Restarting HiveServer2 if the component using RangerFileBasedGeolocationProvider is Hive causes reloading the file. Doing "Save" a policy also causes reloading the file, no modification required. This is very inconvenient. (Of course, just "Save" an already-existence policy is very light, but hacky a little bit) I want only to reload the file if possible. ... View more

How can I take tags those are not associated with any entity by Atlas Export API? As far as I tried, Atlas Export API does not return tags those are not associated with any entity. My current request body for Atlas Export API is below. { "itemsToExport" : [ { "typeName" : "Asset", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "DataSet", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "fs_path", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hdfs_path", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hive_column", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hive_column_lineage", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hive_db", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hive_process", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hive_storagedesc", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "hive_table", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "Infrastructure", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "Process", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "Referenceable", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "sqoop_dbdatastore", "uniqueAttributes" : { "name" : ".+" } }, { "typeName" : "sqoop_process", "uniqueAttributes" : { "name" : ".+" } } ], "options" : { "fetchType" : "FULL", "matchType" : "matches" } } ... View more

How can I take backup of App Timeline Server and restore App Timeline Server from backup? Taking backups of yarn.timeline-service.leveldb-state-store.path and yarn.timeline-service.leveldb-timeline-store.path is enough for backup? And Copying the backups to yarn.timeline-service.leveldb-state-store.path and yarn.timeline-service.leveldb-timeline-store.path and starting ATS is enough for restoring? ... View more

As far as I saw, running ZooKeeper's "Restart All" on Ambari looks rolling-restart style. That is when one ZK is restarting, others are waiting. Is it right? ... View more

What does MapReduce2 History Server depend on? Does it only depend on HDFS? Doesn't it depend on App Timeline Server? ... View more