@LPottakkattil  Do the following  to resolve that problem Clean out all packages and meta data from cache # Update the system metadata yum clean all yum check-update # Update the system yum update # update the Java connect you don't need to stop Ambari yum install -y mysql-connector-java (Optional) ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar   Hope that helps     ... View more

@Seeker90  Zookeepers by default are supposed to be running in an ensemble [french together] thats a quorum in English. Minimum number of servers required to run the Zookeeper is called   Quorum. Zookeeper replicates whole data tree to all the quorum servers. This number is also the minimum number of servers required to store a client’s data before telling the client it is safely stored. Quorum size should be calculated by Majority Rule   Majority rule: QN = (N + 1) / 2   QN : Minimum number of servers in quorum N : Total number of servers. (should be an odd number) So, if we have 5 servers, then Quorum should be minimum of 3 servers. As long as a majority of the ensemble are up, the service will be available. Because Zookeeper requires a majority, it is best to use an odd number of machines  ZooKeeper uses Quorums by default to prevent the "brain split" phenomenon. That is, only more than half of the nodes in the cluster can vote for the Leader. This way can ensure the uniqueness of the leader,  Split-Brain is like when you have two nodes in your cluster, they both know that a master needs to be elected in this cluster. Then when there is no problem in the communication between the two of them, a consensus will be reached and one of them will be selected as the master. But if there is a problem with the communication between them, both nodes will feel that there is no master now, so each elects itself as the master, so there will be two masters in the cluster.   So to avoid the current problem you are facing add 2 more Zk'ers and that problem will vanish Happy hadooping    ... View more

@vidanimegh  Ye that s the usual format but I didn't want to complicate the  explanation  the alias is optionaél anyways. But you see the logic the 2 hosts must be pingable for  eachother.   Happy hadooping ... View more

An easy way to solve this problem is to change the port that Apache - Nifi uses to run to another port that is different than 8080. We can make the change in the file: nifi.property under the folder: .\conf. 🙂 ... View more

Resolved the issue using below steps; Cloudera mangement service was added through UI using below steps: On clusters home page in UI, near Status, right corner -> Add -> cloudera managemnet service Before starting the service configure the TLS/SSL keystore like that of HDFS and YARN and restart the service. ... View more

@Shelton  Excuse for the delayed response.   I have attached the screenshot of the policy in ranger. hivedev policies Test policy -1 Test policy - 2   So according to policy, 'shaz' should have access 'qubz' database and 'edward' should not have access to the 'qubz' database.   Beeline:   edward@dev-2:~$ beeline SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/apache-hive-3.1.0-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/hadoop-3.1.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Beeline version 3.1.0 by Apache Hive beeline> !connect jdbc:hive2://dev-2:10000/; Connecting to jdbc:hive2://dev-2:10000/; Enter username for jdbc:hive2://dev-2:10000/: edward Enter password for jdbc:hive2://dev-2:10000/: Connected to: Apache Hive (version 3.1.0) Driver: Hive JDBC (version 3.1.0) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://dev-2:10000/> show databases; INFO : Compiling command(queryId=root_20210407130935_a28fc25d-2398-4e61-a205-f24e3ba937f1): show databases INFO : Concurrency mode is disabled, not creating a lock manager INFO : Semantic Analysis Completed (retrial = false) INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=root_20210407130935_a28fc25d-2398-4e61-a205-f24e3ba937f1); Time taken: 0.01 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=root_20210407130935_a28fc25d-2398-4e61-a205-f24e3ba937f1): show databases INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=root_20210407130935_a28fc25d-2398-4e61-a205-f24e3ba937f1); Time taken: 0.015 seconds INFO : OK INFO : Concurrency mode is disabled, not creating a lock manager +----------------+ | database_name | +----------------+ | default | | kylin | | mytesting | | qubz | +----------------+ 4 rows selected (0.115 seconds)   0: jdbc:hive2://dev-2:10000/> show tables from qubz; INFO : Compiling command(queryId=root_20210407131007_896cb7af-5452-4b5e-bd6c-1393c25e1bd7): show tables from qubz INFO : Concurrency mode is disabled, not creating a lock manager INFO : Semantic Analysis Completed (retrial = false) INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=root_20210407131007_896cb7af-5452-4b5e-bd6c-1393c25e1bd7); Time taken: 0.016 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=root_20210407131007_896cb7af-5452-4b5e-bd6c-1393c25e1bd7): show tables from qubz INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=root_20210407131007_896cb7af-5452-4b5e-bd6c-1393c25e1bd7); Time taken: 0.013 seconds INFO : OK INFO : Concurrency mode is disabled, not creating a lock manager +-----------+ | tab_name | +-----------+ +-----------+ No rows selected (0.043 seconds) But, edward was able to see 'qubz' db ( to which he was restricted access, In old Ranger, DB will not be even visible - Is this expected here? ) but no tables visible, whereas 'shaz' user was able to view the DB as well as tables as expected.     ... View more

Hello Michael,    In your insert query are you referring to table like "databasename.tablename"?, if yes, then first try to execute    "use databasename"    And then execute the query. ... View more

Hi @Ninads ,   I am also using CDP 7.1.4 and having same error when spark connects to hbase, did you manage to identify the issue ?    Best Regards,  ... View more

Edit: I've learned that decommissioning nodes might not be the preferred way. otherwise, the problem statement remains the same. i.e. The job should not get scheduled on nodes where a particular service is down. ... View more

Hi @Priya09, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. ... View more

@Shelton I sent you a request. ... View more

Thanks @Shelton for the response. I believe I had already tried this. I tried to replace the ambari tez configuration again but still the same error. Also our production cluster currently has the old value, but for some reason the production cluster is working fine. ... View more

Thank you for your reply  I don't want to use spark warehouse, I want to use hive warehouse the global hive  ... View more

@Jay2021  Impala and hive share metadata catalog ie Hive MetaStore , when a database/table is created in HIVE it's readily available for hive users but not Impala! To successfully query a table or database created in HIVE there is a caveat you need to run the INVALIDATE METADATA from the impala-shell before the table is available for Impala queries. INVALIDATE METADATA reloads all the metadata for the table needed for a subsequent query  The next time the current Impala node performs a query against a table whose metadata is invalidated you definitely will run into errors you could use the REFRESH in the common case where you add new data files for an existing table it reloads the metadata immediately, but only loads the block location data for newly added data files, making it a less expensive operation overall. INVALIDATE METADATA [[db_name.]table_name] Example   $ impala-shell > INVALIDATE METADATA new_db_from_hive.new_table_from_hive; > SHOW TABLES IN new_db_from_hive; +---------------------+ | new_table_from_hive | +---------------------+   That should resolve your issue Happy hadooping ... View more

@SnehasishRSC  REFRESH in the common case where you add new data files for an existing table it reloads the metadata immediately, but only loads the block location data for newly added data files, making it a less expensive operation overall. It is recommended to run COMPUTE STATS when 30 % of data is altered in a table, where altered means the addition or deletion of files/data. INVALIDATE METADATA is a relatively expensive operation compared to the incremental metadata update done by the REFRESH statement, so in the common scenario of adding new data files to an existing table, prefer REFRESH rather than INVALIDATE METADATA which marks the metadata for one or all tables as stale. The next time the Impala service performs a query against a table whose metadata is invalidated, Impala reloads the associated metadata before the query proceed. Hope that helps ... View more

Yes @PabitraDas , it's accessible now. It would be helpful to have a secondary repository or maintenance windows announced. ... View more

@rohit_sharma  Any updates? Do you still need help? ... View more

@Alex_IT  From my Oracle knowledge, there are 2 options for migrating the same Oracle_home [DB] from 12C to 19C  if you are running 12.1.0.2 then you have the direct path see the attached matrix. With this option, you won't need to change the hostname.  The other option is to export your current schema CM ,oozie,hive,hue,Ranger etc schemas   install a fresh Oracle 19c box with an empty database, and import the old schemas this could be a challenge as you might have to rebuild indexes or recompile some database packages etc but bot are doable. Hope that helps ... View more

Hi @rguruvannagari ,   How does Ambari get the kadmin principal service ticket? Is it possible to get it by running a command in the shell?   I'm trying to figure out a way of logging into the kadmin shell.   My setup is integrated with AD as the KDC. I'm trying to kadmin -p myadminprincipal@MYREALM on the terminal but it says "kadmin: Database error! Required KADM5 principal missing while initializing kadmin interface". It makes me wonder how ambari server manages to create principals and keytabs with those same credentials.   I even tried to decode the ambari-agent kerberos_common.py files to see the flow and there also it seems that Ambari Server indeed uses kadmin for the same.   Would really appreciate any help in this regard. Thanks, Megh ... View more

It is not recommended to update ambari DB directly but as in this case the cmd 'ambari-server update-host-names host_names_changes.json' is not a much help, we can perform below actions by taking ambari DB back up. I tested and it worked.   Note: Please take DB backup! and PLEASE only do at your own risk.   In this case, the new host_name is ambarinn.cluster.com and new name is ambarinn261.cluster.com We need to concentrate on the host_name that says healthStatus":"UNKNOWN".   Steps to resolve: ambari-agent and ambari-server successfully stopped [root@ambarinn261 ~]# su - postgres Last login: Wed Jan 27 13:14:40 EST 2021 on pts/0 -bash-4.2$ psql psql (9.2.24) Type "help" for help. postgres=# \c ambari You are now connected to database "ambari" as user "postgres". ambari=# select host_id,host_name,discovery_status,last_registration_time,public_host_name from ambari.hosts; host_id | host_name | discovery_status | last_registration_time | public_host_name ---------+-------------------------+------------------+------------------------+------------------------- 201 | ambarinn261.cluster.com | | 1611772466200 | ambarinn261.cluster.com 1 | ambarinn.cluster.com | | 1611768543457 | ambarinn.cluster.com (2 rows) ambari=# select * from ambari.hoststate; agent_version | available_mem | current_state | health_status | host_id | time_in_state | maintenance_state -----------------------+---------------+---------------+----------------------------------------------+---------+---------------+------------------- {"version":"2.6.1.5"} | 279948 | INIT | {"healthStatus":"HEALTHY","healthReport":""} | 201 | 1611772466200 | {"version":"2.6.1.5"} | 2120176 | INIT | {"healthStatus":"UNKNOWN","healthReport":""} | 1 | 1611768543457 | (2 rows) ambari=# UPDATE ambari.hosts SET host_name='ambarinn261.cluster.com' WHERE host_id=1; ERROR: duplicate key value violates unique constraint "uq_hosts_host_name" DETAIL: Key (host_name)=(ambarinn261.cluster.com) already exists. ambari=# UPDATE ambari.hosts SET public_host_name='ambarinn261.cluster.com' WHERE host_id=1; UPDATE 1 ambari=# UPDATE ambari.hosts SET public_host_name='ambarinn261a.cluster.com' WHERE host_id=201; UPDATE 1 ambari=# UPDATE ambari.hosts SET host_name='ambarinn261a.cluster.com' WHERE host_id=201; UPDATE 1 ambari=# UPDATE ambari.hosts SET host_name='ambarinn261.cluster.com' WHERE host_id=1; UPDATE 1 ambari=# \q -bash-4.2$ exit logout [root@ambarinn261 ~]# ambari-server start       ... View more

@sow    I am also having the same issue, did you get any resolution for this issue? ... View more

If we want to limit interaction of hdp/hadoop developers/data analyst or scientist, does it mean we don't need to install client in all workernodes? And we have ever found that for special case, sqoop and oozie client, are needed to be installed in all nodes include master-worker nodes, Is it related to how sqoop and oozie works? ... View more

@akriti Thank you !   Well, then we will continue using express edition as it's not going to be affected. and we aren't going for the subscription.    We have already got License of CDP . Hope we continue our testings with our express edition without going for subscription and we are allowed to use it without it being disrupted.  ... View more