Member since
01-19-2017
3681
Posts
633
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1640 | 06-04-2025 11:36 PM | |
| 2089 | 03-23-2025 05:23 AM | |
| 997 | 03-17-2025 10:18 AM | |
| 3776 | 03-05-2025 01:34 PM | |
| 2601 | 03-03-2025 01:09 PM |
04-19-2021
08:07 AM
@vidanimegh Can you try it differently like below and let me know? hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://<source-nameservice>/path/to/source/folder hdfs://<destination-nameservice>/path/to/destination/folder Always try to use nameservice !
... View more
03-01-2021
12:58 PM
@ryu My advice is just don't attempt because the HDP software is closely wired. Vigorous unit testing and compatibility are implemented before certifying a version. HDP is a packaged software when you update it's either all or none, you can't update only a component except Ambari and the underlying databases for hive,oozie,ranger etc Yes, the old good days of real open source is gone.I loved HWX If you are running production clusters then you definitely need to a subscription Hope that helps
... View more
03-01-2021
12:41 PM
@totti1 You will need to copy the hdfs/core-site.xml to a local path accessible to your windows. And you will need to update your host's file entry to make the VM reachable from the windows machine. You should be able to ping your vm from the windows machine and vice versa. Edit and change core-site.xml and hdfs-site.xml files and remove the FQDN:8020 to an IP ie for class C network like 192.168.10.201:8020 restart the processors and let me know. Hope that helps?
... View more
03-01-2021
10:11 AM
@totti1 Nifi cluste is not aware of your Hadoop cluster until you copy these 2 files from your cluster /etc/hadoop/conf/hdfs-site.xml or /etc/hadoop/conf/core-site.xml to your local nifi installation Hadoop configuration resources=/local/dir/hdfs-site.xml,/local/dir/core-site.xml look for any of these processor group for HDFS Hope that helps
... View more
01-12-2021
11:58 AM
1 Kudo
@zetta4ever In a Hadoop cluster, three types of nodes exist Master, Worker and edge nodes. The distinction of roles helps maintain efficiency. Master nodes control which nodes perform which tasks and what processes run on what nodes. The majority of work is assigned to worker nodes. Worker node store most of the data and perform most of the calculations Edge nodes aka gateway facilitate communications from end users to master and worker nodes. The 3 masternodes should have the Namenode[Active & Standby],YARN [Active & Standby], Zookeeper Quorum [3 masters] and the other component you intend to install and on the 6 worker node aka slave nodes you will install the Nodemanager,Datanodes and the all the clients. There is no need to install the client on the master nodes, Some nodes have important tasks, which may impact performance if interrupted. Edge nodes allow end-users to contact worker nodes when necessary, providing a network interface for the cluster without leaving the entire cluster open to communication. That limitation improves reliability and security. As work is evenly distributed between work nodes, the edge node’s role helps avoid data skewing and performance issues. See my document on edge node https://community.cloudera.com/t5/Support-Questions/Edge-node-or-utility-node-packages/td-p/202164# Hope that helps
... View more
01-04-2021
09:47 AM
@Mondi The simple answer is YES and the best source is the vendor itself Rack awareness CDP as computations are performed with the assistance of rack awareness scripts. Hope that helps Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, Kudos this answer by hitting the thumbs up button.
... View more
01-03-2021
01:45 PM
1 Kudo
@bvishal You don't really need to mix 2 different databases [Postresql and MySQL]. You can use MySQL or MariaDB free version of MYSQL in the advanced database configuration for you cluster. MySQL has the typical SQL syntax Postgresql is another world ! You don't need to install MySQL on the ambari agent hosts because that will mean if you have 20 nodes you will be running 20 MySQL/MariaDB databases. Usually, you install the MySQL/MariaDB on the Ambari host and you create apart from Ambari database, hive,oozie, ranger,RangerKMS etc. If you are deploying using Ambari then the ambari-agents are deployed automatically and configured by Ambari. Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, kudos this answer by hitting the thumbs up button.
... View more
01-03-2021
12:17 PM
@PauloNeves Yes, the command show databases will list all databases in a Hive instance whether you are authorized to access it or not. I am sure this is cluster devoid of Ranger or Sentry which are the 2 authorization tools in Cloudera!!! Once the ranger plugin is enabled then authorization is delegated to Ranger to provide fine-grained data access control in Hive, including row-level filtering and column-level masking. This is the recommended setting to make your database administration easier as it provides a centralized security administration, access control, and detailed auditing for user access within the Hadoop, Hive, HBase, and other components in the ecosystem. Unfortunately, I had already enabled the Ranger plugin for hive on my cluster but all the same, it confirms what I wrote above. Once the ranger plugin is enabled for a component ie. hive,HBase or Kafka then the authorization is managed exclusively through Ranger Database listing before Ranger Below is what happens if my user sheltong has not explicitly been given authorization through Ranger, see [screenshots] I see no database though I have over 8 databases See the output of the hive user who has explicit access to all the tables due to the default policy he could see the databases. Database listing after Ranger After creating a policy explicitly giving the user sheltong access to the 3 databases Policy granting explicit access to 3 databases Now when I re-run the show databases bingo! Back to your question show tables from forbidden_db, it returns an empty list, this can be true especially if the database is empty! has not table like the screenshot below though I have access to the database it's empty Now I create a table and re-run the select now I am able to see the table I hope this demonstrates the power of Ranger and explains maybe what you are encountering, I am also thinking if your cluster has Ranger hive plugin enabled you could have select on the databases but you will need explicit minimum select or the following permission on the underlying database tables to be able to see them. Happy Hadooping
... View more
01-01-2021
02:00 PM
@chhaya_vishwaka Can you confirm you went through all the Prerequisites for adding classic clusters and checked against the Cloudera Support Matrix.?? Please revert
... View more
01-01-2021
01:45 PM
1 Kudo
@brunokatekawa What is happening if my guess is right is you are trying to use your community username/password this will definitely fail. Ambari 2.7.x is available for companies with valid HDP 3.x support licenses you have an active subscription with Cloudera as you can see below access is denies as I used my community login. Here is the HDP support Matrix Starting with the HDP 3.1.5 release, access to HDP repositories requires authentication. To access the binaries, you must first have the required authentication credentials (username and password). Read accessing HDP repositories Hope that helps
... View more