Member since
01-19-2017
3679
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 927 | 06-04-2025 11:36 PM | |
| 1529 | 03-23-2025 05:23 AM | |
| 757 | 03-17-2025 10:18 AM | |
| 2728 | 03-05-2025 01:34 PM | |
| 1808 | 03-03-2025 01:09 PM |
03-01-2021
10:11 AM
@totti1 Nifi cluste is not aware of your Hadoop cluster until you copy these 2 files from your cluster /etc/hadoop/conf/hdfs-site.xml or /etc/hadoop/conf/core-site.xml to your local nifi installation Hadoop configuration resources=/local/dir/hdfs-site.xml,/local/dir/core-site.xml look for any of these processor group for HDFS Hope that helps
... View more
02-16-2021
11:04 PM
@rohit_sharma Can you change your syntax as below, note the zookeeper ensemble /bin/kafka-topics.sh --create \
--zookeeper zk1:2181,zk2:2181,zk3:2181 \
--topic "topic_name" \
--partitions 1> \
--replication-factor 2 Hope that helps
... View more
01-12-2021
11:58 AM
1 Kudo
@zetta4ever In a Hadoop cluster, three types of nodes exist Master, Worker and edge nodes. The distinction of roles helps maintain efficiency. Master nodes control which nodes perform which tasks and what processes run on what nodes. The majority of work is assigned to worker nodes. Worker node store most of the data and perform most of the calculations Edge nodes aka gateway facilitate communications from end users to master and worker nodes. The 3 masternodes should have the Namenode[Active & Standby],YARN [Active & Standby], Zookeeper Quorum [3 masters] and the other component you intend to install and on the 6 worker node aka slave nodes you will install the Nodemanager,Datanodes and the all the clients. There is no need to install the client on the master nodes, Some nodes have important tasks, which may impact performance if interrupted. Edge nodes allow end-users to contact worker nodes when necessary, providing a network interface for the cluster without leaving the entire cluster open to communication. That limitation improves reliability and security. As work is evenly distributed between work nodes, the edge node’s role helps avoid data skewing and performance issues. See my document on edge node https://community.cloudera.com/t5/Support-Questions/Edge-node-or-utility-node-packages/td-p/202164# Hope that helps
... View more
01-05-2021
12:28 PM
@Ninads Here is a community article by @kramalingam Connecting to Kerberos secured HBase cluster from Java application it's a walkthrough that should give you ideas Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, Kudos this answer by hitting the thumbs up button.
... View more
01-05-2021
12:14 PM
@sass You should get worried if you are using CDH express because once the trial period expires, a valid subscription will be required to continue the use of the software. This blanket change of policy will affect all legacy versions for Cloudera Distribution including Apache Hadoop (CDH), Hortonworks Data Platform (HDP), Data Flow (HDF/CDF), and Cloudera Data Science Workbench (CDSW). Here is a good read from Cloudera and the details of want you should know and expect come January 31, 2021 Paywall Expansion Update Happy hadooping Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, Kudos this answer by hitting the thumbs up button.
... View more
01-05-2021
11:49 AM
@MayankJ Your suspicion is spot on !! Note: Sentry only allows you to grant roles to groups that have alphanumeric characters and underscores (_) in the group name. When Sentry is enabled, you must use Beeline to execute Hive queries. Hive CLI is not supported with Sentry and must be disabled. See Disabling Hive CLI for information on how to disable the Hive CLI. The GRANT ROLE statement can be used to grant roles to groups and Only Sentry admin users can grant roles to a group. Create a role CREATE ROLE datascientist;
GRANT ROLE datascientist TO GROUP gurus; Grant to the database test GRANT ALL ON DATABASE test TO ROLE datascientist; Grant to a table lesson in test database GRANT ALL ON TABLE test.lesson TO ROLE datascientist; The reason Sentry grants ROLES to GROUPS is logic to simplifies management where you bundle privileges and grant it to a group so if that the only moving part is the user, so the below statement will effectively disable mayankj's grants to do anything privileges the datascientist roles # gpasswd -d mayankj gurus Removing user mayankj from group datascientist Quite simple and effective Roles are created to group together privileges or other roles. They are a means of facilitating the granting of multiple privileges or roles to groups. Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, kudos this answer by hitting the thumbs up button.
... View more
01-05-2021
11:06 AM
@saivenkatg55 My Assumptions You already executed the HDP environment preparation. If not see prepare the environment https://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/prepare_the_environment.html You are running on Linux [RedHat, Centos] and you have root access! Note: Replace test.ambari.com with the output of your $ hostname -f Re-adapt to fit your cluster # root password = welcome1
# hostname = test.ambari.com
# ranger user and password is the same Steps Install the MySQL connector if not installed [Optional] # yum install -y mysql-connector-java Shutdown Ambari # ambari-server stop Re-run the below command it won't hurt # ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar Backup the ambari server properties file # cp /etc/ambari-server/conf/ambari.properties /etc/ambari-server/conf/ambari.properties.bak Change the timeout of the ambari server # echo 'server.startup.web.timeout=120' >> /etc/ambari-server/conf/ambari.properties
# echo 'server.jdbc.connection-pool.acquisition-size=5' >> /etc/ambari-server/conf/ambari.properties
# echo 'server.jdbc.connection-pool.max-age=0' >> /etc/ambari-server/conf/ambari.properties
# echo 'server.jdbc.connection-pool.max-idle-time=14400' >> /etc/ambari-server/conf/ambari.properties
# echo 'server.jdbc.connection-pool.max-idle-time-excess=0' >> /etc/ambari-server/conf/ambari.properties
# echo 'server.jdbc.connection-pool.idle-test-interval=7200' >> /etc/ambari-server/conf/ambari.properties Recreate a new ranger schema & Database # mysql -u root -pwelcome1
CREATE USER 'rangernew'@'%' IDENTIFIED BY 'rangernew';
GRANT ALL PRIVILEGES ON *.* TO 'rangernew'@'localhost';
CREATE USER 'rangernew'@'%' IDENTIFIED BY 'rangernew';
GRANT ALL PRIVILEGES ON rangernew.* TO 'rangernew'@'%';
GRANT ALL PRIVILEGES ON rangernew.* TO 'rangernew'@'localhost' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON rangernew.* to 'rangernew'@'localhost' identified by 'rangernew';
GRANT ALL PRIVILEGES ON rangernew.* to 'rangernew'@'test.ambari.com' identified by 'rangernew';
GRANT ALL PRIVILEGES ON rangernew.* TO 'rangernew'@'test.ambari.com';
GRANT ALL PRIVILEGES ON rangernew.* TO 'rangernew'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
quit; Create the new ranger database # mysql -u rangernew -prangernew
create database rangernew;
show databases;
quit; Start the ambari server # ambari-server start
......Desired output.........
..................
.................
Ambari Server 'start' completed successfully. For ranger Ambari UI setup Use the hostname in this example test.ambari.com and the corresponding passwords Test the Ranger DB connectivity The connection test should succeed if it does then you can now start Ranger successfully. Drop the old Ranger DB # mysql -u root -pwelcome1
mysql> Drop database old_Ranger_name; The above steps should resolve your Ranger issue. Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, Kudos this answer by hitting the thumbs up button.
... View more
01-04-2021
12:55 PM
@ibrahima This community helps in 2 of the most used Hadoop flavors Cloudera and Hortonworks and these 2 software vendors handled and configured differently their Kerberos. In cloudera the keytabs are found in /run/cloudera-scm-agent/process/* while in hortonworks it's in /etc/security/keytabs/* so it would be good if you clearly stated. Please include the description of your cluster too like HA or not I see from the log failover to rm16 which suggest you have RM HA? Has the user kinited before attempting the operation. Is user impersonating cabhbwg Happy hadooping
... View more
01-04-2021
09:58 AM
@HoldYourBreath To add to @GangWar answer Azure is your best bet as you want to install Oracle VirtualBox and import your Cloudera Quickstart VM image. Don't forget to set up a Windows 10 with at most 16GB with enough CPU's and remember to set up auto-shutdown to avoid extra costs when your VM isn't running Create-windows-virtual-machine-in-azure How to install windows 10 in Azure Hope this information is useful Happy hadooping
... View more
01-04-2021
09:47 AM
@Mondi The simple answer is YES and the best source is the vendor itself Rack awareness CDP as computations are performed with the assistance of rack awareness scripts. Hope that helps Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, Kudos this answer by hitting the thumbs up button.
... View more