Member since
01-19-2017
3598
Posts
593
Kudos Received
359
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
119 | 10-26-2022 12:35 PM | |
273 | 09-27-2022 12:49 PM | |
341 | 05-27-2022 12:02 AM | |
276 | 05-26-2022 12:07 AM | |
467 | 01-16-2022 09:53 AM |
01-17-2023
01:11 PM
@admin007 How are you trying to connect? Can yo share the error ? When you are connecting to an impalad running on the same machine the prompt will reflect the current hostname. $ impala-shell If you are connecting to an impalad running on a remote machine, and impalad is listening on a non-default port [21000] $ impala-shell -i some.other.hostname:port_number Hope that starts the conversation
... View more
01-11-2023
11:00 AM
@mike_bronson7 Hadoop uses the attribute dfs.hosts.exclude in hdfs-site.xml as a pointer to a file where node exclusions should be adequately documented. Since there is no default value for this attribute, the Hadoop cluster will not exclude any nodes in the absence of a file location and a file in the absence of dfs.hosts.exclude If dfs.hosts.exclude is not set in your cluster, take the actions listed below. Shutdown the Namenode. Edit hdfs-site.xml and add a dfs.hosts.exclude entry with the file's location. This can be a text file with the hostname that you intend to remove should be added to the file described in dfs.hosts.exclude. Start the namenode Add the hostname to the file specified in dfs.hosts.exclude that you intend to remove when dfs.hosts.exclude is already configured. Run the following command to exclude a data node After adding the hostname to the exclusion run the below command to exclude the node from functioning as a Datanode after adding the hostname to the exclusion. $ hdfs dfsadmin -refreshNodes Below command will exclude the node from functioning as a Node Manager $ yarn rmadmin -refreshNodes After the above actions, you should see one data node marked as decommissioned in Ambari. No data blocks will be sent to this data node as YARN has already marked it as unusable Hope that answers your question
... View more
11-12-2022
01:09 PM
@hassan-ki5 This looks a typical CM database connection issue can you check and compare the entries in cat /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.type=[Oracle/mysql/postgresql] com.cloudera.cmf.db.host=localhost com.cloudera.cmf.db.name=scm com.cloudera.cmf.db.user=scm com.cloudera.cmf.db.setupType=EXTERNAL com.cloudera.cmf.db.password=scm Ensure the DB.password, name, and user are correct since you seem to be running Mysql can you check this page CM using Mysql
... View more
11-12-2022
12:56 PM
@yomz Can you adjust the below parameters restart Postgres and restest pg_hba.conf # TYPE DATABASE USER ADDRESS METHOD host all all 0.0.0.0/0 md5 Locate the postgresql.conf and add the below entry password_encryption = md5 Please let me know
... View more
11-04-2022
03:36 PM
@lysConsulting Are you using embedded DB? if not can you log in to the HUE database from the CLI?
... View more
11-03-2022
03:50 PM
@lysConsulting This looks like a driver issue can you run the below it would affect your ambari server ambari-server setup --jdbc-db=postgres --jdbc-driver=/path/to/postgres/postgresql.jar Then retry the earlier steps or just create HUE database manually then with the credentials connect through the HUE service setup
... View more
10-31-2022
06:00 AM
@Ninja I see a difference in the networking can you change that and retry
... View more
10-30-2022
01:19 PM
@Ninja downloaded the 2.6.5 and there is no difference in the steps above. Please let me know if you still need help
... View more
10-30-2022
06:10 AM
@Ninja Your problem is resolved and here are the steps Downloaded the images in the link H DP3.0.1 extracted successfully Virtualbox setup ensure you enable only bridged adapter and deselect the NAT on Adapater1 gave my sandboy 10 GB plus 4 CPU's Extraction of the HDP it really takes a while Extraction Post extraction you are presented with this screen note the ssh IP linked to the bridge adapter IP Opened the web host CLI using 192.168.0.103 IP in my case your could be different you will be prompted for root default password hadoop and immediately forced to change the default password to something stronger . After successfully changing then you and now in business at the root prompt type the below # ambari-admin-password-reset The above will reset the ambari password I went the simple way with admin/admin when you see that ambari is listening to port 8080 then you are done Using Chrome opened the Ambari UI Using the password set above admin/admin in my case open the Ambari web UI Logged in successfully There you go
... View more
10-27-2022
04:48 AM
@Ninja Cool send me the link I will download it and give you the steps.
... View more
10-26-2022
12:35 PM
@drewski7 Ranger plugins that use Ranger as an authorization module will cache local policy and use the same for authorization purposes Ranger plugins cache the tags and periodically poll the tag store for any changes. When a change is detected, the plugins update the cache. In addition, the plugins store the tag details in a local cache file – just as the policies are stored in a local cache file. When the component restarts, the plugins will use the tag data from the local cache file if the tag store is not reachable. At periodic intervals , a plugin polls the Ranger Admin to retrieve the updated version of policies. The policies are cached locally by the plugin and used for access control The Policy evaluation and policy enforcement happens within the service process. The heart of this processing is the “Policy Engine”. It uses a memory resident-cached set of policies. Ranger takes 30secs to refresh policies check the "Plugin" option in the ranger UI but you can change the refresh time In Ambari UI->HDFS->Services->Configs->"Advance ranger-hdfs-security" you can change the poll interval here[refresh time]. Geoffrey
... View more
10-26-2022
12:16 PM
@Ninja is this a sandbox if yes can you share the download link? Have you tried the below and restarting ambari? # Update password ambari-admin-password-reset # If Ambari doesn't restart automatically, restart ambari service ambari-agent restart Please let me know Geoffrey
... View more
10-03-2022
12:24 PM
@Venkatd Can you share the walkthrough of what you have already done and the errors you are encountering? It will be easier to help you from that standpoint and don't forget to mention the versions of HDP/Atlas
... View more
10-02-2022
10:29 AM
@imule Firstly I would like to ask if there were any changes in the cluster i.e Patching or rpm ? if the spark-submit was running successfully before you need to know this could be linked to the python version On the edge/gateway node # python -V # Python 3.7.7 # conda deactivate # Python 2.7.5 Then try relaunching your spark-submit
... View more
09-27-2022
12:49 PM
@MISAKIGA Have you had a look at this configuring Kerberos using restAPI
... View more
09-21-2022
05:08 AM
@dmharshit The 2 SQL inserts are not identical the latter only updates the password while the former user and password. Prior to your update did your backup the table? Geoffrey
... View more
09-17-2022
01:10 PM
@dmharshit To resolve this issue, you will have to update Ranger’s backend database directly to reset the password you will need to write access to Ranger’s backend database. Using MySQL you know how to log into MySQL from either the command line The users who can access Ranger’s Admin Web UI, also call Portal, are stored in a table called “x_portal_user“. mysql> show tables; +------------------------------+ | Tables_in_ranger | +------------------------------+ | vx_trx_log | | x_access_type_def | .... | x_portal_user | | x_portal_user_role | ... +------------------------------+ 67 rows in set (0.00 sec) Query this table to confirm your Admin user is present in the table: mysql> SELECT id, first_name, last_name, login_id, password, status \ FROM x_portal_user WHERE login_id = 'admin'; +----+------------+-----------+----------+----------------------------------+--------+ | id | first_name | last_name | login_id | password | status | +----+------------+-----------+----------+----------------------------------+--------+ | 1 | Admin | | admin | fdea9c9f5fabb9c4120b9834684f0ff0 | 1 | +----+------------+-----------+----------+----------------------------------+--------+ 1 row in set (0.00 sec) The password corresponds to the hashed value of “fdea9c9f5fabb9c4120b9834684f0ff0”, reset it to default value of “admin”, which has the hash value of “ceb4f32325eda6142bd65215f4c0f371”. Please run the below query: UPDATE x_portal_user SET password = 'ceb4f32325eda6142bd65215f4c0f371' WHERE login_id = 'admin' ; After that, just log into Ranger Web UI using “admin” as username and “admin” as password, then update the password IMMEDIATELY through web UI. Hope that helps Geoffrey
... View more
09-01-2022
10:57 AM
@hebamahmoud You first need to integrate Kudu with HMS I would advise you to read the whole document Geoffrey
... View more
06-09-2022
06:10 AM
@vkotsas First, you need to establish communication between the 2 [server01 and server02] to ensure the DNS can resolve, ssh is okay and Kerberos cross-realm is working if the clusters are kerberized. Next copy hdfs-site.xml and core-site.xml from Hadoop server01 to Nifi server02 note the paths as you will need that information during the setup, in the Nifi perspective, the core-site.xml and hdfs-site.xml has all the necessary Hadoop connection information needed. Use the FetchHDFS and not GetHDFS as the latter deletes the source, this skeleton procedure should help you fine-tune your HDFS copy from server01 to Server02 please let me know if you need more help? Geoffrey
... View more
06-06-2022
11:07 AM
@FLYs Can you have a look at this solution it could be a classpath issue. Sqoop error
... View more
05-31-2022
01:00 PM
@andrea_pretotto Did you use the -skipTrash option during the deletion?
... View more
05-27-2022
04:23 AM
1 Kudo
@RZ0 Ambari or Cloudera Manager is an admin UI to make our lives easier. Yes you can run a cluster without Ambari because all the admin scripts [start,stop, status] and Ambari/Ranger/Atlas use API's so the effort will be in the scripting. You will miss the Metrics in one place option of Ambari UI Hope that answers
... View more
05-27-2022
12:02 AM
1 Kudo
@novice_tester This is the answer Hue query location
... View more
05-26-2022
11:58 PM
@Nifi_Noob Followed this link for creation is not a usable link. Can you update. Geoffrey
... View more
05-26-2022
12:40 AM
@Love_Cat Can you share the steps before you invoke Ambari? Did you install the agent and edit the below portion of the ambari-agent.ini? [server]
hostname=<Current_Ambari_host>
url_port=8440
secured_url_port=8441
connect_retry_delay=10
max_reconnect_retry_delay=30 And restarted the agent thereafter this should show in the ambari UI in the Add host Wizard assuming you generated and copied the SSH Private key with the correct permissions Please share your steps so we could help you resolve
... View more
05-26-2022
12:27 AM
@akhan001 For Question 1: see S3 with ranger
... View more
05-26-2022
12:07 AM
@George-Megre The Master nodes firstly are not meant for launching tasks or interaction. Edge or gateway nodes are used to run client applications and cluster administration tools. Setup of Edge/gateway node is similar to any Hadoop node except no Hadoop cluster services runs on the GW/Edge nodes they are mere entry points and connection gateway to the Master Components like HDFS (NN) HBASE etc provide you have installed the client libraries. In your case, I am sure you have the Hbase client/gateway roles on the 3 nodes and not on the master noded. The HBase client role gives the connectivity to Hbase but again I don't see why you would like to connect /initiate the HBase shell from the master node? Geoffrey
... View more
05-25-2022
11:28 PM
@TonyQiu Sorry been away for a while. Creating a table in a kerberized cluster should be possible as hbase. I just tried it out see the flow below.. I didn't need to specifically kinit as the user hbase had a valid Kerberos ticket Switch to hbase user [root@malindi keytabs]# su - hbase
Last login: Wed May 25 21:51:56 CEST 2022 Check if kerberos ticket is valid [hbase@malindi ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1016
Default principal: hbase-jair@KENYA.KE
Valid starting Expires Service principal
05/25/2022 21:51:57 05/26/2022 21:51:57 krbtgt/KENYA.KE@KENYA.KE Connect to hbase shell as hbase [hbase@malindi ~]$ hbase-shell List existing tables hbase(main):006:0> list
TABLE
ATLAS_ENTITY_AUDIT_EVENTS
atlas_titan
2 row(s) in 0.6660 seconds
=> ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"] Create table emp hbase(main):007:0> create 'emp', 'personal data', 'professional data'
0 row(s) in 4.9500 seconds
=> Hbase::Table - emp List tables validate emp was created hbase(main):009:0> list
TABLE
ATLAS_ENTITY_AUDIT_EVENTS
atlas_titan
emp
3 row(s) in 0.0130 seconds
=> ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan", "emp"] Describe table emp hbase(main):010:0> describe 'emp'
Table emp is ENABLED
emp
COLUMN FAMILIES DESCRIPTION
{NAME => 'personal data', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS
=> '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
{NAME => 'professional data', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERS
IONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
2 row(s) in 0.8770 seconds Update table hbase(main):011:0> alter 'emp', {NAME => 'metrics', BLOCKSIZE => '16384', COMPRESSION => 'SNAPPY'}
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 4.9660 seconds Can you share your snippet?
... View more
03-30-2022
12:53 PM
@iamfromsky True let me check on that and revert on the config. Can you share your integration steps or document?
... View more
03-30-2022
11:38 AM
@iamfromsky The path you are mentioning has permissions issues. As the root user can you # chmod 777 /run/cloudera-scm-agent/process/9392-yarn-NODEMANAGER/creds.localjceks Then retry if that's successful then fine-tune the permissions . Hope that helps
... View more