Member since
01-16-2019
73
Posts
5
Kudos Received
4
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1458 | 12-08-2021 06:08 AM | |
| 1470 | 09-23-2021 04:25 AM | |
| 4916 | 01-06-2021 12:45 AM | |
| 2367 | 03-28-2019 02:01 PM |
12-08-2021
06:08 AM
I understand that you are trying to hide the values in the non-relevant fields in the Hue UI. For ex: _version_ But, this would not be possible by making changes in the managed-schema (or schema.xml) of a Solr collection. Changing a schema would be changing the behaviour of how the indexing will happen in the Solr service for the collection. Adding to the use case, Hue, will only show what is indexed by solr and given back. It apparently, has very less control over the actual data. It is sort of proxy between you and Solr. The one who is just shows what Solr service gives. For "stored=false" & "indexed=false": When you set "indexed=false" -> You might totally skip the indexing feature for that particular field.So, you will never be able to search a document in Solr with that field detail. Ex: id. When you set "stored=false" -> The purpose of the stored attribute: to tell Solr to store the original text in the index somewhere. Sometimes, there are fields which aren't searched, but need to be displayed in the search results. I hope this answers your questions.
... View more
09-29-2021
10:12 PM
Pre-requisites:
1. You need to have the correct passwords 2. The PrivateKey of the CA is required to sign off all the agent certificates. 3. Password for the pass.txt file will be present in clear-text format. And currently, there is no way to encrypt it.
Steps:
Back up and delete all the crt and csr files that start with hostname at /var/lib/ambari-server/keys. tar -cvzf keys_directory.tar.gz /var/lib/ambari-server/keys/*
Empty /var/lib/ambari-server/keys/db/index.txt file mv /var/lib/ambari-server/keys/db/index.txt /tmp
Delete any certificates under /var/lib/ambari-server/keys/db/newcerts/ mv /var/lib/ambari-server/keys/db/newcerts/* /tmp
Configure CA-signed certificate artifacts in Ambari Server by:
Copy the CA-signed certificate, key files to /var/lib/ambari-server/keys/ e.g. rootca.crt, root_server.key
Create PKCS keystore file from your certificate and key files. openssl pkcs12 -export -in rootca.crt -inkey /var/lib/ambari-server/keys/root_server.key -certfile rootca.crt -out keystore_CA.p12 -password pass:hadoop1120 -passin pass:hadoop1120
Create a text file with the appropriate password that has been provided for the keystore in /var/lib/ambari-server/keys directory: echo hadoop1120 > pass_CA.txt
Next, configure Ambari to use the new cert, keys, and keystore files.
Edit the /etc/ambari-server/conf/ambari.properties and set the following keys and values: security.server.cert_name=rootca.crt
security.server.key_name=root_server.key
security.server.keystore_name=keystore_CA.p12
security.server.truststore_name=keystore_CA.p12
security.server.crt_pass_file=pass_CA.txt
security.server.two_way_ssl=true
After adding the properties, clean up existing Ambari agent keys information by removing all files in the /var/lib/ambari-agent/keys/ directory on each node that has Ambari agent installed.
Note: Ambari server uses the following command to generate the certificates using the configs above. This command requires the ca.config to be present at the location. (Content of ca.config does not matter):
openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/node1.coelab.cloudera.com.csr -out /var/lib/ambari-server/keys/node1.coelab.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/root_server.key -cert /var/lib/ambari-server/keys/rootca.crt
openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/node2.coelab.cloudera.com.csr -out /var/lib/ambari-server/keys/node2.coelab.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/root_server.key -cert /var/lib/ambari-server/keys/rootca.crt
openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/c2218-node4.coelab.cloudera.com.csr -out /var/lib/ambari-server/keys/c2218-node4.coelab.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/root_server.key -cert /var/lib/ambari-server/keys/rootca.crt
openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/hostname.cloudera.com.csr -out /var/lib/ambari-server/keys/hostname.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/root_server.key -cert /var/lib/ambari-server/keys/rootca.crt
Sample logs from ambari-server.log file after implementation:
[root@node1 ambari-server]# grep "Validating agent hostname:" ambari-server.log -A3
2021-09-22 10:52:46,862 INFO [qtp-ambari-agent-222] CertificateManager:238 - Validating agent hostname: hostname.cloudera.com
2021-09-22 10:52:46,862 INFO [qtp-ambari-agent-222] CertificateManager:250 - Verifying passphrase
2021-09-22 10:52:46,891 INFO [qtp-ambari-agent-222] ShellCommandUtil:63 - Command openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/hostname.cloudera.com.csr -out /var/lib/ambari-server/keys/hostname.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/rootca_PKey.key -cert /var/lib/ambari-server/keys/rootca.crt was finished with exit code: 0 - the operation was completely successfully.
2021-09-22 10:52:47,522 INFO [agent-register-processor-0] HeartBeatHandler:321 - agentOsType = centos7
--
2021-09-22 10:52:51,929 INFO [qtp-ambari-agent-222] CertificateManager:238 - Validating agent hostname: node4.coelab.cloudera.com
2021-09-22 10:52:51,929 INFO [qtp-ambari-agent-222] CertificateManager:250 - Verifying passphrase
2021-09-22 10:52:51,954 INFO [qtp-ambari-agent-222] ShellCommandUtil:63 - Command openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/hostname.coelab.cloudera.com.csr -out /var/lib/ambari-server/keys/hostname.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/rootca_PKey.key -cert /var/lib/ambari-server/keys/rootca.crt was finished with exit code: 0 - the operation was completely successfully.
2021-09-22 10:52:52,450 INFO [agent-register-processor-1] HeartBeatHandler:321 - agentOsType = centos7
--
2021-09-22 10:52:53,227 INFO [qtp-ambari-agent-222] CertificateManager:238 - Validating agent hostname: hostname.coelab.cloudera.com
2021-09-22 10:52:53,227 INFO [qtp-ambari-agent-222] CertificateManager:250 - Verifying passphrase
2021-09-22 10:52:53,253 INFO [qtp-ambari-agent-222] ShellCommandUtil:63 - Command openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/hostname.cloudera.com.csr -out /var/lib/ambari-server/keys/hostname.cloudera.com.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/rootca_PKey.key -cert /var/lib/ambari-server/keys/rootca.crt was finished with exit code: 0 - the operation was completely successfully.
2021-09-22 10:52:53,683 INFO [agent-register-processor-2] HeartBeatHandler:321 - agentOsType = centos7
Sample logs after successful registration of the ambari-agent with the ambari-server from the ambari-agent.log:
INFO 2021-09-22 10:52:52,880 NetUtil.py:61 - Connecting to https://hostname.cloudera.com:8440/connection_info
INFO 2021-09-22 10:52:52,952 security.py:59 - Server require two-way SSL authentication. Use it instead of one-way...
INFO 2021-09-22 10:52:52,952 security.py:61 - Connecting to wss://hostname.cloudera.com:8441/agent/stomp/v1
INFO 2021-09-22 10:52:52,952 security.py:221 - Server certicate not exists, downloading
INFO 2021-09-22 10:52:52,953 security.py:244 - Downloading server cert from https://c2218-node1.coelab.cloudera.com:8440/cert/ca/
INFO 2021-09-22 10:52:53,019 security.py:229 - Agent key not exists, generating request
INFO 2021-09-22 10:52:53,020 security.py:301 - openssl req -new -newkey rsa -nodes -keyout "/var/lib/ambari-agent/keys/hostname.com.key" -subj /OU=hostname.cloudera.com/ -out "/var/lib/ambari-agent/keys/hostname.com.csr"
INFO 2021-09-22 10:52:53,158 security.py:237 - Agent certificate not exists, sending sign request
INFO 2021-09-22 10:52:53,349 transport.py:329 - Starting receiver loop
INFO 2021-09-22 10:52:53,352 security.py:88 - SSL connection established. Two-way SSL authentication completed successfully.
INFO 2021-09-22 10:52:53,674 HeartbeatThread.py:127 - Sending registration request
... View more
Labels:
09-23-2021
04:25 AM
Hello @jxerome, 1. After you install Atlas in the CDP cluster, you need to configure "Atlas" dependency in Hive on Tez (OR Hive if you use Spark as the execution engine) service, you can do that by following: CM UI > Hive on Tez > Configurations > Check the "atlas" checkbox. This would configure Atlas-Hive hooks for your cluster. EX: 2. After this please restart Atlas and Hive service. 3. For the first time there will be no metadata synced from Hive to Atlas. You need to manually sync all the data using the atlas bridge (import-hive.sh script). This should be run with an admin user who has permissions in Hive as well as Atlas. 4. In your case, if 1. is not configured properly, then metadata will not be synced automatically. Let me know how that goes for you!
... View more
09-23-2021
04:14 AM
Hello @ryu , Please let us know what is the version of the Ambari server that you are running and what was the operation performed? Was it adding the Hbase service? How was it added? Using Ambari APIs OR UI?
... View more
01-06-2021
12:45 AM
3 Kudos
This happens because the "Read Type" permission is absent in the Ranger-Atlas plugin repository for the Atlas service. During the upgrade, a step to upgrade the ranger DB with the patches is skipped. Which is causing this permission to be not present after the CDP upgrade. The update handler needs to call the right script to install the DB patches for ranger which should automatically update the DB patches (during the upgrade process) and fix the issue. Issue is addressed and should be fixed in the upcoming releases of the the Cloudera Manager product line. The workaround for this issue: - Stop the Ranger Admin service. 1. Login in into CM UI -> Ranger -> Instances -> Select Ranger Admin. 2. From the drop down actions menu, select "Upgrade Ranger database and apply patches". 3. This step will manually apply the patches and will finish. 4. Once finished, you can restart the Ranger Admin service. The atlas policy which is missing the permission (the id of policy can be different in your environment): We need something similar to this:
... View more
09-20-2019
04:21 AM
Hello @dvillarreal, Thank you for the script files, makes life easy! Just a small edit: The proc.psql file was missing one SQL statement @line 96: delete from x_user_module_perm where user_id = x_portal_user_id; We were facing the below issue while trying the commands from psql file manually: ERROR: update or delete on table "x_portal_user" violates foreign key constraint "x_user_module_perm_fk_userid" on table "x_user_module_perm"
DETAIL: Key (id)=(19) is still referenced from table "x_user_module_perm" Because of the above blocker, the script was not deleting the users from the DB. Once we added the missing SQL statement and ran the deleteRangerUser.sh we were able to use it with successful results.
... View more
03-28-2019
02:01 PM
1 Kudo
This happens since the "version" fields in the DB remain un-updated. You can follow: 1. Login into Ambari DB. 2. Look out for the "version" column in "hostcomponentstate" table. 3. Update the column with the current active version of your cluster. 4. Restart ambari server process. This is an example from my test cluster: ambari=> select * from hostcomponentstate where component_name='SOLR_SERVER';
id | cluster_id| component_name| version| current_state| host_id| service_name| upgrade_state|security_state
----+-----------+---------------+--------+--------------+--------+-------------+--------------+---------------
100| 2| SOLR_SERVER | UNKNOWN| INSTALLED | 2| SOLR | NONE |SECURED_KERBEROS
(1 row) ambari=> update hostcomponentstate set version = '2.6.5.108-1'
where component_name = 'SOLR_SERVER';
UPDATE 1 ambari=> select * from hostcomponentstate where component_name='SOLR_SERVER';
id |cluster_id| component_name| version |current_state|host_id|service_name|upgrade_state| security_state
----+----------+---------------+-----------+-------------+-------+------------+-------------+---------------
100| 2| SOLR_SERVER |2.6.5.108-1|INSTALLED | 2|SOLR |NONE |SECURED_KERBEROS
(1 row) # ambari-server restart Hope this addresses your concern.
... View more