Member since
03-25-2020
132
Posts
4
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2192 | 09-21-2021 08:41 PM | |
2361 | 06-07-2021 05:11 AM | |
3844 | 06-06-2021 10:34 PM |
01-31-2024
09:39 AM
1 Kudo
I'm not sure if we have example scripts or custom functions, but the general idea to achieve what you're trying to do would be - 1. Referring the HDFS reporting in CM or parse the fsimage (for large files). 2. Write a MapReduce/Spark job to scan files, or run a query in Hive/Impala/SparkSQL to see if data files are mapped to Hive tables. You can also implement access controls through Ranger, enable transparent disk encryption (TDE) using Ranger KMS, tag sensitive hive columns (containing PII, PCI, PHI) using Atlas classification and assign tag-based masking policies from Ranger, implement navencrypt to encrypt the spill files, et. - if you're planning to protect the sensitive data from unauthorised access.
... View more
01-30-2024
02:33 AM
1 Kudo
@phir1 , Can you give us more context on the use-case, please? What type of monitoring you're planning to implement? And, by Hue directory, which path are you referring to?
... View more
09-21-2021
08:41 PM
1 Kudo
@vorraluck, Following is the error I see, which points to the parameter 'SamplePassword123'. You're missing "=" sign in this parameter as the error hints. # openssl req -new -sha256 -nodes -newkey rsa:2048 -config /tmp/openssl.cnf
error on line 13 of /tmp/openssl.cnf
140163633919888:error:0E079065:configuration file routines:DEF_LOAD_BIO:missing equal sign:conf_def.c:345:line 13 Change SamplePassword123 to SamplePassword = 123. Once the change is made, both the private key and the CSR file can be created. [root@node1 ~]# openssl req -new -sha256 -nodes -newkey rsa:2048 -config <(
>
> cat <<-EOF
>
> [req]
>
> default_bits = 2048
>
> prompt = no
>
> default_md = sha256
>
> req_extensions = req_ext
>
> distinguished_name = dn
>
> SamplePassword = 123
>
> [ dn ]
>
> C=TH
>
> ST=Bangkok
>
> L=Chaxxx
>
> O=xtac Public Company Limited
>
> OU=Enterprise Service Support
>
> emailAddress=john.doe@xtac.co.th
>
> CN = rs-xxx-hmb-201.xtac.dev
>
>
>
> [ req_ext ]
>
> subjectAltName = @alt_names
>
>
>
> [ alt_names ]
>
> DNS.1 = rs-xxx-hmb-201
>
> DNS.2 = rs-xxx-hmb-201.xtac.dev
>
>
>
> EOF
>
> )
Generating a 2048 bit RSA private key
.+++
.....................................................................................................................................................................................................................+++
writing new private key to stdout
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC0PnXQOdpbKGso
wNY7K+pYItm3G4/yL7HYjViUzxQsEZasIypZrYIY8KqF64bndEf002YkaHcEDCFR
xr59eBfpmXYWezl89ZeRU2YEL5188ubSrkdo/P0IdBSGAbu1/2WlQ26AyIHTkG96
ppsOa25vbKPdOmPh37ylB0C1ghCk6utib5/HAiIt0X9YqGERfnTVXpo/fRzxSHdb
7eioxFx0CP38Vzo/wXQP2FDwa3bw0MTNNF813ttC0yzRpcFf4ACde2jzFpehgtY0
E3yvq7/gVURibZgg6MSVVrg/dzjlReiRFxQOO4XDcuNn1VLSJ23UOT4wXwszKIe2
e7nNPtb5AgMBAAECggEAEhorzuvgiEM47/DeEzdH4+4sG33DKTmtDOi1OszJY9uo
XEVz3WnVpReWqLiM5fYBvA73NTTgryFgv0vMmeFT1Xw6JeveTDSGMmxD1KvLj179
267xMRQnfY/a99J4vcCWvtBU6s44a70X1SOicwJLJxmsI23X1jmYQqLF1vTnIwjh
of6e6jQWeE1nyrhHctqlyYfFNat6uK8NfJ21PTb4yfjp5dhefgaIK/9qOB9SKMwk
LFXLGVlePBFziadY8spbRl56mcbEC6mgbXff4Zbzy5QhXxQaN38358G5fSGpsgiQ
44rTFrC2PTPzS1HeGpqdAwhqFU5lEXnMVegS43ZeAQKBgQDrngKDgKnXY/HBO9E0
OAzC2j+O6BBalPl65f9l6rhiZo8QwhcJciMyCkgnDxnXb/FkKcBmLx+QJXjYguQ4
zkXMEqCS5Bz+jrwnQrnAh/TIlKyZPxy2qpJUvUJND8DZ7ZMVmBycOPI4gFjoIx/G
U0kPPdgHhdRtL/Uv2XsXBM4/4QKBgQDD1ibpDuvAd2x1EKAEWwj3v/Ee41am+Q9Z
WjFWDmS7jV8EYPqOrYedKyaY0t5ArOOKUHRiOp0VOsTWF7m7NAzwmvdspV0YeShh
qCPuWWOGBATjVOMmPgqKBUn3U67y7pZRX6tpOlcwj8sB6KQRpm9NpLcMSFrDWxEz
oCgLHVXaGQKBgQDQirxC1GB/SfCyTVVvWKTC2hUjUXcYFX9zLZsOA+BLB+dct81Y
CPPp2HvgRldi/au0MdgfGVpgZSo+yCtjs/7HDz2chda74G3cegyawjsARcc2pEuv
ye1Wn2TNfEH/IW3r0QSRqT2KkN8gJ+Z5zUF/AgfxMJzCP45OWbm+t/wtQQKBgCou
Yksor5bRIkdEwXKuuQvECAeDKBLm6mtwhdfnWcMb/C9RRCafMeqkdLfOE5kSpCAS
nD7cKSF9exAyJAsyducMOebo51hyIESIltSr5EflbbgZfKOsEVEROpFPMQuaYD4+
wQj1S/plvnA2z8ANfUPYCqVWoFYbs2TPRlC+jNNhAoGAPgGiUz+4lDvQL62R6xDP
R8OUGhmfB37nQy4NFNaiRxQW9TSdrg09cb3CS04IjyH9QQmetRyhZSXCYCu7H/bf
H/rR5S7/e5WKgtgAIpNhYNXg1p6zoac6cXBYUO4ekf/PR70cFXAQdzFt/sQIExiy
SV6SL1F3ja/sn9/wUXkvZgc=
-----END PRIVATE KEY-----
-----
-----BEGIN CERTIFICATE REQUEST-----
MIIDTDCCAjQCAQAwgcExCzAJBgNVBAYTAlRIMRAwDgYDVQQIDAdCYW5na29rMQ8w
DQYDVQQHDAZDaGF4eHgxJDAiBgNVBAoMG3h0YWMgUHVibGljIENvbXBhbnkgTGlt
aXRlZDEjMCEGA1UECwwaRW50ZXJwcmlzZSBTZXJ2aWNlIFN1cHBvcnQxIjAgBgkq
hkiG9w0BCQEWE2pvaG4uZG9lQHh0YWMuY28udGgxIDAeBgNVBAMMF3JzLXh4eC1o
bWItMjAxLnh0YWMuZGV2MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
tD510DnaWyhrKMDWOyvqWCLZtxuP8i+x2I1YlM8ULBGWrCMqWa2CGPCqheuG53RH
9NNmJGh3BAwhUca+fXgX6Zl2Fns5fPWXkVNmBC+dfPLm0q5HaPz9CHQUhgG7tf9l
pUNugMiB05BveqabDmtub2yj3Tpj4d+8pQdAtYIQpOrrYm+fxwIiLdF/WKhhEX50
1V6aP30c8Uh3W+3oqMRcdAj9/Fc6P8F0D9hQ8Gt28NDEzTRfNd7bQtMs0aXBX+AA
nXto8xaXoYLWNBN8r6u/4FVEYm2YIOjElVa4P3c45UXokRcUDjuFw3LjZ9VS0idt
1Dk+MF8LMyiHtnu5zT7W+QIDAQABoEUwQwYJKoZIhvcNAQkOMTYwNDAyBgNVHREE
KzApgg5ycy14eHgtaG1iLTIwMYIXcnMteHh4LWhtYi0yMDEueHRhYy5kZXYwDQYJ
KoZIhvcNAQELBQADggEBAH9kfI4ZCJRDF2lKjCjLgj/UaRpdO9yIcWUsQHU6jpSU
meKpjkDgn8zTB8hTJ8C8K5rIp1BbGB8Oe/ppdbgzmIKPLjEKUGxIIkBMdSnPRmAg
BUy+nArMmVGrv1weA7F1E/ECzdR1NeXl6eWLYDbmHIwT1/dkpL7evCE8zYufQb9u
H1/U/zzyWiiOuneL+GzhCh7srQkQCtjiWrnuhjZnANvRSlR+FIYY7PWlY1e2m7go
kL5CRVbSUZaYvknIAgOBJs2fkvyrR+YFlKqzlhN3i9yXdfAC09D0kr0FE9CUedh1
HJsxwMB5S1L2zZXAS0iNqoTTb0wHi3UbKLJIMeQx0RU=
-----END CERTIFICATE REQUEST-----
[root@node1 ~]# Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
08-07-2021
07:41 PM
1 Kudo
Hue is unable to handle the slow responses from Hive servers and hence the error. This happens when Hive is unable to return the query handle to Hue in a timely manner. So, the ideal solution here would be to investigate Hive slowness and not Hue. Hue relies on server_conn_timeout parameter in its configuration (default 120s), after which the active sessions with hiveserver2 is teared down. If Hue is using MySQL as the underlying DB, then we can tweak the innodb-lock-wait-timeout to be greater than the server_conn_timeout in Hue's configuration, that way we get fail-fast system and the locked tables are released. Please note that this is only a workaround until the cause of Hive performance issue is found and remediated.
... View more
06-07-2021
05:11 AM
Hello @Amn_468, That's quite strange, as the service should generally look for the log file before starting. The only possibility that I can think of here is that the log file would have been removed manually while the service was running. Having said that, yes, let's try creating the log file manually with appropriate permissions as suggested by @nthomas, and restart Hue kt_renewer service to see if it is able to write to the newly created file.
... View more
06-06-2021
10:34 PM
@PrernaU The below exception hints that 'hdfs' user is not authorised to submit jobs in the Yarn queue 'default', and this is due to ACL in place. Caused by: org.apache.hadoop.security.AccessControlException: User hdfs does not have permission to submit application_1622150236559_0013 to queue default Is ACL managed by Ranger please? If so, we will need to add 'hdfs' user to have complete access over Yarn queue 'default' under Yarn policies in Ranger UI.
... View more
06-02-2021
03:42 AM
Hello @Amn_468 , Are we setting up Hue KT_renewer for the first time? What is the status of Hue kt_renewer service please? stderr logs from Hue kt_renewer can give more insights. The below commands can take us to the directory were stderr log is stored. # export HUE_DIR="/var/run/cloudera-scm-agent/process/`ls -alrt /var/run/cloudera-scm-agent/process | grep hue-KT_RENEWER | tail -1 | awk '{print $9}'`" # cd $HUE_DIR/logs
... View more
03-29-2021
09:47 PM
Hello @srinikar87 The error indicates that your KDC server requires clients to pre-authenticate themselves before it can issue a Ticket Granting Ticket (TGT). If your KDC is a MIT kerberos, then probably a command line argument was passed by your administrator to require all clients to use pre-authentication. In this case, we can run the following command on your KDC server which will disable pre-authentication only for the impala principal, and the generate missing credentials should work. kadmin.local: modprinc -requires_preauth impala/e2e-02-cdlkc1.nokia.com@NOKIA.COM If your KDC is an AD server, then your AD administrator must enable the “Do not require Kerberos pre-authentication” checkbox in the user properties of the newly created impala principal. Refer the attachment. Please let us know how this goes. Reference: http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-admin/Adding-or-Modifying-Principals.html Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
02-26-2021
07:18 AM
@nj20200 As @GangWar mentioned, this is more of a yum related issue. Cloudera manager requires openssl, openssl-libs, and openssl-devel packages to be installed. It appears your yum repo server does not contain openssl-devel-1.0.2k-19.el7.x86_64.rpm, and hence the error. Once your OS team or the team that manages the repo server is able to sync or copy the required openssl rpms from satellite server, you should be able to install cloudera manager package without any issue.
... View more
12-17-2020
06:52 PM
Hello @prabhat10 , Try this - Backup your /etc/krb5.conf on all the hosts Verify the encryption types supported from your Kerberos server (If MIT - Check "supported_enctypes" in /var/kerberos/krb5kdc/) Check the "Kerberos Encryption Types" under CM > Administration > Security > Kerberos Credentials > Configuration. Include the encryption types supported by your KDC. Enable "Manage krb5.conf through Cloudera Manager" from the same configuration page. Select "Deploy Kerberos client configuration" from the drop-down near your cluster. Once deployed, verify if the krb5.conf on the agent nodes have the encryption types included as mentioned in CM. If CM server is running on stale kerberos configuration, copy the krb5.conf from one of the agent nodes to CM server. Regenerate the principals from CM. (If this is success, you should be able to restart CM and CDH services).
... View more