Member since
08-14-2013
558
Posts
13
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1215 | 10-29-2019 11:47 AM | |
1117 | 07-15-2019 09:02 AM | |
3751 | 06-06-2019 07:56 AM | |
4196 | 02-28-2019 11:34 AM | |
5186 | 01-07-2019 01:08 PM |
01-06-2020
10:59 AM
Hi Robert, thanks for replying! The response rates for the curls you posted are very fast (sub-second), and DNS resolution works just fine root@ip-10-71-8-214:~# ping archive.cloudera.com PING prod.cloudera.map.fastly.net (151.101.188.167) 56(84) bytes of data. 64 bytes from 151.101.188.167: icmp_req=1 ttl=52 time=2.11 ms 64 bytes from 151.101.188.167: icmp_req=2 ttl=52 time=1.70 ms 64 bytes from 151.101.188.167: icmp_req=3 ttl=52 time=1.71 ms 64 bytes from 151.101.188.167: icmp_req=4 ttl=52 time=1.68 ms 64 bytes from 151.101.188.167: icmp_req=5 ttl=52 time=1.70 ms ^C --- prod.cloudera.map.fastly.net ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4005ms rtt min/avg/max/mdev = 1.682/1.784/2.117/0.166 ms root@ip-10-71-8-214:~# root@ip-10-71-8-214:~# root@ip-10-71-8-214:~# dig archive.cloudera.com ; <<>> DiG 9.8.1-P1 <<>> archive.cloudera.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64476 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;archive.cloudera.com. IN A ;; ANSWER SECTION: archive.cloudera.com. 5 IN CNAME prod.cloudera.map.fastly.net. prod.cloudera.map.fastly.net. 19 IN A 151.101.188.167 ;; Query time: 0 msec ;; SERVER: 10.71.0.2#53(10.71.0.2) ;; WHEN: Mon Jan 6 18:47:41 2020 ;; MSG SIZE rcvd: 96 Just judging by the output of strace -fs 256 apt-get update, I actually suspect that the issue is with the version of apt-get that precise uses. I should have mentioned in the first post that this issue doesn't happen on trusty or later. If we look at the output of strace, starting from when apt-get starts hanging and just saying "Waiting for headers...", we see: [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336552, 341881}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336552, 842608}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336553, 343340}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000} <unfinished ...> [pid 22048] <... select resumed> ) = 0 (Timeout) [pid 22048] close(3) = 0 [pid 22048] read(0, 0x7ffc5bc919a0, 64000) = -1 EAGAIN (Resource temporarily unavailable) [pid 22048] close(4294967295) = -1 EBADF (Bad file descriptor) [pid 22048] write(1, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/\nMessage: Connecting to archive.cloudera.com (151.101.188.167)\n\n", 177) = 177 [pid 22048] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3 [pid 22048] fcntl(3, F_GETFL) = 0x2 (flags O_RDWR) [pid 22048] fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0 [pid 22048] connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("151.101.188.167")}, 16) = -1 EINPROGRESS (Operation now in progress) [pid 22048] select(4, NULL, [3], NULL, {120, 0} <unfinished ...> [pid 22037] <... select resumed> ) = 1 (in [9], left {0, 342725}) [pid 22037] read(9, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/\nMessage: Connecting to archive.cloudera.com (151.101.188.167)\n\n", 64000) = 177 [pid 22037] gettimeofday({1578336553, 501069}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000} <unfinished ...> [pid 22048] <... select resumed> ) = 1 (out [3], left {119, 998086}) [pid 22048] getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 [pid 22048] stat("/var/lib/apt/lists/partial/archive.cloudera.com_cm5_ubuntu_precise_amd64_cm_dists_precise-cm5.4.5_contrib_i18n_Index", 0x7ffc5bca0d80) = -1 ENOENT (No such file or directory) [pid 22048] open("/etc/apt/auth.conf", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 22048] open("/etc/apt/auth.conf", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 22048] write(1, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/\nMessage: Waiting for headers\n\n", 144) = 144 [pid 22048] select(4, [0 3], [3], NULL, {120, 0}) = 1 (out [3], left {119, 999997}) [pid 22048] write(3, "GET /cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/ HTTP/1.1\r\nHost: archive.cloudera.com\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\nAccept: text/*\r\nUser-Agent: Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.16)\r\n\r\n", 240) = 240 [pid 22048] select(4, [0 3], [], NULL, {120, 0} <unfinished ...> [pid 22037] <... select resumed> ) = 1 (in [9], left {0, 497912}) [pid 22037] read(9, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/\nMessage: Waiting for headers\n\n", 64000) = 144 [pid 22037] gettimeofday({1578336553, 503490}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000} <unfinished ...> [pid 22048] <... select resumed> ) = 1 (in [3], left {119, 988654}) [pid 22048] read(3, "HTTP/1.1 404 Not Found\r\nServer: Varnish\r\nRetry-After: 0\r\nContent-Type: text/html\r\nAccept-Ranges: bytes\r\nContent-Length: 739\r\nAccept-Ranges: bytes\r\nDate: Mon, 06 Jan 2020 18:49:13 GMT\r\nVia: 1.1 varnish\r\nConnection: keep-alive\r\nX-Served-By: cache-pao17445-PA"..., 65536) = 1070 [pid 22048] gettimeofday({1578336553, 514918}, NULL) = 0 [pid 22048] read(3, 0x95fb0e, 64466) = -1 EAGAIN (Resource temporarily unavailable) [pid 22048] write(1, "400 URI Failure\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/\nMessage: 404 Not Found \nFailReason: HttpError404\n\n", 170) = 170 [pid 22048] open("/dev/null", O_RDWR) = 4 [pid 22048] fcntl(4, F_SETFD, FD_CLOEXEC) = 0 [pid 22048] select(5, [0 3], [4], NULL, {120, 0}) = 1 (out [4], left {119, 999997}) [pid 22048] write(4, "<!DOCTYPE html>\n<html>\n<head>\n<style>\n.error {\n margin: 0 auto;\n text-align: center;\n}\n\n.error-code {\n bottom: 60%;\n color: #2d353c;\n font-size: 96px;\n line-height: 100px;\n}\n\n.error-desc {\n font-size: 12px;\n color: #647788;\n}\n\n.m-b-10 {\n margin-bo"..., 739) = 739 [pid 22048] close(4) = 0 [pid 22048] select(1, [0], NULL, NULL, NULL <unfinished ...> [pid 22037] <... select resumed> ) = 1 (in [9], left {0, 488432}) [pid 22037] read(9, "400 URI Failure\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index/\nMessage: 404 Not Found \nFailReason: HttpError404\n\n", 64000) = 170 [pid 22037] stat("/var/lib/apt/lists/archive.cloudera.com_cm5_ubuntu_precise_amd64_cm_dists_precise-cm5.4.5_contrib_i18n_Translation-en%5fUS", 0x7ffe0c0db4b0) = -1 ENOENT (No such file or directory) [pid 22037] write(1, "Ign http://archive.cloudera.com precise-cm5.4.5/contrib TranslationIndex\n", 73Ign http://archive.cloudera.com precise-cm5.4.5/contrib TranslationIndex ) = 73 [pid 22037] gettimeofday({1578336553, 515779}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [14], NULL, {0, 500000}) = 1 (out [14], left {0, 499996}) [pid 22037] write(14, "600 URI Acquire\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2\nFilename: /var/lib/apt/lists/partial/archive.cloudera.com_cm5_ubuntu_precise_amd64_cm_dists_precise-cm5.4.5_contrib_i18n_T"..., 311 <unfinished ...> [pid 22048] <... select resumed> ) = 1 (in [0]) [pid 22048] read(0, "600 URI Acquire\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2\nFilename: /var/lib/apt/lists/partial/archive.cloudera.com_cm5_ubuntu_precise_amd64_cm_dists_precise-cm5.4.5_contrib_i18n_T"..., 64000) = 311 [pid 22048] stat("/var/lib/apt/lists/partial/archive.cloudera.com_cm5_ubuntu_precise_amd64_cm_dists_precise-cm5.4.5_contrib_i18n_Translation-en%5fUS", 0x7ffc5bca0cc0) = -1 ENOENT (No such file or directory) [pid 22048] open("/etc/apt/auth.conf", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 22048] open("/etc/apt/auth.conf", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 22048] read(0, 0x7ffc5bc919a0, 64000) = -1 EAGAIN (Resource temporarily unavailable) [pid 22048] write(1, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2\nMessage: Waiting for headers\n\n", 159) = 159 [pid 22048] select(4, [0 3], [3], NULL, {120, 0}) = 1 (out [3], left {119, 999998}) [pid 22048] write(3, "GET /cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2 HTTP/1.1\r\nHost: archive.cloudera.com\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\nUser-Agent: Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.16)\r\n\r\n", 239) = 239 [pid 22048] select(4, [0 3], [], NULL, {120, 0} <unfinished ...> [pid 22037] <... write resumed> ) = 311 [pid 22037] gettimeofday({1578336553, 516518}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 1 (in [9], left {0, 499997}) [pid 22037] read(9, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2\nMessage: Waiting for headers\n\n", 64000) = 159 [pid 22037] gettimeofday({1578336553, 516663}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000} <unfinished ...> [pid 22048] <... select resumed> ) = 1 (in [3], left {119, 997859}) [pid 22048] read(3, "HTTP/1.1 302 /cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2/\r\nServer: Varnish\r\nRetry-After: 0\r\nLocation: /cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2/\r\nVia: 1.1 varnish\r\nConten"..., 64466) = 513 [pid 22048] read(3, 0x95fd0f, 63953) = -1 EAGAIN (Resource temporarily unavailable) [pid 22048] open("/dev/null", O_RDWR) = 4 [pid 22048] fcntl(4, F_SETFD, FD_CLOEXEC) = 0 [pid 22048] close(4) = 0 [pid 22048] write(1, "103 Redirect\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2\nNew-URI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2/\n\n", 255) = 255 [pid 22048] read(0, 0x7ffc5bc919a0, 64000) = -1 EAGAIN (Resource temporarily unavailable) [pid 22048] write(1, "102 Status\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2/\nMessage: Waiting for headers\n\n", 160) = 160 [pid 22048] select(4, [0 3], [], NULL, {120, 0} <unfinished ...> [pid 22037] <... select resumed> ) = 1 (in [9], left {0, 497715}) [pid 22037] read(9, "103 Redirect\nURI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2\nNew-URI: http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Translation-en_US.bz2/\n\n1"..., 64000) = 415 [pid 22037] gettimeofday({1578336553, 519282}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336554, 20030}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336554, 520421}, NULL) = 0 It looks like it's trying to download a file that isn't there; one of the particular endpoints it's trying to fetch is http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/dists/precise-cm5.4.5/contrib/i18n/Index. This file does not exist, and in fact, the i18n directory doesn't exist. I'm not sure if apt-get just always tries to find these files, or if a redirect was recently put into place, but it seems like apt-get doesn't know what to do. After these files fail to fetch, strace shows it waiting, repeatedly printing [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336573, 546170}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336574, 46926}, NULL) = 0 [pid 22037] select(20, [5 6 7 9 11 13 15 17 19], [], NULL, {0, 500000}) = 0 (Timeout) [pid 22037] gettimeofday({1578336574, 547674}, NULL) = 0 until it shows the HTTP failures above again. This is why I wasn't sure if maybe some files moved on Cloudera's side and redirects were put into place that this old version of apt-get doesn't know how to handle.
... View more
11-20-2019
11:19 AM
Hi @mokkan ,
Oozie High Availability is an Active - Active type of high availability, with a load balancer in front of both servers to distribute the load:
https://oozie.apache.org/docs/5.0.0/AG_Install.html#High_Availability_HA
... View more
11-11-2019
02:09 AM
If there is nothing to download then could you please let us know how to go ahead with the installation process with the CLI, any command of any guide is available for that
... View more
11-01-2019
08:35 AM
1 Kudo
@Rak ,
I'm suspecting you are using Impala, because of the statement of the values returning null, but please confirm so I can properly label this question so others can see it. Please see the following in the Impala documentation regarding the proper format for casting a string into a timestamp datatype:
https://docs.cloudera.com/documentation/enterprise/latest/topics/impala_timestamp.html
Restrictions:
If you cast a STRING with an unrecognized format to a TIMESTAMP , the result is NULL rather than an error. Make sure to test your data pipeline to be sure any textual date and time values are in a format that Impala TIMESTAMP can recognize.
Currently, Avro tables cannot contain TIMESTAMP columns. If you need to store date and time values in Avro tables, as a workaround you can use a STRING representation of the values, convert the values to BIGINT with the UNIX_TIMESTAMP() function, or create separate numeric columns for individual date and time fields using the EXTRACT() function.
Examples:
The following examples demonstrate using TIMESTAMP values with built-in functions:
select cast('1966-07-30' as timestamp);
select cast('1985-09-25 17:45:30.005' as timestamp);
select cast('08:30:00' as timestamp);
select hour('1970-01-01 15:30:00'); -- Succeeds, returns 15.
select hour('1970-01-01 15:30'); -- Returns NULL because seconds field required.
select hour('1970-01-01 27:30:00'); -- Returns NULL because hour value out of range.
select dayofweek('2004-06-13'); -- Returns 1, representing Sunday.
select dayname('2004-06-13'); -- Returns 'Sunday'.
select date_add('2004-06-13', 365); -- Returns 2005-06-13 with zeros for hh:mm:ss fields.
select day('2004-06-13'); -- Returns 13.
select datediff('1989-12-31','1984-09-01'); -- How many days between these 2 dates?
select now(); -- Returns current date and time in local timezone.
... View more
10-29-2019
11:47 AM
2 Kudos
Hello @luisfeg,
The files are only listed in the security scan out if the cgroups feature is in effect (CM -> Hosts -> Configuration -> "Enable Cgroup-based Resource Management"). The default permission of the cgroup_event_control files are indeed world writable without any other permission bit set. However, this has not too much effect in accordance with this document which explains why cgroup files are typically world writable: https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt
This seems to be created by the supervisor code cgroups feature, and Red Hat appears to have a subscription article on a similar concern elsewhere on the file system: https://access.redhat.com/solutions/377043. These files being world writable is an inherent part of Linux cgroups and users being able to use the cgroup API. Cloudera can only use this cgroup feature if the user configures it explicitly thru Cloudera Manager. As Red Hat states in the document mentioned above:
"Changing the world writable bit is not recommended on these files as it would break notification features of the cgroup API. Changing the permissions would have no effect. Cgroups has it's own pseudo-file-system and manages all the files therein. Each time the cgconfig init script is run, the world writable bit will be set on these files. The cgroup.event_control files are world writeable. This is because they are used in conjunction with the cgroups notification API to allow notifications to be sent about a changing status in the control group."
... View more
09-26-2019
11:15 AM
I wanted to interject that while both the above are definite valid possible causes of Oozie jobs stuck in PREP state, there may be several other possible causes which may need to be resolved such as:
1. Issues with the Yarn Resource Manager / MR Job Tracker, lack of resources either for the RM or queues for the user running the job.
2. Problems with the Oozie server getting to the oozie database server, the database server itself, or locks on tables.
3. Lack of resources to Oozie such as callable queues, java heap, GC thrashing, etc.
The above is a brief shortlist from review of support cases relating to Oozie jobs stuck in PREP. I want to emphasize that deleting records from the Oozie database should be ONLY done the last resort to solving this problem, and only needed if you have a very large mass of oozie workflows that cannot be killed in a timely fashion by an oozie CLI script. This should be only done at the direction of support, people knowledgeable with SQL, and the relationship between tables, columns, and rows in the oozie database as referential integrity and constraints are lacking in the schema design. The above post from 2017 also missed one key table COORD_ACTIONS, where if this data was not properly cleaned up, would break your Oozie purge and possibly cause other serious problems.
... View more
07-15-2019
09:02 AM
1 Kudo
Hi @Harish19 , the best place for information regarding TCP-DS tests on Impala would be (follow the README.md): https://github.com/cloudera/impala-tpcds-kit Once the data is populated in HDFS and tables are created, you likely can run most the same queries in tree/master/queries/ on Hive and/or Hive on Spark to test. IBM and Databricks have githubs with some SparkSQL tests, which you can Google for, but I have not personally evaluated them, or know if they work. Thanks,
... View more
06-06-2019
07:56 AM
1 Kudo
@Prav , This appears to have been listed as a bug (which is actually a longstanding limitation due to the definition of files and directories with _ and . being considered as "hidden" in FileInputFormat in Hadoop) of Hive since the 0.12 version: https://issues.apache.org/jira/browse/HIVE-6431 https://stackoverflow.com/questions/19830264/which-files-are-ignored-as-input-by-mapper If these files are needed to be seen, please consider using a pre-process script to rename them after loading. Thanks,
... View more
05-25-2019
06:38 PM
Hi, If you can share your workflow.xml and job.properties files, that can help us to see if your workflow is complete or not. Cheers Eric
... View more
05-06-2019
03:27 PM
1 Kudo
Thanks to everyone who replied. It turns out that references to truststores and server keys, etc., and associated passwords may be cached, so when we changed these after moving the cluster, creating new cerrts and replacing the passwords in CDH was insufficient. So, after DELETING all fields containing passwords, cert locations, key locations, etc.,unchecking SSL, restarting the cluster, and adding the references back in, everything works. Uugghhh - who knew! 🙂 B
... View more
03-12-2019
02:55 PM
As upsert is not SQL-92 syntax, the Impala JDBC driver does not currently support it. However, we are currently testing a newer version of JDBC driver and hope to support upsert in the near future. For the time being, please break these statements into a combination of insert and update statements.
... View more
03-01-2019
12:50 PM
Hello, Most likely, either you need to kinit and use curl -k, if kerberos is enabled, and/or additionally specify the 11443 oozie port with https, if TLS/SSL is enabled. With TLS/SSL, please remember to import the certificate into the cacerts file for the JDK so commands such as oozie and curl trust your oozie server's certificate. https://blog.cloudera.com/blog/2013/06/how-to-use-the-apache-oozie-rest-api/ https://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_ssl_oozie.html https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cm_sg_tls_troubleshoot.html
... View more
03-01-2019
12:02 AM
1 Kudo
Yes, I found the jar in other places, I remove the link, now the oozie web ui work fine! Thank you! Now I try to fix the oozie ha problem.
... View more
01-30-2019
07:30 AM
Hello, At this current point, the Altus Data Engineering and Altus Data Warehouse platform does not support Google Compute Engines and Google Compute Storage. Altus Director does provide these capabilities. As of yet, support for GCE and GCS is not on the immediate roadmap, but I will add your interest for this support.
... View more
01-23-2019
02:57 PM
Hi, I noticed you are using a quickstart VM, from your nameNode name ( hdfs : //quickstart.cloudera:8020) in your job.properties. Most likely, when the oozie launcher launches and stays in RUNNING status, but no further additional hive job launches, yarn does not have enough resources to launch an additional job. Please take a look at the Yarn Resource Manager role log and Yarn Resource Manager scheduler page for clues. This could be due to lack of memory (AM, nodemanager, or scheduler), vcores, or several other factors in yarn tuning. The quickstart VM is tuned for a very small demo environment. You may need to add additional memory to the VM and/or cores, then tune yarn to have more resources for the second hive job from oozie to launch. I will provide a link to a blog to aid you in this tuning below. https://blog.cloudera.com/blog/2015/10/untangling-apache-hadoop-yarn-part-2/
... View more
01-18-2019
04:36 AM
In the Sentry logs, shows this error: mismatch: Client thrift version is: 1 , server thrift version is 2 In Impala, it throws this error: org.apache.impala.catalog.AuthorizationException: User 'xxxxx' does not have privileges to access: server I tried to see if there is any mismatch in the cloudera libraries but didnt find anything... Any clue about it?
... View more
01-08-2019
03:57 PM
Please ensure that the hue, hive, impala, hue, solr, kakfa, and hbase group have not been removed from sentry.service.admin.group, and that the hue, hive, impala, hue, hdfs, solr, kakfa, and hbase users have not been removed in sentry.service.allow.connect. This is often the cause of the "Connection to sentry service denied due to lack of client credentials" exception. Please see the following documentation: https://www.cloudera.com/documentation/enterprise/latest/topics/hue_sec_sentry_auth.html#hue_sec_sentry_auth
... View more
01-08-2019
12:42 PM
There is no new product name. This will be a new feature within Sentry called Attribute Based Access Control using Navigator Lineage tags for metadata. At present, there is no estimated release date, but we will be sure to announce that.
... View more
01-07-2019
01:08 PM
1 Kudo
Typically, we see this type of stack trace when the keytab for the service principals do not match the principals in the KDC. Also, occasionally, the enctypes for the KDC do not match up with encryption types in your krb5.conf on the client side, with something like arcfour-hmac missing, but this is less likely the cause. https://web.mit.edu/kerberos/kfw-4.1/kfw-4.1/kfw-4.1-help/html/encryption_types.htm Our first suggestion would be to: Stop all services on the cluster Regenerating all of the existing principals beneath: Administration > Security > Kerberos Credentials > Select all > Regenerate Selected Start all services on the cluster
... View more
01-03-2019
08:10 AM
We are sorry to hear your cluster was compromised. Please see this recent security blog by Cloudera on malware attacks on un-secure Hadoop clusters [1]. Regardless of if you have a paid subscription, a distcp job can still be performed to another Hadoop cluster [2]. Please keep in mind that if the malware that has compromised your cluster has not yet been eradicated, and your target cluster is not secured with kerberos security as the blog mentions, there is a possibility your compromised cluster could be a source of attack to your target distcp cluster. As a last resort, you could possibly use hdfs dfs -copyToLocal for data that could fit into a local disk, then copy critical data elsewhere, in batches, prior to the remediation of the master node. [1] https://blog.cloudera.com/blog/2018/11/protecting-hadoop-clusters-from-malware-attacks/ [2] https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_admin_distcp_data_cluster_migrate.html [3] https://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-project-dist/hadoop-common/FileSystemShell.html
... View more
12-17-2018
10:40 AM
As this is an edge node, Spark 2 is not supported in Oozie actions until CDH 6, and it sounds like these source files may be located specifically on this edgenode, I would suggest using the Oozie SSH action to SSH to this specific edgenode. You would run a script in the SSH action to run the spark2-submit. Please see: https://oozie.apache.org/docs/4.1.0/DG_SshActionExtension.html
... View more
08-29-2017
10:15 AM
One typo correction to Anthony's reply above, the CLI command to retrieve information about a cluster is as 'describe-cluster' and not 'describe-clusters'.
... View more
06-22-2017
06:26 PM
Hi, If you are comfortable with Docker, you might as well use this Docker image for Altus CLI (currently for Altus CLI 1.1.0): https://github.com/tsuyo/cloudera-altuscli $ docker pull kirasoa/cloudera-altuscli $ docker run -ti -v ~/.altus:/root/.altus kirasoa/cloudera-altuscli (Change ~/.altus to whatever path on your Windows system)
... View more
04-19-2017
02:21 PM
Question Can I use a bootstrap script to further configure the cluster?
Answer Instance bootstrap scripts are executed on all the cluster instances immediately after startup before any services are configured and started. You can use it to install additional OS packages or add application dependencies (e.g. copy a jar to the cluster, etc). This is not intended to be a mechanism for cluster configuration.
... View more
04-19-2017
08:05 AM
1 Kudo
Question I noticed the Cloudera Altus documentation is not avaialable at the standard location for Cloudera documentation at: https://www.cloudera.com/documentation.html Where is this documentation available?
Answer When you login with your Cloudera credentials to: https://console.altus.cloudera.com/ There is a pulldown available in the upper right corner for Support -> Documentation that links to the docummentation at: https://console.altus.cloudera.com/support/documentation.html Documentation will also be available with the standard Cloudera documentation in an upcoming release.
... View more
04-17-2017
01:16 PM
1 Kudo
Symptoms When creating an AWS Environment through Quickstart, the environment attempts to create, but shows Failed.
Applies To Cloudera Altus Environments setup with Quickstart
Cause There can be several causes: Your limit of VPCs have been reached. Your Cloudwatch log group may already exist. Some other limit in AWS has been reached. You may not have proper permissions in AWS.
Troubleshooting Steps Beneath the AWS Console -> CloudFormation, select the Stack that just failed to deploy. Expand Events and work from the bottom up to see what the event that failed first. If a limit was exceeded, open an issue with AWS support or request a limit increase beneath EC2 -> limits. If a authorization role is needed for your account, contact your AWS account owner to provide you with that role. Otherwise, correct any failure other failure cause you see (e.g. duplicate CloudWatch log group, etc), delete the stack, and attempt to Quickstart the environment again.
References https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html
... View more
02-28-2017
10:49 PM
I recently upgraded CM to 5.10.0. Trying to upgrade CDH from the Manager from 5.8.0 throws up the same issue. I cannot see or select the CDH. Attached is a screenshot. Any help in figuring out next steps to get this running will help.
... View more
10-22-2015
11:55 AM
Neil, You could scp the .impalarc to the /home directory on all nodemanager nodes and make sure this was readable by all. Thanks, Robert Justice
... View more