Member since
04-22-2014
1218
Posts
341
Kudos Received
157
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 26232 | 03-03-2020 08:12 AM | |
| 16373 | 02-28-2020 10:43 AM | |
| 4703 | 12-16-2019 12:59 PM | |
| 4470 | 11-12-2019 03:28 PM | |
| 6648 | 11-01-2019 09:01 AM |
05-24-2019
07:33 AM
Based on the latest news and this article, Apache Ranger will replace Sentry in the next CDP (the new version of Cloudera Hadoop after they merged with Hortonworks): Ranger will be pushed in detriment of Sentry, but some of the features from Sentry will be integrated in later Ranger versions. We're months away from the release.
... View more
05-23-2019
01:56 AM
Thanks @bgooley ! That confirms that we are not totally off-track. Best regards, rdbb
... View more
05-22-2019
01:13 PM
Not able to tag you in another thread. let me know if anything else need to be tried.
... View more
05-10-2019
12:32 AM
@JoaquinS Your CM agent config file does not have TLS enabled, but the CM server configuration has. I would suggest to disable the Use TLS Encryption for Agents, Use TLS Authentication of Agents to Server, Verify Agent Hostname Against Certificate settings and restart CM to proceed to make the installation work in a first step. Once that is done, please follow the steps in this documentation chapter to enable TLS encryption for CM agent communication.
... View more
05-07-2019
01:24 AM
Hi Prash, Since i upgraded the packages in the early stage of the setup. I just created a dump (backup) for the Kerberos , and use the yum update command to upgrade it. After updated the packages, the cluster could be started.
... View more
04-24-2019
02:27 AM
@bgooley , thank you for your feedback and your clear explanation , in fact the problem was resolved by removing the contents of /var/lib/hadoop-yarn/yarn-nm-recovery/ directory and then the Nodemanager role started successfully. the solution that I've found was from : https://community.cloudera.com/t5/Batch-Processing-and-Workflow/Yarn-NodeManager-fails-to-start-and-crashing-with-SIGBUS/m-p/66590#M3611
... View more
04-23-2019
10:26 AM
@wusj, I was a bit unclear in my previous update regarding the packages to install. The CM 6.2 Agent package needs to be installed on each of your cluster hosts and the agent needs to be restarted with "service cloudera-scm-agent restart" or "systemctl restart cloudera-scm-agent" in order for the code fix to be used.
... View more
04-12-2019
02:04 AM
Dear Romainr: I follow your advise to modify the srcCode and made the hue3.9 notebook worked on my CDH5.7.1 Kerberized cluster with livy0.5.0.Now i can using spark-shell on notebook and its run well. but on yarn i saw the spark job user was always livy(i set livy be a haddop.proxyuser), seems its bind on the user which the keytab of livy-server-luancher. so i can't control the notebook authority by hue. i can see the hue set proxyuser on a the token, but the spark job's user was not be reset. livy-conf livy.impersonation.enabled=true
livy.repl.enable-hive-context=true
livy.spark.deploy-mode=client
livy.spark.master=yarn
livy.superusers=hue
livy.server.auth.type=kerberos
livy.server.auth.kerberos.keytab=/etc/security/keytabs/spnego.keytab
livy.server.auth.kerberos.principal=HTTP/xxx.com@xxx.COM
livy.server.launch.kerberos.keytab=/etc/security/keytabs/livy.keytab
livy.server.launch.kerberos.principal=livy/xxx.com@xxx.COM livy-log 19/04/12 14:48:14 INFO InteractiveSession$: Creating Interactive session 3: [owner: hue, request: [kind: pyspark, proxyUser: Some(baoyong), heartbeatTimeoutInSecond: 0]]
19/04/12 14:48:25 INFO LineBufferedStream: stdout: client token: Token { kind: YARN_CLIENT_TOKEN, service: }
19/04/12 14:48:25 INFO LineBufferedStream: stdout: diagnostics: N/A
19/04/12 14:48:25 INFO LineBufferedStream: stdout: ApplicationMaster host: 192.168.103.166
19/04/12 14:48:25 INFO LineBufferedStream: stdout: ApplicationMaster RPC port: 0
19/04/12 14:48:25 INFO LineBufferedStream: stdout: queue: root.livy
19/04/12 14:48:25 INFO LineBufferedStream: stdout: start time: 1555051701595
19/04/12 14:48:25 INFO LineBufferedStream: stdout: final status: UNDEFINED
19/04/12 14:48:25 INFO LineBufferedStream: stdout: tracking URL: http://bigdata166.xxx.com:8088/proxy/application_1555044848792_0063/
19/04/12 14:48:25 INFO LineBufferedStream: stdout: user: livy
19/04/12 14:48:28 INFO InteractiveSession: Interactive session 3 created [appid: application_1555044848792_0063, owner: hue, proxyUser: None, state: idle, kind: pyspark, info: {driverLogUrl=null, sparkUiUrl=null}]
... View more
04-10-2019
04:30 AM
1 Kudo
put it at the end of $PATH .....
... View more