Member since
10-01-2018
802
Posts
143
Kudos Received
130
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3057 | 04-15-2022 09:39 AM | |
| 2467 | 03-16-2022 06:22 AM | |
| 6533 | 03-02-2022 09:44 PM | |
| 2897 | 03-02-2022 08:40 PM | |
| 1907 | 01-05-2022 07:01 AM |
07-19-2020
03:07 AM
Hi @GangWar Thanks for your reply.
... View more
07-19-2020
01:40 AM
1 Kudo
@mike_bronson7 I meant to say there is not a bunch of people using package install because of following reason[1]: Cloudera does not support clusters that are not deployed and managed by Cloudera Manager. If you choose to install CDH manually using these instructions, you cannot use Cloudera Manager to install additional parcels. This can prevent you from using services that are only available via parcel. For testing purpose you are free to choose any method but it depends what you are looking i.e easy managing of cluster using CM web UI or just want to play with a demo installation. Regarding the end of life question, Yes that's the nature of software support that end of the day it will be end of life after some years. For Cloudera software support cycle you can here is a glance: https://www.cloudera.com/legal/policies/support-lifecycle-policy.html Cheers! [1] https://docs.cloudera.com/documentation/enterprise/5-16-x/topics/install_cloudera_packages.html
... View more
07-18-2020
06:34 AM
@prabhat The error seems misleading as per my experience but I would like to check the ownership and permission on the directory /var/run/cloudera-scm-agent/process/ this should be like below: ll /var/run/cloudera-scm-agent/process/ | grep -i hdfs drwxr-x--x. 3 hdfs hdfs 420 Jun 26 15:18 1805-hdfs-DATANODE drwxr-x--x. 4 httpfs httpfs 360 Jun 26 15:18 1806-hdfs-HTTPFS drwxr-x--x. 3 hdfs hdfs 420 Jun 26 15:30 1857-hdfs-DATANODE drwxr-x--x. 4 httpfs httpfs 360 Jun 26 15:30 1858-hdfs-HTTPFS drwxr-x--x. 3 hdfs hdfs 420 Jun 26 16:40 1902-hdfs-DATANODE drwxr-x--x. 4 httpfs httpfs 360 Jun 26 16:40 1903-hdfs-HTTPFS If the you there is permission issue then fix them else you can try to hard start agent once and see how this goes.
... View more
07-17-2020
02:15 PM
MLOps will eventually be available as part of Cloudera Data Science Workbench (CDSW) product. Keep an eye out for new releases coming soon.
... View more
07-07-2020
03:02 AM
1 Kudo
Hi @GangWar This is what I was aiming at. My current Dockerfile roughly looks as follows: #Dockerfile
FROM docker.repository.cloudera.com/cdsw/engine:10
WORKDIR /tmp
ENV R_HOME=/usr/local/lib/R
RUN wget http://cran.rstudio.com/src/base/R-3/R-3.6.3.tar.gz && \
tar xvf R-3.6.3.tar.gz && \
cd R-3.6.3 && \
./configure --prefix=/usr/local --enable-R-shlib && \
make && \
make install && \
rm -rf /usr/local/bin/R && \
rm -rf /usr/local/bin/Rscript && \
ln -s /usr/local/lib/R/bin/R /usr/local/bin/R && \
ln -s /usr/local/lib/R/bin/Rscript /usr/local/bin/Rscript && \
echo -e "# make libR.so visible to ld.so\n/usr/local/lib/R/lib" > /etc/ld.so.conf.d/libR.conf && \
ldconfig && \
cd .. && \
rm -rf R-3.6.3.tar.gz && \
rm -rf R-3.6.3
# the java installation is mounted at CDSW session run time - copy it to the build context here
COPY ./java /usr/lib/jvm/java-openjdk
RUN export JAVA_HOME=/usr/lib/jvm/java-openjdk && \
R CMD javareconf
RUN Rscript -e "update.packages(checkBuilt=TRUE, ask=FALSE, repos='https://cloud.r-project.org')"
# remove java installation again since it is mounted at runtime
RUN rm -rf /usr/lib/jvm I am successfully able to launch CDSW sessions with this container with the following editors: Workbench - Python Workbench - Scala RStudio Jupyter Notebook (even with IRkernel) The issue remains though, that launching a session with the Workbench - R editor is not possible. The session is immediately exited and the following error visible in the log: ...
PID of main R process is 205
PID of parser R process is 207
R has exited with code 2 and signal null
Exiting with code 2
... Using the base image which works with the Workbench - R editor, I checked which processes CDSW tries to launch in this case. These are the following two: cdsw 197 51 0 08:29 ? 00:00:00 /usr/local/lib/R/bin/exec/R --sense --no-readline --args
cdsw 198 51 0 08:29 ? 00:00:00 /usr/local/lib/R/bin/Rserve --RS-socket /tmp/cdsw-rserve-x0jbdet9dfe1428o.sock --RS-source /usr/local/lib/node_modules/r-engine/lib/parse.utils.r --slave Using my own image and a Workbench - Python session, I was able to successfully execute these two processes. Process 1: bash$ /usr/local/lib/R/bin/exec/R --sense --no-readline --args
WARNING: unknown option '--sense'
R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> Process 2: bash$ /usr/local/lib/R/bin/Rserve --RS-socket /tmp/cdsw-rserve-x0jbdet9dfe1428o.sock --RS-source /usr/local/lib/node_modules/r-engine/lib/parse.utils.r --slave
Rserve started in non-daemon mode. To conclude: There seems to be something behind the scenes which CDSW does when launching the Workbench - R editor, which causes it to crash, but I cannot get to the bottom of what the issue is. Any ideas?
... View more
07-06-2020
09:14 AM
i , did you find a solution for that , i have the same request by audit team.
... View more
06-29-2020
01:31 AM
@Tulasi For the best information around licensing, I'll refer you to the Cloudera Licensing Policy FAQs page. If you still have questions I suggest using the contact us page of Cloudera.com to connect with sales for further.
... View more
06-26-2020
07:43 AM
Thank you for this link: trying this out. (Yet) another bizarre thing happened with 7.1.x installation - after several attempts I was able to install and get the Cloudera Manager up with web UI. Most of the services were unavailable even after manually starting them (either through 'service' command or through web UI). Upon observing settings, system activities and logs, I noticed Cloudera was using 6.x binaries AND there were no 7.x files (except for the client libraries). Hoping to see 7.x running with minimal difficulties.
... View more
06-24-2020
02:45 PM
@Mondi As far as I know real time replication is not possible but tow cluster can be in sync using BDR feature. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_bdr_replication_intro.html
... View more