I have a Dell 7920 server with Ubuntu 16.04 installed. I have OpenJDK 8 installed. Earlier in the year, I had installed CDH 5.16.2 in anticipation of carrying out a project, which was subsequently carried out on a separate platform (management decision) Last week I installed CUDA and tested it to ensure that it was working. On running nvcc -V, I get the following: Built on Sun_Jul_28_19:07:16_PDT_2019 Cuda compilation tools, release 10.1, V10.1.243 I had been using CDH 5.16.2 the instance for training myself up and have decided to upgrade it. Through my investigations, I observed through the documentation that the amount of work required to upgrade from CDH 5.16.2 to CDH 6.3.1 was far more than a mere uninstalling of CDH 5.16.2 and subsequent installation of 6.3.1. Therefore I proceeded with the uninstallation of 5.16.2 as per https://docs.cloudera.com/documentation/enterprise/5-16-x/topics/cm_ig_uninstall_cm.html Upon completion of uninstallation, I found that many CDH 5.16 files still existed in my system, especially in /etc/apt/sources.list.d, which I removed and stored elsewhere. I then carried out the following Step 1 and 2 (already done since I have OpenJDK 😎 in the installation process https://docs.cloudera.com/documentation/enterprise/latest/topics/configure_cm_repo.html#cm_repo 1. Download the cloudera.list file for your OS version to the /etc/apt/sources.list.d/ directory on the Cloudera Manager Server host. You can find the URL in the Repo File column in the Cloudera Manager 6 Version and Download Information table for the Cloudera Manager version you want to install. 2. Import the repository signing GPG key: (I heard removed the previous key prior to this step) wget https://archive.cloudera.com/cm6/6.3.0/ubuntu1604/apt/archive.key sudo apt-key add archive.key Update your system package index by running: sudo apt-get update I then carried executed the commands in Step 3 https://docs.cloudera.com/documentation/enterprise/latest/topics/install_cm_server.html sudo apt-get install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server for which I receive the following messages: Reading package lists... Done Building dependency tree Reading state information... Done You might want to run 'apt-get -f install' to correct these: The following packages have unmet dependencies: cloudera-manager-server-db-2 : Depends: cloudera-manager-server (= 5.16.1-1.cm5161.p0.1~xenial-cm5) but 6.3.1~1466458.ubuntu1604 is to be installed nvidia-cuda-toolkit : Depends: nvidia-cuda-dev (= 7.5.18-0ubuntu1) but it is not going to be installed E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution). I am now a little lost as to how I should proceed. Please advice. Thank you ... View more

I have installed Cloudera 5.16.2 on Ubuntu 16.04.    Most of the data I need to work on are in MySQL or Postgresql databases and I want to use Apache Spark to work on this data directly. I prefer working with PySpark and have installed and configured Anaconda through Cloudera Manager.    How would I connect to MySQL / Postgres using Apache Spark on a Jupyter Notebook? Could you give me a step by step guide to achieve this?   Thank you  ... View more

I have completed the installation of CDH 5.16.1 on Ubuntu 16.04 LTS with OpenJDK 8. I am now in the process of testing the different components to ensure each works as predicted.   In my attempt to test Sqoop using the Cloudera Hadoop Tutorial, I ran the following command:    sqoop import-all-tables \ -m {{cluster_data.worker_node_hostname.length}} \ --connect jdbc:mysql://{{cluster_data.manager_node_hostname}}:3306/retail_db \ --username=retail_dba \ --password=cloudera \ --compression-codec=snappy \ --as-parquetfile \ --warehouse-dir=/user/hive/warehouse \ --hive-import  but received the following error:   Could not load db driver class: com.mysql.jdbc.Driver   In my search for a solution, I downloaded and placed the mysql conector in the following folders according to the advice given:   /var/lib/sqoop   /opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/sqoop   I am still getting the error above. Please help. ... View more