Hi,   are you trying to run spark-shell or pyspark on a node where the Spark Gateway is not installed by any chance? The classes should come from a JAR file called spark-hive_<version>.jar on the host in the dir /opt/cloudera/parcels/CDH/jars Could you check if that exists? Could you tell a bit more about what kind of jobs you are trying to run and from where?   Regards, Zsombor ... View more

Hi There,   sure, the on-premises part of the CDP stack is called CDP Private Cloud Base. To install this on the machines you can refer to the following documentation page: https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/installation/topics/cdpdc-installation.html   On top of that, you will need to add Spark 3 as a custom service descriptor. We refer this package as CDS, and you can find the installation guide here: https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/cds-3/topics/spark-install-spark-3-parcel.html   Hope that helps, Zsombor  ... View more

A bit late to the party, but hope the following will help.   By calling the main functions of the classes,  UnixUserGroupBuilder, PolicyMgrUserGroupBuilder or LdapUserGroupBuilder is not going to work, since the main classes of these are only initializing the classes. In order to start the actual sync, the function updateSink needs to be called. During startup this is handled by the class org.apache.ranger.usergroupsync.UserGroupSync thus, calling its main function will trigger the syncing using the configuration that you set in your cluster.   A complete example for triggering the usersync manually could be: java -Dlogdir=/var/log/ranger/usersync -cp "/usr/hdp/current/ranger-usersync/dist/unixusersync-1.2.0.3.1.5.135-2.jar:/usr/hdp/current/ranger-usersync/lib/*:/etc/ranger/usersync/conf" org.apache.ranger.usergroupsync.UserGroupSync for HDP and java -Dlogdir=/var/log/ranger/usersync -cp "/opt/cloudera/parcels/CDH/lib/ranger-usersync/dist/unixusersync-2.1.7.1.7.0-460.jar:/opt/cloudera/parcels/CDH/lib/ranger-usersync/lib/*:/etc/ranger/usersync/conf" org.apache.ranger.usergroupsync.UserGroupSync for CDP. ... View more