Hi Team,   CDP uses the "org.apache.spark.internal.io.cloud.PathOutputCommitProtocol" OutputCommitter which does not support dynamicPartitionOverwrite.   You can set the following parameters into your spark job.   code level:   spark.conf.set("spark.sql.sources.partitionOverwriteMode", "dynamic") spark.conf.set("spark.sql.parquet.output.committer.class", "org.apache.parquet.hadoop.ParquetOutputCommitter") spark.conf.set("spark.sql.sources.commitProtocolClass", "org.apache.spark.sql.execution.datasources.SQLHadoopMapReduceCommitProtocol") spark-submit/spark-shell:   --conf spark.sql.sources.partitionOverwriteMode=dynamic --conf spark.sql.parquet.output.committer.class=org.apache.parquet.hadoop.ParquetOutputCommitter --conf spark.sql.sources.commitProtocolClass=org.apache.spark.sql.execution.datasources.SQLHadoopMapReduceCommitProtocol  Note: If you are using S3, you can disable it by specifying spark.cloudera.s3_committers.enabled parameter.   --conf spark.cloduera.s3_committers.enabled=false    ... View more

KT should be installed in it's own cluster. From a security standpoint, you don't want other services on the box.   ... View more

@Mondi    The simple answer is YES and the best source is the vendor itself  Rack awareness CDP as computations are performed with the assistance of rack awareness scripts.   Hope that helps Was your question answered? If so make sure to mark the answer as the accepted solution. If you find a reply useful, Kudos this answer by hitting the thumbs up button. ... View more

@Mondi you can follow below doc for reference. All you have to do is modify the db.properties file and point to the server which is having CM DB installed.  https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5 ... View more

@Mondi You can choose any convenient path. Both CM/CDH 5.x and 6.x support RHEL7.2.  So you can upgrade the CM/CDH first and then perform an OS upgrade. See the below docs for help.    https://docs.cloudera.com/cdp-private-cloud/latest/upgrade-cdh/topics/ug_os_upgrade.html https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_os_requirements.html#c63_supported_os   ... View more

Hello @Mondi ,   thank you for asking for clarification if CDP has a Cloudera Express-like "no license required" distribution. Our licensing strategy changed and you can find the details about it here. For your convenience, I am pasting here a part of the "Subscription Agreement" section that might be relevant for you:   "3. Will Cloudera offer Developer or Trial software Subscriptions? Customers and developers will be able to access our products with a subscription agreement with Cloudera.  We will have partner development subscription agreements for developers through our Cloudera Connect Partner Program, and free (unsupported) short-term trial subscriptions through Cloudera.com."   Based on the documentation Cloudera Express is discontinued from version 6.3.3.   Hope it clarifies, let us know, please!   Kind regards: Ferenc     ... View more

Hello @Mondi ,   When you install CDP Trial Version, it includes an embedded PostgreSQL database and is not suitable for a production environment. Please check this information  for more details.   Also, find how to end the trial or upgrade trial version  and  Managing licenses      ... View more

@Mondi    Please refer below document, you can setup a local package repository for Cloudera Manager upgrade.   https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_create_local_package_repo.html#internal_package_repo   For CDH upgrade you can follow below document https://docs.cloudera.com/cdp-private-cloud-base/7.1.3/installation/topics/cdpdc-using-local-parcel-repository.html   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

You cannot perform an in-place encryption of an existing directory. You need to create an encryption zone and move data to the encryption zone. Here's the docs with the procedure.   ... View more