Hello @roshanbi     JSON is one of the best data exchange format  which is lightweight and compact. C/DSV can be used when large dataset, with bandwidth limits are to be sent, but DSV cannot be used where the data is complex and unstructured; there comes in JSON.   So, as you said, you are are seeing a better performance with DSV, that is because it does not contain attribute-value pair as in JSON (which makes it structured). To fine tune you application, you can use  Compression. ... View more

Yes, there has to be a corresponding user in Ranger to authorise, it can not be just a certificate.  You can use Kafka SSL Authentication by setting it up for 2 way  https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/kafka-securing/topics/kafka-secure-auth-tls-broker.html and if you want Authorisation(via Ranger) add the user https://docs.cloudera.com/runtime/7.2.8/security-ranger-authorization/topics/security-ranger-users-groups-permissions.html   The same is also discussed here - https://cwiki.apache.org/confluence/display/KAFKA/KIP-371%3A+Add+a+configuration+to+build+custom+SSL+principal+name   Hope this helps.  ... View more

Hello @DA-Ka    The data might be skewed in one of the disks because some of the heavily used topics/partitions are in that particular disk.    You may want to profile the data residing (sort the kafka logs.dir) and reassign them to the other disk using the partition reassignment tool kafka-reassign-partition.sh or manually.      du -sh /kafka-logs/*       ... View more

Hello,    Please refer the document to help you with SSL authentication enabled - https://registry-project.readthedocs.io/en/latest/security.html?highlight=ssl   Thanks & Regards, Nandini ... View more

Hello @PabloMO ,   As the Spoiler Error pointed by you,the versions are not matching. You can check it by running "which python"   You can override the below two configs in /opt/cloudera/parcels/CDH-<version>/lib/spark/conf/spark-env.sh and restart pyspark. export PYSPARK_PYTHON=<same version of python> export PYSPARK_DRIVER_PYTHON=<same version of python>    Hope it helps.   Thanks & Regards, Nandini ... View more

Hello @d33play  I checked the documents again and found there is an internal jira CDH-72683 which was tracked for CDH versions to support Kafka Storage Handler.  Kafka Storage Handler is a part of Hive 3.x version but CDH 6.1.1 comes with Apache Hive 2.1.1 Hence it is not supported in CDH versions but is available with CDP. https://docs.cloudera.com/runtime/7.2.0/release-notes/topics/rt-runtime-component-versions.html https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/runtime-release-notes/topics/rt-pvc-runtime-component-versions.html   So, the KafkaStorageHandler package will not work with Hive 2.1.1.   Hope this information helps.   Thanks and Regards, Nandini   ... View more