Member since
05-20-2020
125
Posts
4
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
477 | 10-25-2021 01:17 AM | |
476 | 10-18-2021 07:13 AM | |
368 | 10-03-2021 11:54 PM | |
4045 | 04-19-2021 02:50 AM | |
489 | 03-16-2021 01:28 AM |
01-10-2022
01:37 PM
@DataMike Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!
... View more
12-20-2021
10:45 AM
@Kallem Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!
... View more
12-06-2021
09:50 PM
@jaeseung, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
10-28-2021
11:33 PM
@roshanbi, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
10-26-2021
09:46 AM
Depends on your brokers version to support, as the error says. Related KIP-133 (implemented in Kafka 0.11, according to JIRA)
... View more
10-21-2021
10:44 AM
@Jarinek, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
10-21-2021
10:31 AM
@Mrju, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information @Nandinin has requested?
... View more
10-03-2021
11:54 PM
Hello @DA-Ka The data might be skewed in one of the disks because some of the heavily used topics/partitions are in that particular disk. You may want to profile the data residing (sort the kafka logs.dir) and reassign them to the other disk using the partition reassignment tool kafka-reassign-partition.sh or manually. du -sh /kafka-logs/*
... View more
08-11-2021
10:18 PM
https://www.youtube.com/watch?v=UrX2RWM2vQQ
... View more
07-29-2021
11:08 PM
@smkmuthu, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information as requested?
... View more
07-11-2021
01:41 PM
Hello, Every service should have the KRB5_CONFIG variable exported in its own safety valves. As per the doc - https://community.cloudera.com/t5/Support-Questions/Kerberos-How-to-setup-Cloudera-Manager-in-order-to-use-a/td-p/81138 Goto CM UI -> Schema Registry -> Configuration -> Service Environment Advanced Configuration Snippet (Safety Valve) ADD key: KRB5_CONFIG value : /etc/custom-krb5.conf Save and restart Thanks & Regards, Nandini
... View more
07-11-2021
01:21 PM
Hello, Please refer the document to help you with SSL authentication enabled - https://registry-project.readthedocs.io/en/latest/security.html?highlight=ssl Thanks & Regards, Nandini
... View more
07-11-2021
02:26 AM
It is too late to reply to this post, but was wondering if you found something and are using? Did you want something like ZooNavigator - https://zoonavigator.elkozmon.com/en/docs-pre-v1/
... View more
06-11-2021
06:56 AM
Hello Abdul, Can you please check and share the error seen in broker logs during this time frame on bigdatalite.localdomain?
... View more
05-06-2021
10:34 PM
Please try Nifi - Kakfa https://community.cloudera.com/t5/Community-Articles/Apache-NiFi-1-10-Support-for-Parquet-RecordReader/ta-p/282390#:~:text=With%20the%20release%20of%20Apache,data%20as%20a%20single%20unit
... View more
05-04-2021
12:08 AM
Hello @BGabor This error "TrustManager is not specified" can be thrown due to some of the following issues - Missing cert files or missing/wrong values for the below configs Make sure following properties are set:- xasecure.policymgr.clientssl.keystore.credential.file=jceks://file/{{credential_file}}
xasecure.policymgr.clientssl.truststore.credential.file=jceks://file/{{credential_file}}
xasecure.policymgr.clientssl.truststore=/path/to/truststore Also came across these Ranger jiras which indicates that the truststore info not specified in cacert or cacert needs to be manually configured. Note - they are fixed in Ranger 2.0.1, so you may also want to check the ranger version. https://issues.apache.org/jira/browse/RANGER-2611 https://issues.apache.org/jira/browse/RANGER-2907
... View more
04-19-2021
02:50 AM
1 Kudo
Hello @PabloMO , As the Spoiler Error pointed by you,the versions are not matching. You can check it by running "which python" You can override the below two configs in /opt/cloudera/parcels/CDH-<version>/lib/spark/conf/spark-env.sh and restart pyspark. export PYSPARK_PYTHON=<same version of python>
export PYSPARK_DRIVER_PYTHON=<same version of python> Hope it helps. Thanks & Regards, Nandini
... View more
04-19-2021
01:18 AM
Hello @Christ , Yes, it is possible. You can write a custom script or java code(atlas-client) that would extract the metadata from your source file and use the below REST APIs to insert it into Atlas. http://atlas.incubator.apache.org/api/v2/index.html For eg here, you can see a spark atlas connector - https://github.com/hortonworks-spark/spark-atlas-connector/blob/master/README.md#to-use-it-in-secure-environment Hope these documents help - https://docs.cloudera.com/runtime/7.2.7/atlas-leveraging-business-metadata/topics/atlas-business-metadata-bulk-import.html https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/atlas-import-utility/topics/atlas-importing-hive-metadata-utility.html Thanks, Nandini P.S. As always, if you find this post helpful, don't forget to upvote/accept the answer.
... View more
03-29-2021
06:43 PM
As you mentioned certain cli tools are marked not to be supported, for your requirement being the kafka-server-startup and shutdowns scripts. It is not guaranteed that the manual setup initializes the context of Kafka the same way, as Cloudera Manager would. While it may work for the restart but there are environment scripts and some other process management that Cloudera Manager does. Hence, when you restart using Cloudera Manager there would be uncertainties on the state of the process. As you said you have problems accessing the Cloudera Manager UI, you can also use the Cloudera Manager API for your requirement. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_intro_api.html
... View more
03-17-2021
05:21 PM
1 Kudo
This error shows up if you have selected Sentry/Ranger as dependencies but not checked true for the below config (i.e. did not enable Kerberos) kerberos.auth.enable
... View more
03-16-2021
01:28 AM
Hello @d33play I checked the documents again and found there is an internal jira CDH-72683 which was tracked for CDH versions to support Kafka Storage Handler. Kafka Storage Handler is a part of Hive 3.x version but CDH 6.1.1 comes with Apache Hive 2.1.1 Hence it is not supported in CDH versions but is available with CDP. https://docs.cloudera.com/runtime/7.2.0/release-notes/topics/rt-runtime-component-versions.html https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/runtime-release-notes/topics/rt-pvc-runtime-component-versions.html So, the KafkaStorageHandler package will not work with Hive 2.1.1. Hope this information helps. Thanks and Regards, Nandini
... View more
03-10-2021
03:25 AM
Hello @bhara, Do you have Ranger enabled? What is the Data center version? (I see a bug OPSAPS-58584 on 7.1.5 which was fixed in 7.2.4 , 7.3.0 )
... View more
03-09-2021
10:43 PM
Summarising all of the above, assuming you need the spark - atlas hook in your system, the solution is as follows. You need to add the file atlas-application.properties and spark-sql-kafka-0-10_2.11-<version>.jar to the Oozie shared spark library. The library is located on HDFS at <home>/oozie/share/lib/lib_<date>/spark. The application properties file can be found in various places such as /etc/hive/conf.cloudera.hive/. The jar file can be found in the parcels directory in /opt/cloudera/parcels/CDH/jars/. When you copy the files they need to be owned by oozie.oozie and be world readable. If you make any changes to the shared library you must then restart the Oozie server before jobs can find the new files.
... View more
10-21-2020
07:38 PM
We have two filter options external.kafka.metrics.exclude.prefix and external.kafka.metrics.include.prefix. https://github.com/hortonworks/ambari/blob/AMBARI-2.7.0-maint/ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java#L417
... View more