Member since
07-19-2017
53
Posts
3
Kudos Received
3
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2532 | 08-23-2019 06:51 AM | |
| 6554 | 08-23-2019 06:45 AM | |
| 4983 | 08-20-2019 02:06 PM |
10-02-2019
06:05 AM
Hi @ravikiran_sharm I'm sorry you had to experience this mishap. I'll reach out to my managers to locate someone internally who can help you out with this. Sorry again for the inconvenience!
... View more
08-23-2019
06:51 AM
@paleerbccm Briefly looking at the message, I would assume 'error_code=0' actually means that no errors occurred. It would need quite a bit of digging in the code to understand, but generally speaking, I wouldn't worry too much about TRACE level logs. Ideally, and especially that this is a production environment, you would normally set logging level to INFO and that's about all you would need. Unless you have an intimate knowledge of the code and you're chasing after a specific issue, it's rare that you would ever need TRACE level logs.
... View more
08-23-2019
06:45 AM
Hi @iamabug It's a known limitation in Kafka where the kafka-topics tool communicates directly with Zookeeper. When you create a topic, all the tool does is connect to Zookeeper, creates a znode representing this topic and then sets some data as a JSON string (the metadata for the topic). There has been work to develop Java admin clients which made some progress: https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+admin+operations#KIP-117:AddapublicAdminClientAPIforKafkaadminoperations-FutureWork However, all that's left is to have command line tools that leverage those Java APIs and that's a work in progress: https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations
... View more
08-21-2019
06:06 AM
Thanks for confirming. Yes, I see that this is in fact possible in CDH 5 as well when I did a quick check. Just remember that starting with CDH 6.0, both Spark 2.x and Kafka are bundled in CDH, so the only way you would have two different versions is by running two different CDH versions from Cloudera Manager.
... View more
08-21-2019
05:59 AM
Hi @iamabug Yes, you can definitely have two different clusters managed by the same Cloudera Manager instance. In return, you can have different CDH versions for the two clusters, however, my initial thoughts was that you wouldn't be able to activate two different Kafka parcel versions, for example CDK 3.0 on Cluster A and CDK 2.2 on Cluster B. But I need to double check this because I have some doubts if that is in fact the case in CDH 5.x or not. In CDH 6.x you're bound to a specific version of Kafka depending on your CDH version since that's not released as a separate customer service so my response was geared more towards C6 rather than C5. Do you have two different versions of Kafka running on cluster 1 and cluster 2?
... View more
08-20-2019
08:04 PM
Two different clusters managed by two different Cloudera Managers.
... View more
08-20-2019
02:06 PM
Hi @iamabug This is not possible unfortunately. You could only have one parcel for a specific service activated at any given time. You would need to have a separate cluster managed by a different Cloudera Manager installation in order to activate a different Kafka parcel version.
... View more
08-15-2019
08:19 PM
Hi Chittu, Your issue here is that your JVM process is running out of memory, specifically heap space: java.lang.OutOfMemoryError: Java heap space Judging from the output you shared, I believe this is your driver that's running out of memory and so you would need to increase the maximum heap size for the driver. That's done by configuring the spark.driver.memory parameter or by passing the --driver-memory flag to the Spark command being used.
... View more