Trying to add Kafka to our CDH cluster

[#1]

Hi Experts:

Our Big Data cluster is running CDH 6.3.4. Today we are trying to install and activate Kafka. It seems that the "KAFKA-4.1.0-1.4.1.0.p0.4-el7.parcel" downloading worked fine. It failed to activate the parcel.

In the file "cloudera-scm-server.log", I see this error:

2022-05-24 11:05:56,777 INFO scm-web-268076:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/parcel/details, Status:200
2022-05-24 11:05:58,414 INFO scm-web-267966:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/clusters/1/activateParcel
2022-05-24 11:05:58,418 INFO scm-web-267966:com.cloudera.parcel.components.ParcelManagerImpl: Activating parcel KAFKA:4.1.0-1.4.1.0.p0.4 on cluster Peel
2022-05-24 11:05:58,429 WARN scm-web-267966:com.cloudera.server.web.cmf.WebController: Failed to activate parcel KAFKA:4.1.0-1.4.1.0.p0.4 for cluster DbCluster{id=1, name=Peel}
2022-05-24 11:05:58,429 ERROR scm-web-267966:com.cloudera.server.web.common.JsonResponse: JsonResponse created with throwable:
com.cloudera.parcel.ParcelRelationsException: CDH 6.3.4-1.cdh6.3.4.p0.6626826 replaces KAFKA 4.1.0-1.4.1.0.p0.4.
at com.cloudera.parcel.components.ParcelDependencyManagerImpl.validateDependencies(ParcelDependencyManagerImpl.java:418)
at com.cloudera.parcel.components.ParcelDependencyManagerImpl.validateDependenciesForActivation(ParcelDependencyManagerImpl.java:375)
at com.cloudera.parcel.components.ParcelManagerImpl.activateParcel(ParcelManagerImpl.java:295)
at com.cloudera.parcel.components.ParcelManagerImpl.activateParcel(ParcelManagerImpl.java:246)
at com.cloudera.server.web.cmf.parcel.ParcelActivationController.activateParcel(ParcelActivationController.java:61)
at com.cloudera.server.web.cmf.parcel.ParcelActivationController$$FastClassBySpringCGLIB$$9b87c4cb.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at com.cloudera.server.web.cmf.aop.RetryExecution$1.call(RetryExecution.java:32)
at com.cloudera.server.common.RetryWrapper.executeWithRetry(RetryWrapper.java:32)
at com.cloudera.server.common.RetryUtils.executeWithRetryHelper(RetryUtils.java:210)
at com.cloudera.server.common.RetryUtils.executeWithRetry(RetryUtils.java:131)
at com.cloudera.server.web.cmf.aop.RetryExecution.retryOperation(RetryExecution.java:24)
at sun.reflect.GeneratedMethodAccessor1683.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)

Could you please give some hint where to check for the problem? Thank you!

[#2]

Hi,

This error may occure when CDH parcel is not activated properly or the version do not match, You can double check it.

And we may need to check complete log file, The above log snippet may not be enough.

Regards,

Chethan YM

[#3]

Hi Chethan,

Thank you for the reply. The parcel version was selected automatically after we clicked 'download' in the Cloudera Manager. I suppose it is the correct version. 

2022-05-24 11:04:57,774 INFO New I/O worker #340:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: https://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.1.0-1.4.1.0.p0.4-el7.parcel

However from this page, it shows for CDH 6.3.4, the Apache Kafka version is 2.2.1

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_63_packaging.html

Not sure how the parcel version and Apache version relate to each other. We have the complete log file, but I hesitate to send the log here in an open forum. Is there a way to send to you in a protected mode? Thanks again!