question Re: Apache Flume required to be run in CDP environment in Support Questions

Apache Flume required to be run in CDP environment

Uday_Singh2022 — Fri, 17 Jun 2022 06:57:01 GMT

Hi,

I have done migration from CDH to CDP 7.1.7. We were using Apache Flume previously. But since Flume is removed from CDP, our data flow is broken. I want to know if there is any documents available for setting up standalone Flume that can connect CDP Kafka consumer and save data to HBase tables.

Also want to note that I am using Kerberos for my cluster.

Re: Apache Flume required to be run in CDP environment

mszurap — Fri, 17 Jun 2022 08:39:43 GMT

Hi @Uday_Singh2022 , yes, Flume is not a supported component in CDP.

You can find documentations on Flume on it's official website:

https://flume.apache.org/

Have you considered to use CDF / Nifi for this usecase?

https://docs.cloudera.com/cdf-datahub/latest/nifi-hbase-ingest/topics/cdf-datahub-nifi-hbase-ingest.html

Thanks, Miklos

Re: Apache Flume required to be run in CDP environment

Uday_Singh2022 — Fri, 17 Jun 2022 10:54:42 GMT

hi @mszurap ,

Thanks for you reply.

Yes I am aware of NiFi but client is not ready to migrate to it. They want to use Flume itself. So I am looking if there is any documentation from Cloudera side that can help.

Re: Apache Flume required to be run in CDP environment

ask_bill_brooks — Fri, 17 Jun 2022 12:19:43 GMT

@Uday_Singh2022 There won't be any documentation available from Cloudera regarding Flume because that lack of availability is part of what "not a supported component" means. If you must use Flume, then the documentation on the Apache site that @mszurap pointed you to above is your only option.

Re: Apache Flume required to be run in CDP environment

Uday_Singh2022 — Fri, 17 Jun 2022 12:34:29 GMT

Thanks Guys.