I'm using Cloudera QuickStart VM 5.13 and I installed their Kafka version. Listing Kafka Topics: /usr/bin/kafka-topics --list --zookeeper quickstart.cloudera:2181 Creating Kafka Topic: /usr/bin/kafka-topics --create --zookeeper quickstart.cloudera:2181 --replication-factor 1 --partitions 3 --topic myFirstTopic Start a Producer: /usr/bin/kafka-console-producer --broker-list quickstart.cloudera:9092 --topic myFirstTopic Start a Consumer: /usr/bin/kafka-console-consumer --bootstrap-server quickstart.cloudera:9092 --topic myFirstTopic --from-beginning   Notes for your issue: Replications need to be less than or equal to the number of brokers. I changed offsets.topic.replication.factor in Kafka configuration from Cloudera Manager and I set it to 1 (Cause I have 1 broker)  You can delete brokers from Zookeeper as shown in the link below. Then restart Kafka to recreate these brokers. Zookeeper CLI: Accessing Zookeeper CLI: /usr/bin/zookeeper-client List All: ls / Output: [cluster, controller, brokers, zookeeper, admin, isr_change_notification, log_dir_event_notification, ngdata, controller_epoch, solr, consumers, latest_producer_id_block, config, hbase] List Kafka Brokers: ls /brokers Output: [ids, topics, seqid] List Kafka Topics in Zookeeper: ls /brokers/topics Output: [myFirstTopic, __consumer_offsets]  Delete a Path in Zookeeper: rmr /brokers   ... View more

I managed to integrate Airflow with redis into Cloudera Manager. To run custom DAG, they need to be uploaded to the airflow dag folder on the node, where airflow scheduler and workers are dunning ... View more

Hi @anbazhagan_muth  You don't need to worry about those two configurations unless you're using Kafka MirrorMaker: Destination Broker List bootstrap.servers   Source Broker List source.bootstrap.servers The Kafka MirrorMaker is used to replicate data from one Kafka service to the other. With that said, the configurations should be self explanatory, where the source broker list (source.bootstrap.servers) is the list of your brokers in the source Kafka service the MirroMaker is going to read data from, and the destination broker list (bootstrap.servers) is the list of brokers in your destination Kafka service where the MirrorMaker is going to write the data to.  This is a comma separated list and the format would be something like: BROKER1_HOSTNAME:PORT_NUMBER, BROKER2_HOSTNAME:PORT_NUMBER PORT_NUMBER is going to be either 9092 for PLAINTEXT or SASL_PLAINTEXT, or 9093 for SSL or SASL_SSL. ... View more

@anbazhagan_muth    You can leave them blank unless you're using mirror maker, in that case, you have to fill up those values using:   Destination Broker List bootstrap.servers     Comma-separated list of IP:port (or hostname:port) pairs of brokers on destination cluster. This should be more than one, for high availability, but there's no need to list all brokers.     Source Broker List source.bootstrap.servers     Comma-separated list of IP:port (or hostname:port) pairs of brokers on source cluster. This should be more than one, for high availability, but there's no need to list all brokers.       Documentation below: https://docs.cloudera.com/documentation/enterprise/5-7-x/topics/cm_props_cdh540_kafka.html#concept_5.7.x_kafkabroker_props     ... View more

Hi,   You can use couple of parameters for strings columns and non string column.   ## string columns:- --null-string   ## non string columns:- --null-non-string   Link:- https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_null_string_handling   Above is the link will help you to handle these characters.   Regards Nitish ... View more

Could you fire the below command . it will fix the permission error    sudo -u hdfs hadoop fs -chmod 755 /tmp let me know if that helps .      Guna ... View more

Hi @anbazhagan_muth,   I would suggest starting by looking over the required skills section of the CCA Spark and Hadoop Developer and CCA Data Analyst certification pages. Think of each skill as a possible task you could be asked to perform as part of the exam. Also keep in mind the exam question format listed on each certification page. For example, from the CCA Spark and Hadoop Developer certification page:   Exam Question Format Each CCA question requires you to solve a particular scenario. In some cases, a tool such as Impala or Hive may be used. In other cases, coding is required. In order to speed up development time of Spark questions, a template may be provided that contains a skeleton of the solution, asking the candidate to fill in the missing lines with functional code. This template will either be written in Scala or written in Python, but not necessarily both. You are not required to use the template and may solve the scenario using a language you prefer. Be aware, however, that coding every problem from scratch may take more time than is allocated for the exam.   I would also check over the Exam delivery and cluster information on each certification page and become familiar with which tools will be available to you during the exam.   I hope this helps. 🙂 ... View more

i am not seeing any screenshot that you had attached .  ... View more