As you know,   We have limitation with source kafka connector that it works for HDFS  objects/files created only by the HDFS 2 Sink Connector for Confluent Platform and how we can pull the files if created by other spark,mapreduce or any other jobs on HDFS?   The use case of HDFS source connector is only to mirror the same data on kafka.     ... View more

How to read parquet files using Kconnect.? In simple,We just want to read the parquet files on HDFS using kconnect and without spark jobs? Please let us know if there is a solution or not?         ... View more

Hello @Nandinin , We have gonbe through this already. Anything without Scala/Spark ?     ... View more

@Nandinin , Yes the flow is correct The files are in parquet format.     ... View more

@Nandinin , We have requirement like spark program are writing the files into HDFS. So we want to read those files and send to kaka.   We know HDFS sink connector is useful for writing to HDFS as well and HDFS source connector is useful when the files written by hdfs sink connector. HDFS source connector is also not the solution if files written by spark programming. Please let us know if there is any solutions for this requirement?       ... View more

Same as previous request. We are looking for source connector here as well to pull the data from Amazon s3 and put in kafka.   ... View more

We already have the data in HDFS and we want to pull the data from HDFS and put in in kafka topic. So, we are looking for source connector here in pulling the data from HDFS and placing in kafka. ... View more

Thank you.   We will work on this solution and get back. Do we have the same to read data from Amazon-s3 and put into the topic?     ... View more

Can you help to provide more insight on how we can able to achieve this?   Thank you Srinu ... View more