Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to get data from a txt file (lets say from my desktop ) and insert into kafka topic(In avro format) from nifi processor. I am very new to Nifi and egar to learn it, COuld you please provide me detailed process of it.

Highlighted

how to get data from a txt file (lets say from my desktop ) and insert into kafka topic(In avro format) from nifi processor. I am very new to Nifi and egar to learn it, COuld you please provide me detailed process of it.

New Contributor
 
1 REPLY 1

Re: how to get data from a txt file (lets say from my desktop ) and insert into kafka topic(In avro format) from nifi processor. I am very new to Nifi and egar to learn it, COuld you please provide me detailed process of it.

Super Guru

@Ankit Singh

-> You can use List+Fetch File/SFTP processors to get the file from your local.

-> Then use Record oriented Kafka processors i.e PublishKafkaRecord* processor to read your incoming txt file
- Configure CSVReader controller service and give the matching AVRO schema.

- Configure Record Writer as AvroRecordSetWriter.

-> Now PublishKafkaRecord* processor reads the text file and writes the data in avro format to Kafka topic.

Flow:

1.ListFile (or) ListSFTP..etc
2.FetchFile (or) FetchSFTP..etc
3.PublishKafkaRecord*

Refer to these links for more details link1,link2