Member since
09-15-2020
1
Post
0
Kudos Received
0
Solutions
09-15-2020
03:01 AM
Hi In my enviroment applications writes logs to Kafka. I want to read logs (from all kafka topics) and put to HDFS but i want to do this in one flow processing (one consumer and one PutHDFS processor). It's possible to read from multiple kafka topics (in one consumer processor) and save data to multiple directions by putHDFS processor (one) according to topic name? Each application has a kafka topic and name of topic is from pattern: AppName_app (example: sap_app is a topic for SAP application) In Kafka consumer processor description i see: The name of the Kafka Topic(s) to pull from. More than one can be supplied if comma separated. Supports Expression Language: true Is it mean that i can use as exapmle name of topic like: "^.*_app" In PutHDFS process i want to use direction like: /{$topic_name} to save file in HDFS but it is possible to do it in one processor for multiple topics (data from topics)
... View more
Labels:
- Labels:
-
Apache NiFi