Member since
06-27-2019
2
Posts
0
Kudos Received
0
Solutions
06-28-2019
06:19 AM
Thank you for your answer. The problem is ListHDFS is a starting processor - it does not accept incoming connections, so I can't provide the changing input directory using a flowfile.
... View more
06-27-2019
06:38 PM
Hello, In my flow I need to copy HDFS files based on dynamic wildcard to another HDFS location within the same cluster. I have Process Group Variables: source_path = 'hdfs:///source/' file_prefix = 'myflow_' And a Flowfile Attribute: file_timestamp = '20190520' The source directory contains 4 files, and I need to copy the bolded two of them. The filenames to be copied are "${source_path}${file_prefix}${file_timestamp}.part*". hdfs:///source/myflow_20190412.part000 hdfs:///source/myflow_20190520.part000 hdfs:///source/myflow_20190520.part001 hdfs:///source/otherflow_20190625.part000 The MoveHDFS processor in NiFi v1.8.0 does not support Expression Language in the File Filter Regex field. How could I achieve this functionality - except for using ExecuteStreamCommand with "hdfs dfs -cp"? Thank you for your help, Piotr
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi