@Abdelkrim Hadjidj @Matt Burgess The use case here is getting the files from Mainframe, once a each day, however I got to know that there is no concept directory structure in Mainframes ( no idea how mainframes work), so it is not able to list the files (GetFTP as well as ListFTP and FetchFTP) Is there any other way to get around? I read some blogs and answers which suggested to use syncsort or Informatica powercenter. We tried using our current way of running shell script which goes and fetch the files. We can run the script using executeProcess and save it on one of the nodes (primary nodes), however primary node keeps changing. yesterday it was different primary today it is different one. In addition, if we mount it to share the directory across the node, it will be against the policies (too much admin work) Any help, thoughts ... View more

Hi Matt, Do you have any template for that ? ... View more

Thx for answer. We are using a older version today. As showed only some of the fields is supported for Expression Language. HDF Version 2.1.1 - Powered by Apache NiFi - Version 1.1.0.2.1.2.0-10 ... View more

In Apache NiFi 1.5.0 (not yet released at the time of this writing), SelectHiveQL (via NIFI-4473) will have a property to Normalize Names for Avro, so you won't have to do the alias. ... View more

@Abdelkrim Hadjidj Perfect! Much clearer now. Thanks. ... View more

@Hameed Abdul This is the expected behavior. Your file name is local to your client environment and NiFi have no clue on where it comes from. You need to send this information together with your file. The number that you are seeing is the ID that NiFi generated for this received flow file. How to send data depends on your client. If I use CURL to upload data I can do curl --form "fileupload=@/tmp/file1.txt;filename=file1.txt" localhost:7878 and I receive the following in NiFi --------------------------2c1843649b5760e1 Content-Disposition: form-data; name="fileupload"; filename="file1.txt" Content-Type: text/plain this is file 1 --------------------------2c1843649b5760e1-- You can use ExtractGrok or ExtractText to get the filename and update the flow file attribute. EDIT: As I said, this depends on your client. I did the same test with Postman and here's how to get the right information. Add a header to your POST query. I added a header called originalfilename I get this information as an attribute of my flow file With this you only need to use UpdateAttribute processor to copy this attribute to filename. Hope this helps ... View more

Try || instead of CONCAT or +, the former is the standard and the latter are not, according to this. ... View more

Hi, New question added here: https://community.hortonworks.com/questions/135939/compare-attributes-of-different-flowfiles.html Hope you can help me. Jon ... View more