@Vengai Magan Please refer to this and this links describes how to install NiFi as Service and this link to setup high performance NiFi. ... View more

@Raju Chigicherla Instead of splitting the file in one SplitText processor try with series of SplitText/SplitContent processors to split the 10GB file. (or) Use record oriented processors like SplitRecord and configure the processor to records per split that gives 50MB files, if you are still having issues with SplitRecord processor then use series of SplitRecord processors to get 50MB files. In addition to split xml files NiFi 1.7 introduced XmlReader/Writer controller services by using them we can split xml data in split record processor. Refer to this and this links to split big file by using series of Split processors. ... View more

@Vengai Magan To parse Fixed width file you can use Replace Text processor and keep the matching regex that extracts each field into a capture group then use some delimiter while replacing the data as shown above if you know how many words that each field is going to be then capture each field into one capture group then replace the content with some delimiter. In addition if your file having space delimiter then use this (.*)\s(.*) regex and replace with some delimiter. Change the ReplaceText evolution mode to line-by-line, Now we are reading fixed width file then replacing the contents of flowfile with some delimiter. Once you have delimiter on field then you can use Convert Record processor to read and write the data in your required format. If the file is couple of gigs then it's better to split the file to small chunks before ReplaceText processor then feed the splitted file to Replace Text processor. In addition there is scripted reader/writer controller service in ConvertRecord processor which allows to read the incoming flowfile by using the script that you have given and writes the flowfile contents as per your ScriptedWriter controller service configured. Some references regarding parsing fixed width file are here and here, References regarding splitting big csv file into smaller chunks are here and here ... View more

@Souveek Ray Use regexp_extract function as you are doing split on space, write an matching regex to extract until first space as first word and next capture group excludes the first word. Example: I'm having firstword secondword thirdword as a value hive> select regexp_extract("firstword secondword thirdword","^(.*?)\\s(.*)",1)first_word, regexp_extract("firstword secondword thirdword","^(.*?)\\s(.*)",2) not_first_word; +-------------+-----------------------+--+ | first_word | not_first_word | +-------------+-----------------------+--+ | firstword | secondword thirdword | +-------------+-----------------------+--+ So in the above example we have extracted the first word by using regexp_extract and 1 capture group and in the next we have used regexp_extract and 2 capture group(excluding firstword). ... View more

@Murat Menteşe As your xml doc having array [] in it and i'm not sure how to write matching xslt. As the current xslt converts the array xml into object/element and adding "" for array[]. In case of large data you have to increase the Maximum Buffer Size 1 MB //increase based on your flowfile size as this processor works takes the whole flowfile into memory and does all replacing based on our configs. ... View more

@Amira khalifa Use one of the way from the above shared link to take out only the header from the csv file then in replace text keep the then search for (&|\(|\)|\/_|\s) and in Replacement value keep as empty string, now we are searching for all the special characters in the header flowfile then replacing with empty string.Now add this header flowfile with the other non header flowfile. all the explanation and template.xml are shared in this link. ... View more

@Murat Menteşe Could you share your sample xml file data so that i can reproduce the issue on my side by converting to json format? ... View more