hi @kumsath, If you want to split your output from ExecuteSQL, you could try replacing ConvertAvroToJson with a SplitRecord in which you will configure the Record Reader as AVRO and the Record Writer as JSON, whereas the Records per Split can be set by you to any value. In this way, your ExecuteSQL results are always split into smaller chunks, based on your desired output. Of course, you can use the NiFi Expression Language and calculate the Records per Split dynamically and so on. ... View more

As I can see, the problem is not strictly related to NiFi, but to the query you are executing. If you try and execute this query directly on your Database, does it work? You first need to identify where the issue occurs, when you query the database or when you make the conversion. If the problem occurs when fetching the data, I think you will need to use the convert function from sql to achieve your goal. Something like: CONVERT(VARCHAR,your_date_Value,103), where your_date_value is your date and 103 is the date format, which can be found here: https://www.mssqltips.com/sqlservertip/1145/date-and-time-conversions-using-sql-server/ This is just an example of that function, you will need to apply it based on your use case. You could also try fetching another column instead of dt_change to further debug your flow. ... View more

Hi @Daniel_AB, As far as I know, when executing QueryDatabaseTable, the output you have is in AVRO Format. In order to fully use the AVRO capabilities and the data types, I would recommend to set the property "Use Avro Logical Types" to true and test how your data look like afterwards. Do you have it on false by any chance? ... View more

As far as I know, there is no Kotlin in the software stack ... but you should wait for an answer from somebody with more experience. Nevertheless, in the meantime, you can have a look for yourself, as the code is public: https://github.com/apache/nifi ... View more

Now, based on your Flow Logic, where do you encounter problems and what are those problems? In addition, how do you retrieve dynamically the destination and the size at destination? As far as I know, PutFile does not write those attributes so you will have to manually define them in your Flow and I do not see any UpdateAttribute or UpdateContent on your canvas. The size, the file type, the filename and the source are written by default when the file is extracted and your can further use them with the help of the NiFi Expression Language. ... View more

hi @NafeesShaikh93, What logs are you trying to push into Graylog? The logs generated by NiFi when transferring the files from Source to Target, or are you talking about other logs?   What have you tried and why did it not succeed? Let's start with that, as your Flow Logic was built specific for your use case and somebody else's logic might not suite your needs. ... View more

Hi @ManishR, 1. How to find the processor or processor group by Integration Name/Folder/Event Name? What do you mean by integration name, event name and folder? Folder I assume you refer to Processor Group (external or normal) but integration name and event name are unknown to me. If you are trying it identify Processors, Processor Group, Queues or any other objects from within your Canvas Board, you mostly have two options: 1-you either use the NiFi REST Api (https://nifi.apache.org/docs/nifi-docs/rest-api/index.html ) or 2-you use the search bar in the top right of the screen. 2. How to find the source and destination file details in any of the processor or processor group? What do you mean with this question? In NiFi there are a series of processors which can be used to retrieve SOURCE files (ListSFTP/FetchSFTP/GetSFTP,GenerateFlowFile,InvokeHTTP,ConsumeKafka,ExecuteSQLRecord,etc) and there is a series of processors used to save your data in a TARGET location (like PutFile,PublishKafka, PutGCSObject, PutHDFS, PutDatabaseRecord,PutS3Object, etc). If you are to speak about a certain processor, the source data is within the queue linked to the specific processor, whereas the target data is located within the queue linked from your specific processor to another processor. Yet again, your question is quite general and ambiguous to get a specific answer.   Sorry but I do not understand what exactly you are trying to achieve so if you require a more technical and more elaborate answer, you will have to come with a more detailed questions, based on a specific use case - maybe even your NiFi Flow. ... View more

@ManishRcan you please describe your problem a little bit better? What did you try to do? Why did it not work? What should have been the expected outcome? What are you actually trying to achieve?   What you have provided so far is quite general like " which is the best car in the world " / " how can I find the best milk " / "who came first? the chicken or the egg". As you described your question, the answer is: your existing processors are present on the canvas. The files are in the source, the target or the queues between the processors, whereas the integrations are the integrations which you have constructed or used so far. ... View more

@ManishRcan you please describe your problem a little bit better? What did you try to do? Why did it not work? What should have been the expected outcome? What are you actually trying to achieve?   What you have provided so far is quite general like " which is the best car in the world " / " how can I find the best milk " / "who came first? the chicken or the egg". As you described your question, the answer is: you can find the source files in the source and the target files in the target, where the source is what you have defined your source to be and the target is the target you have set. ... View more