In an upcoming release you will be able to keep stateful variables using UpdateAttribute (NIFI-1582), the author envisions it being able to support running averages. Also I started an AggregateValues processor under NIFI-2735, but haven't been able to finish it yet, and it works on micro-batches (such as files created from a split processor like SplitJson) rather than rolling windows. In the meantime if you are familiar with a scripting language such as Groovy, Jython, Javascript, or JRuby you could use ExecuteScript or InvokeScriptedProcessor, they have access (via the ProcessContext) to the StateManager, where you could keep state, averages, etc, across flow files over time. ... View more

When using the Jython script engine in ExecuteScript, the Module Directory property works somewhat like the standard Python module system. I believe you'll want to have the location(s) specified as the directories containing the scripts, not the scripts themselves. Each entry in Module Directory is added to the script engine via a "sys.path.append()" call, so if you can get the modules to load for a regular python script (using sys.path.append() with full path names), then you can use that same list for Module Directory. I'd try: C:\Users\pjalla\AppData\Local\Programs\Python\Python35-32\Lib\site-packages\requests,C:\Users\pjalla\AppData\Local\Programs\Python\Python35-32\Lib\json,C:\Users\pjalla\AppData\Local\Programs\Python\Python35-32\Lib\site-packages\requests_oauthlib or whichever variations to avoid the drive letter, backslashes, etc. In this unit test you can see the Module Directory set to a test resources folder, the test uses "from callbacks import ReadFirstLine", and there are .py files in the "callbacks" folder under the test resources folder specified as Module Directory. Hopefully you can use a similar approach to have the script engine find the folders/locations and import the corresponding code. ... View more

What error are you getting and what approach are you using (and what version of NiFi/HDF)? I recommend setting up a DBCPConnectionPool controller service. Your settings for the controller service will depend on your MySQL instance, here is an example of one I use to connect to a local MySQL running on Docker, using test_schema as the database: The "Database Driver Location(s)" property is set to a URL pointing at my mysql-connector-java-5.1.38-bin.jar. As of NiFi 1.0.0 (via NIFI-2604), this property can include a comma-separated list of folders and/or JARs, and need not be URLs (i.e. you can use relative path names, although according to this post I don't think having the Windows drive letter at the front will work). ... View more