This seems to be an issue for many but I didn't see much in light for Linux systems.   I've done it all picked up the drivers from https://jdbc.postgresql.org/download.html placed these drives in the nifi-1.11.4/lib/ folder.    I've restarted Nifi, verified the database, verified the filepath, altered the path name with "file:///", altered the path to say just lib/postgresql-42.2.12.jar, attempted to use another database user. I know that the connection information is correct because I connected to the DataGrip UI. I'm completely stumped.  Database Connection URL               jdbc:postgresql://localhost:5432/forestdb Database Driver Class Name           org/postgresql/Driver.class Database Driver Location(s)             /home/fg/Documents/nifi-1.11.4-bin/nifi-1.11.4/lib/postgresql-42.2.12.jar Database User                                  admin Password                                          123 Max Wait Time                                   500 millis     16:03:08 CDT ERROR 01711097-9fd7-1476-e26e-a9bf77b2dff7 StandardControllerServiceNode[service=DBCPConnectionPool[id=01711097-9fd7-1476-e26e-a9bf77b2dff7], versionedComponentId=null, processGroup=StandardProcessGroup[identifier=6428af02-0171-1000-124e-9aeed8d62c82], active=true] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Drive       ... View more

My goal is to pass the data coming in as a JSON to my script and update it to UNIX time. using python. but it is saying line 63 is being referenced before called, "json_data". This is at the session writer near the bottom Processors being used is Convert Record -Execute Script--. The message being passed looks like this: json_data =   {"Year":2018,"DOY":12,"Hour":20,"HGI_Lat_of_the_S/C":7.0,"IMF_B_scalar_nT":1.03,"SW_Plasma_Speed_KMs":441.0}   and should pop out like this   {"HGI_Lat_of_the_S/C": 7.0, "IMF_B_scalar_nT": 1.03, "SW_Plasma_Speed_KMs": 441.0, "Unix_time": 1542074400.0}   Error message:   ExecuteScript[id=01711026-9fd7-1476-8c67-e730c921e164] Failed to process session due to javax.script.ScriptException: UnboundLocalError: local variable 'json_data' referenced before assignment in <script> at line number 63: org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: UnboundLocalError: local variable 'json_data' referenced before assignment in <script> at line number 63   Python code:   import json from datetime import datetime import java.io from org.apache.commons.io import IOUtils from java.nio.charset import StandardCharsets from org.apache.nifi.processor.io import StreamCallback class PyStreamCallback(StreamCallback): def __init__(self): self.year=[] self.day=[] self.hour=[] self.month=[] self.key1_year = 'Year' self.key4_month = 'month' self.key2_day = 'DOY' self.key3_hour = 'Hour' # Write bytes that are utf-8 encoded chine word. def process(self, inputStream, outputStream): text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) #obj = json.loads(text) #json_data = json.dumps(text) json_data = json.loads(json_data) try: self.year.append(json_data[self.key1_year]) except KeyError: self.year.append(2000) try: self.month.append(json_data[self.key4_month]) except KeyError: self.month.append(11) try: self.day.append(json_data[self.key2_day]) except KeyError: self.day.append(11) try: self.hour.append(json_data[self.key3_hour]) except KeyError: self.hour.append(11) new = year+month+day+hour # Considering date is in mm/dd/yyyy format #converting the appendd list to strings instead of ints b=[str(x) for x in new] #joining all the data without adding b = '/'.join(b) #convert to unix dt_object2 = datetime.strptime(b, "%Y/%m/%d/%H") timestamp = datetime.timestamp(dt_object2) json_data.update({'Unix_time':timestamp}) #deleting unwanted data from the dict for func in [self.key1_year,self.key4_month,self.key2_day,self.key3_hour]: try: del json_data[func] except KeyError as e: pass outputStream.write(bytearray(json.dumps(json_data).encode('utf-8'))) flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile, PyStreamCallback()) session.transfer(flowFile, REL_SUCCESS)   ... View more

Is there something about configuring Nifi that needs to be done when setting up?   I ask because I've worked on a few different systems and the one I just set up seems to be having issues with Regex, prepend, and other items.  ... View more

Hi, I'm attempting to replace values in an ASCII file that im importing from a website. I've added the header with replace text but, I'm having a hard time replacing the spaces with commas so, I can convert the file into csv and later to json. Here is what I'm dealing with.    Edit: I should add that sometimes there is one space other times there are two spaces. As of now I have two spaces in the search attribute and a comma in the replace attribute works for part of this problem.     Value     Search Value       Replacement Value   ,   Character Set   UTF-8   Maximum Buffer Size   1 MB   Replacement Strategy   Regex Replace   Evaluation Mode   Line-by-Line   Line-by-Line Evaluation Mode   All   Before: Year,Day,Hour,Minute,ID_for_SW_Plasma_spacecraft,Percent_of_interpolation,Timeshift,RMS_Timeshift,RMS_Min_var,Time_btwn_observation_sec,Field_magnutude_average_nT,BY_nT(GSM),BZ_nT_(GSM),RMS_SD_B_scalar_nT,RMS_SD_field_vector_nT,Speed_km/s,Alfven_mach_number,Magnetosonic_Mach_number,BSN_location_Xgse_Re,2019 1 0 0 51 100 2788 164 0.12 999999 5.11 2.00 2.73 0.08 1.03 451.0 9.8 6.5 13.15 2019 1 0 1 51 100 2810 159 0.12 37 5.10 2.33 2.58 0.11 1.04 451.3 9.8 6.5 13.10 2019 1 0 2 51 80 2852 109 0.09 18 4.86 2.37 2.56 0.12 0.56 454.7 10.3 6.7 13.07 2019 1 0 3 51 67 2951 66 0.06 -39 4.78 2.21 2.55 0.03 0.33 452.3 11.0 6.8 13.00 2019 1 0 4 51 100 3025 7 0.00 -13 4.80 2.17 2.37 0.03 0.14 451.4 11.2 6.8 13.00 2019 1 0 5 99 80 2973 111 0.09 111 4.94 2.68 2.39 0.13 0.55 99999.9 999.9 99.9 13.19 2019 1 0 6 51 67 3074 20 0.02 -40 4.88 2.54 2.01 0.02 0.28 451.0 9.8 6.5 13.27 2019 1 0 7 51 50 3114 9 0.00 19 4.82 2.37 2.93 0.02 0.14 451.0 9.9 6.5 13.29 2019 1 0 8 99 999 999999 999999 99.99 999999 9999.99 9999.99 9999.99 9999.99 9999.99 99999.9 999.9 99.9 9999.99 2019 1 0 9 99 100 3036 0 0.00 999999 5.16 3.34 2.44 0.00 0.00 99999.9 999.9 99.9 13.24 2019 1 0 10 99 100 3036 0 0.00 60 5.16 3.34 2.43 0.00 0.00 99999.9 999.9 99.9 13.24   after: Year,Day,Hour,Minute,ID_for_SW_Plasma_spacecraft,Percent_of_interpolation,Timeshift,RMS_Timeshift,RMS_Min_var,Time_btwn_observation_sec,Field_magnutude_average_nT,BY_nT(GSM),BZ_nT_(GSM),RMS_SD_B_scalar_nT,RMS_SD_field_vector_nT,Speed_km/s,Alfven_mach_number,Magnetosonic_Mach_number,BSN_location_Xgse_Re,2019, 1,0,0 51 100, 2788,,164,0.12 999999,,5.11,,2.00,,2.73,,0.08,,1.03, 451.0, 9.8,6.5, 13.15 2019, 1,0,1 51 100, 2810,,159,0.12,, 37,,5.10,,2.33,,2.58,,0.11,,1.04, 451.3, 9.8,6.5, 13.10 2019, 1,0,2 51,80, 2852,,109,0.09,, 18,,4.86,,2.37,,2.56,,0.12,,0.56, 454.7,10.3,6.7, 13.07 2019, 1,0,3 51,67, 2951,, 66,0.06,,-39,,4.78,,2.21,,2.55,,0.03,,0.33, 452.3,11.0,6.8, 13.00 2019, 1,0,4 51 100, 3025,,,7,0.00,,-13,,4.80,,2.17,,2.37,,0.03,,0.14, 451.4,11.2,6.8, 13.00 2019, 1,0,5 99,80, 2973,,111,0.09,,111,,4.94,,2.68,,2.39,,0.13,,0.55 99999.9 999.9 99.9, 13.19 2019, 1,0,6 51,67, 3074,, 20,0.02,,-40,,4.88,,2.54,,2.01,,0.02,,0.28, 451.0, 9.8,6.5, 13.27 2019, 1,0,7 51,50, 3114,,,9,0.00,, 19,,4.82,,2.37,,2.93,,0.02,,0.14, 451.0, 9.9,6.5, 13.29 2019, 1,0,8 99 999 999999 999999 99.99 999999 9999.99 9999.99 9999.99 9999.99 9999.99 99999.9 999.9 99.9 9999.99 2019, 1,0,9 99 100, 3036,,,0,0.00 999999,,5.16,,3.34,,2.44,,0.00,,0.00 99999.9 999.9 99.9, 13.24 2019, 1,0 10 99 100, 3036,,,0,0.00,, 60,,5.16,,3.34,,2.43,,0.00,,0.00 99999.9 999.9 99.9, 13.24 2019, 1,0 11 99 999 999999 999999 99.99 999999 9999.99 9999.99 9999.99 9999.99 9999.99 99999.9 999.9 99.9 9999.99 2019, 1,0 12 99 999 999999 999999 99.99 999999 9999.99 9999.99 9999.99 9999.99 9999.99 99999.9 999.9 99.9 9999.99 ... View more