Have a process that I started yesterday. It was working and processing flow files. It is pulling from SQL and pushing to SQL and the data pull has roughly 15 million rows. When I started yesterday it had 29 files in the queue (where is says 27 now). It did process the initial 2 flow files. This morning I come in and it stopped processing the files right before the ExecuteSQL processor. I read on another thread about number of threads, it was set to 20 and I increased to 30 and then 50 with no change. I did stop the processor and restart, no luck. The server is clustered but I am only using it on a single node. Attached are images of the processing and how it essentially stopped, the queued up files, the schedule and properties of the executesql processor. Thanks in advance! ... View more

Trying to make a generic template that uses some input variables (as what columns to pull) and then put them into a sql destination table. I have it working.....if it's strings but recently was using my template and pulling in Maximo data (don't get me started).....well it will have INT columns and then have nulls in them which messes up everything. So plan was to convert the incoming Avro schema to an all string element avro schema (which I can do)......but my issue is that within the controller, I have to specify each column name in the AvroRecordSetWriter. Is there a way to say field1 is "Field1" AND copy over the value from the original avro schema index[0] element and put it as Field1 in the new all strings Avro schema? In the end....I'm trying to convert incoming schema to have ALL strings and to have the schema/flowfile has element names as "field1", "field2", etc. Thanks in advance! ... View more

New to Nifi but here is my thought, I am trying to make a template for a flow. I would prefer to create as many of the processors to read everything from the variables at the flow level. Essentially....the template would be used to copy data from a sql db to another sql db....only changes would be what server, what copy from/copy to tables by upserting, table columns etc. Will also be adding additional fields for a business and content hashes. Because of the sql and dynamic-ness, I have to use sql.args.$.value to use in the insert statement. So thought is to grab the values and store in a value called "Field##" where ## is the number from the "SourceField" variable (must be in order)....then use this "Field##" to populate the sql args value and then use another variable to populate the destination table (most likely a different column name). So from my thoughts, in the attributes of the flow file, I would have the following....setting from the initial source db query. Field1 "test1" Field2 "test2" In a groovy script I would get the number of "source variables" and then do a for loop and assign all of the sql args from the Field## attributes. Something like: for (int i = 1; i < countargs; i++) { def myType = "sql.args." + i + ".type" ff = session.putAttribute(ff, myType, '-9') //setting it to use a string type def myValue = "sql.args." + i + ".value" ff = session.putAttribute(ff, myValue, '${Field" + i + '}') } The creating of the sql.arg with a # works as it's just a concat string. But I would like the for loop to set the following: sql.args.1.value = test1 sql.args.2.value = test2 Any thoughts appreciated. ... View more