Ah I'm so careless, thanks for pointing that out! ... View more

@mburgess .. Hope you are doing good. I am facing issue while creating a groovy script for Scripted Reader Controller service in Convert Record processor. I want read the Flat width file input but script validation is getting failed again and again. It is giving 'groovy.lang.MissingPropertyException: No such property: session for class: Script'   My code is below (also not sure my logic is correct for reading the Fixed Width File as input) Please let me know what's the issue here.    import org.apache.commons.io.IOUtils import java.nio.charset.StandardCharsets import java.nio.charset.UnsupportedCharsetException; import java.io.IOException; import org.apache.nifi.controller.ControllerService flowFile = session.get() if(!flowFile) return try { def text = '' // Cast a closure with an inputStream parameter to InputStreamCallback session.read(flowFile, {inputStream -> def readIN = IOUtils.toString(inputStream, StandardCharsets.UTF_8) names = [] text = readIN.eachLine { names << [ data: it[0..7].trim(), name: it[8..33].trim(), isr: it[34..43].trim() ]} readIN.close() } as InputStreamCallback) session.transfer(flowFile, REL_SUCESS) } catch(e) { log.error("Error while processing the flowfile") session.transfer(flowFile, REL_FAILURE) } ... View more

Hi @Sbofa    Yes you are right. Based on kind it will decide which kind of spark shell needs to start. ... View more

cool tool. I updated to work with 1.12.1: https://gist.github.com/tpanagos/fb8ca4afb16b00862429ffc87dc65348 ... View more

Hi @jerry_pylarinos and @mburgess  I am seeing weird behavior in the same scenario but the only difference is my spec has expression language inside Jspec and it never resolved after pulling from the cache and passing as an attribute to jolttransform. Is that a bug?   like if you jspec.json has [ { "operation": "modify-overwrite-beta", "spec": { "id": "${UUID()}" } } ]   and your generate flow file has  { "id" : "anyname" }   You result looks like this { "id":"${UUID()}" }   How to evaluate expression langauge here? ... View more

@mburgess Helpful information shared. I am using Nifi 1.7.1 For my case, the incremental fetching does not seem to work correctly. All records gets ingested from the database but do not make it all the way to the destination. Processor used is GenerateTableFetch then Execute SQL and the other corresponding processors down the data processing flow. Record id is captured correctly on the GenerateTableFetch property state and its up to date as the record id from the  source (db). However, it will still miss some records when processing the files making the number of records on the destination out of sync with the source from the db. Am i missing something, Would scheduling times for fetching help and how can I do that? ... View more

Hi @Ayaz , @mburgess ! Please have a look at this spec as well! [ { "operation": "shift", "spec": { "*": { "BRANCH_CODE": "[&1].Fields.FLD0001", "CUST_NO": "[&1].Fields.FLD0002", "AC_DESC": "[&1].Fields.FLD0003", "CUST_AC_NO": "[&1].ExternalSystemIdentifier", "#1": "[&1].InstitutionId" } } } ]  Just FYI! ... View more

Hmmm DDL to JSON is not a bad idea. ... View more

Thank you for the response. This was the correct answer, but I was unable to verify until recently.  ... View more

The error message was not very clear but it works for HDF 1.9.2. Thanks a lot! ... View more