Hi Team,   I have an requirement to generate temporary credentials of AWS and fetch files from the S3 bucket  and use this credentials in FetchS3Object But at FetchS3Object processor I am getting below error while fetching files from the bucket:   "FetchS3Object[id=xxx] Failed to retrieve S3 Object for FlowFile[filename=xyz.json]; routing to failure: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider@3075e891: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@2162bad1: Failed to connect to service endpoint: ]"      can you please help me with the issue ... View more

Hi all,   I have an requirement where I need to parse the data into the required format Input:   { "Message" : "\nRecord 1:\nRequired data is missing. \n\nRecord 2:\nprocessing failed\n" }   Here the content and delimiters are not fixed. The fixed part is only /nRecord keyword on which I am writing the Script. But I am not getting desired Output using Groovy.   desired Output:   [{ "Record 1": "nRequired data is missing" }, { "Record 2": "processing failed" }]   I have written Groovy Script for the same but I am getting empty array.   import org.apache.commons.io.IOUtils import groovy.json.* import java.util.ArrayList import java.nio.charset.* import java.nio.charset.StandardCharsets import groovy.json.JsonSlurper import groovy.json.JsonBuilder def flowFile = session.get() if(!flowFile) return try { flowFile = session.write(flowFile, { inputStream, outputStream -> def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) splitted = text.split('\nRecord') int j = splitted.size() final1 = [] for (int i=0;i<j-1;i++) { k = "Record " + splitted[i+1] valid = k.replaceAll("\\n|\"|\\n|}","") final1.add("{\"" + valid.replaceFirst(":",'":"')+ "\"}" ) } def json = JsonOutput.toJson(final1) outputStream.write(JsonOutput.prettyPrint(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) flowFile = session.putAttribute(flowFile, "error", e.getMessage()) session.transfer(flowFile, REL_FAILURE) } Can you please help me with the same. Thank you.  ... View more

I am using ExecuteScript Processor to call the Store procedure. Below is the Groovy Script for the same. import org.apache.commons.io.IOUtils import org.apache.nifi.controller.ControllerService import org.apache.nifi.processor.io.StreamCallback import java.nio.charset.* import groovy.sql.OutParameter import groovy.sql.Sql import java.sql.ResultSet import java.sql.Clob import static java.sql.Types.CLOB import java.nio.charset.StandardCharsets import groovy.json.* def lookup = context.controllerServiceLookup def dbServiceName = ConncationPool.value def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find { cs -> lookup.getControllerServiceName(cs) == dbServiceName } def conn = lookup.getControllerService(dbcpServiceId).getConnection(); def flowFile = session.get() if(!flowFile) return try { sql = Sql.newInstance(conn); flowFile = session.write(flowFile, { inputStream, outputStream -> def inputData = IOUtils.toString(inputStream, StandardCharsets.UTF_8) String sqlString ="""{call MY_PROC(?,?)}"""; def out_res def parametersList = [inputData,Sql.CLOB]; sql.call(sqlString, parametersList) {out_json_response -> out_res = out_json_response?.characterStream?.text }; def json = JsonOutput.toJson(out_res) def parser = new JsonSlurper() outputStream.write(parser.parseText(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) conn?.close() sql.close(); session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) } finally { if (conn != null) conn.close(); if (sql != null) sql.close(); } conn?.close() sql.close(); But I am getting Error "during JSON operations: java.lang.NullPointerException: Cannot invoke method getBytes() on null object while calling store procedure from groovy script" Can you please help me what wrong I am doing here. I am new to the Nifi and groovy. Appreciate your help. Thank you   ... View more

Hi Team,   I am calling store procedure in Nifi using groovy script. The store procedure contains output parameter which is returning huge CLOB which is in array of json format. I am using below groovy script to call the store procedure    import org.apache.commons.io.IOUtils import org.apache.nifi.controller.ControllerService import org.apache.nifi.processor.io.StreamCallback import java.nio.charset.* import groovy.sql.OutParameter import groovy.sql.Sql import java.sql.ResultSet import java.sql.Clob import static java.sql.Types.CLOB import java.nio.charset.StandardCharsets import groovy.json.* def flowFile = session.get() if(!flowFile) return def lookup = context.controllerServiceLookup def dbServiceName = ConncationPool.value def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find { cs -> lookup.getControllerServiceName(cs) == dbServiceName } try { flowFile = session.write(flowFile, { inputStream, outputStream -> def conn = lookup.getControllerService(dbcpServiceId).getConnection(); sql = Sql.newInstance(conn); String sqlString ="""{call MY_STORE_PROC(?)}"""; def out_param def parametersList = [Sql.CLOB]; sql.call(sqlString, parametersList) {out_json_response -> out_param = out_json_response }; def json = JsonOutput.toJson(out_param) def parser = new JsonSlurper() outputStream.write(parser.parseText(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) }   But I am getting below error:   failed to process due to org.apache.nifi.processor.exception.ProcessException: java.lang.StackOverflowError Processing failed: org.apache.nifi.processor.exception.ProcessException: java.lang.StackOverflowError - Caused by: java.lang.StackOverflowError   Can you please help, I am not getting the CLOB datatype issue.  ... View more

Hi,   I have to call the store procedure which is returning the response of huge array of json object. When the data was low its working fine. But it seems due to the huge data store procedure is returning below error while calling it using groovy script: executeScript[id=2c703968-acea-1a72-8299-09548a89bbec] Error during JSON operations: java.lang.reflect.UndeclaredThrowableException - Caused by: java.sql.SQLException: ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1 - Caused by: Error : 6502, Position : 0, Sql = BEGIN MY_DEMO_SP(:1 ); END;, OriginalSql = {call MY_DEMO_SP(?)}, Error Msg = ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1 09:05:14 ISTERROR2c703968-acea-1a72-8299-09548a89bbec ExecuteScript[id=2c703968-acea-1a72-8299-09548a89bbec] Error during JSON operations: java.lang.reflect.UndeclaredThrowableException - Caused by: java.sql.SQLException: ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1 - Caused by: Error : 6502, Position : 0, Sql = BEGIN MY_DEMO_SP(:1 ); END;, OriginalSql = {call MY_DEMO_SP(?)}, Error Msg = ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1     I am using the below script:   import org.apache.commons.io.IOUtils import org.apache.nifi.controller.ControllerService import org.apache.nifi.processor.io.StreamCallback import java.nio.charset.* import groovy.sql.OutParameter import groovy.sql.Sql import java.sql.ResultSet import java.sql.Clob import java.nio.charset.StandardCharsets import groovy.json.* def flowFile = session.get() if(!flowFile) return def lookup = context.controllerServiceLookup def dbServiceName = ConncationPool.value def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find { cs -> lookup.getControllerServiceName(cs) == dbServiceName } try { def slurper = new groovy.json.JsonSlurper() flowFile = session.write(flowFile, { inputStream, outputStream -> def conn = lookup.getControllerService(dbcpServiceId).getConnection(); sql = Sql.newInstance(conn); String sqlString ="""{call MY_DEMO_SP(?)}"""; def out_fg_data def parametersList = [Sql.VARCHAR]; sql.call(sqlString, parametersList) {out_json_response -> out_fg_data = out_json_response }; def json = JsonOutput.toJson(out_fg_data) def parser = new JsonSlurper() outputStream.write(parser.parseText(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) }     Can you please help what's wrong I am doing here.  ... View more

Hi Team,   Need help. I have below two part.   1. merge the original Response with the API Response   Original response [{ "code": "100016", "Date": "2022-09-22" }, { "code": "100017", "Date": "2022-09-02" }, { "code": "100018", "Date": "2022-09-02" }, { "code": "100019", "Date": "2022-09-02" } ]   API Response: [{ "Record": "Record 3", "Description": "Processing failed due Internal Server Error." }, { "Record": "Record 4", "Description": "Processing failed due to error in record: 3" }]   Required output: { "RequestPayload": [{ "code": "100016", "Date": "2022-09-22" }, { "code": "100017", "Date": "2022-09-02" }, { "code": "100018", "Date": "2022-09-02" }, { "code": "100019", "Date": "2022-09-02" } ], "ApiResponse": [{ "Record": "Record 3", "Description": "Processing failed due Internal Server Error." }, { "Record": "Record 4", "Description": "Processing failed due to error in record: 3" }] }   After getting the above output I need to mapped the json objects as below input:   { "RequestPayload": [{ "code": "100016", "Date": "2022-09-22" }, { "code": "100017", "Date": "2022-09-02" }, { "code": "100018", "Date": "2022-09-02" }, { "code": "100019", "Date": "2022-09-02" } ], "ApiResponse": [{ "Record": "Record 3", "Description": "Processing failed due Internal Server Error." }, { "Record": "Record 4", "Description": "Processing failed due to error in record: 3" }] }   I am getting below output:   [ "{\"code\":\"100018\",\"Date\":\"2022-09-22\"}", "{\"code\":\"100019\",\"Date\":\"2022-09-02\"}" ]   Expected Output: [{ "code": "100018", "Date": "2022-09-22", "Record": "Record 3", "Description": "Processing failed due Internal Server Error."." }, { "code": "100019", "Date": "2022-09-02", "Record": "Record 4", "Description": "Processing failed due to error in record: 3" } ]   For that I am using below groovy Script: import org.apache.commons.io.IOUtils import java.nio.charset.StandardCharsets import groovy.json.* import java.util.ArrayList def flowFile = session.get() if(!flowFile) return try { flowFile = session.write(flowFile, { inputStream, outputStream -> def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) def obj = new JsonSlurper().parseText(text) def final1 = [] int j = obj.RequestPayload.size() for (int i=0;i<j;i++){ index = i+1 def rec = 'Record ' + index if (rec in obj.ApiResponse.Record){ final1.add(JsonOutput.toJson(obj.RequestPayload[i])) } } def response1 = JsonOutput.toJson(final1) outputStream.write(JsonOutput.prettyPrint(response1).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) }   Can you please help. I am not familiar with groovy but i want to get the desired output with groovy script. Thanks in Advance        ... View more