Hi Team,   How to invoke api of Content-Type multipart/form-data using invokehttp processor in nifi. It's not working for me.    If anyone has used the same, can you please share. Thank you. ... View more

Hi Team,   I have a requirement, where I am using POST API of form-data using invokehttp. it is expecting few form-data key-values along with the json file that we need to send to the respective server using post API.   But I am getting below error: when Content-Type: multipart/form-data Error: IOException thrown from InvokeHTTP[id=xyzzzzz]: java.net.SocketException: Broken pipe (Write failed)   when Content-Type: application/form-data Error:  IOException thrown from InvokeHTTP[id=xyzzzz]: java.net.SocketException: Connection reset   I did investigation but not able to solve the issue. Can you please help. ... View more

Hi Team,   I have an requirement to generate temporary credentials of AWS and fetch files from the S3 bucket  and use this credentials in FetchS3Object But at FetchS3Object processor I am getting below error while fetching files from the bucket:   "FetchS3Object[id=xxx] Failed to retrieve S3 Object for FlowFile[filename=xyz.json]; routing to failure: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider@3075e891: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@2162bad1: Failed to connect to service endpoint: ]"      can you please help me with the issue ... View more

Hi all,   I have an requirement where I need to parse the data into the required format Input:   { "Message" : "\nRecord 1:\nRequired data is missing. \n\nRecord 2:\nprocessing failed\n" }   Here the content and delimiters are not fixed. The fixed part is only /nRecord keyword on which I am writing the Script. But I am not getting desired Output using Groovy.   desired Output:   [{ "Record 1": "nRequired data is missing" }, { "Record 2": "processing failed" }]   I have written Groovy Script for the same but I am getting empty array.   import org.apache.commons.io.IOUtils import groovy.json.* import java.util.ArrayList import java.nio.charset.* import java.nio.charset.StandardCharsets import groovy.json.JsonSlurper import groovy.json.JsonBuilder def flowFile = session.get() if(!flowFile) return try { flowFile = session.write(flowFile, { inputStream, outputStream -> def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) splitted = text.split('\nRecord') int j = splitted.size() final1 = [] for (int i=0;i<j-1;i++) { k = "Record " + splitted[i+1] valid = k.replaceAll("\\n|\"|\\n|}","") final1.add("{\"" + valid.replaceFirst(":",'":"')+ "\"}" ) } def json = JsonOutput.toJson(final1) outputStream.write(JsonOutput.prettyPrint(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) flowFile = session.putAttribute(flowFile, "error", e.getMessage()) session.transfer(flowFile, REL_FAILURE) } Can you please help me with the same. Thank you.  ... View more

I am using ExecuteScript Processor to call the Store procedure. Below is the Groovy Script for the same. import org.apache.commons.io.IOUtils import org.apache.nifi.controller.ControllerService import org.apache.nifi.processor.io.StreamCallback import java.nio.charset.* import groovy.sql.OutParameter import groovy.sql.Sql import java.sql.ResultSet import java.sql.Clob import static java.sql.Types.CLOB import java.nio.charset.StandardCharsets import groovy.json.* def lookup = context.controllerServiceLookup def dbServiceName = ConncationPool.value def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find { cs -> lookup.getControllerServiceName(cs) == dbServiceName } def conn = lookup.getControllerService(dbcpServiceId).getConnection(); def flowFile = session.get() if(!flowFile) return try { sql = Sql.newInstance(conn); flowFile = session.write(flowFile, { inputStream, outputStream -> def inputData = IOUtils.toString(inputStream, StandardCharsets.UTF_8) String sqlString ="""{call MY_PROC(?,?)}"""; def out_res def parametersList = [inputData,Sql.CLOB]; sql.call(sqlString, parametersList) {out_json_response -> out_res = out_json_response?.characterStream?.text }; def json = JsonOutput.toJson(out_res) def parser = new JsonSlurper() outputStream.write(parser.parseText(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) conn?.close() sql.close(); session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) } finally { if (conn != null) conn.close(); if (sql != null) sql.close(); } conn?.close() sql.close(); But I am getting Error "during JSON operations: java.lang.NullPointerException: Cannot invoke method getBytes() on null object while calling store procedure from groovy script" Can you please help me what wrong I am doing here. I am new to the Nifi and groovy. Appreciate your help. Thank you   ... View more

Hi Team,   I am calling store procedure in Nifi using groovy script. The store procedure contains output parameter which is returning huge CLOB which is in array of json format. I am using below groovy script to call the store procedure    import org.apache.commons.io.IOUtils import org.apache.nifi.controller.ControllerService import org.apache.nifi.processor.io.StreamCallback import java.nio.charset.* import groovy.sql.OutParameter import groovy.sql.Sql import java.sql.ResultSet import java.sql.Clob import static java.sql.Types.CLOB import java.nio.charset.StandardCharsets import groovy.json.* def flowFile = session.get() if(!flowFile) return def lookup = context.controllerServiceLookup def dbServiceName = ConncationPool.value def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find { cs -> lookup.getControllerServiceName(cs) == dbServiceName } try { flowFile = session.write(flowFile, { inputStream, outputStream -> def conn = lookup.getControllerService(dbcpServiceId).getConnection(); sql = Sql.newInstance(conn); String sqlString ="""{call MY_STORE_PROC(?)}"""; def out_param def parametersList = [Sql.CLOB]; sql.call(sqlString, parametersList) {out_json_response -> out_param = out_json_response }; def json = JsonOutput.toJson(out_param) def parser = new JsonSlurper() outputStream.write(parser.parseText(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) }   But I am getting below error:   failed to process due to org.apache.nifi.processor.exception.ProcessException: java.lang.StackOverflowError Processing failed: org.apache.nifi.processor.exception.ProcessException: java.lang.StackOverflowError - Caused by: java.lang.StackOverflowError   Can you please help, I am not getting the CLOB datatype issue.  ... View more

Hi,   I have to call the store procedure which is returning the response of huge array of json object. When the data was low its working fine. But it seems due to the huge data store procedure is returning below error while calling it using groovy script: executeScript[id=2c703968-acea-1a72-8299-09548a89bbec] Error during JSON operations: java.lang.reflect.UndeclaredThrowableException - Caused by: java.sql.SQLException: ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1 - Caused by: Error : 6502, Position : 0, Sql = BEGIN MY_DEMO_SP(:1 ); END;, OriginalSql = {call MY_DEMO_SP(?)}, Error Msg = ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1 09:05:14 ISTERROR2c703968-acea-1a72-8299-09548a89bbec ExecuteScript[id=2c703968-acea-1a72-8299-09548a89bbec] Error during JSON operations: java.lang.reflect.UndeclaredThrowableException - Caused by: java.sql.SQLException: ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1 - Caused by: Error : 6502, Position : 0, Sql = BEGIN MY_DEMO_SP(:1 ); END;, OriginalSql = {call MY_DEMO_SP(?)}, Error Msg = ORA-06502: PL/SQL: numeric or value error ORA-06512: at line 1     I am using the below script:   import org.apache.commons.io.IOUtils import org.apache.nifi.controller.ControllerService import org.apache.nifi.processor.io.StreamCallback import java.nio.charset.* import groovy.sql.OutParameter import groovy.sql.Sql import java.sql.ResultSet import java.sql.Clob import java.nio.charset.StandardCharsets import groovy.json.* def flowFile = session.get() if(!flowFile) return def lookup = context.controllerServiceLookup def dbServiceName = ConncationPool.value def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find { cs -> lookup.getControllerServiceName(cs) == dbServiceName } try { def slurper = new groovy.json.JsonSlurper() flowFile = session.write(flowFile, { inputStream, outputStream -> def conn = lookup.getControllerService(dbcpServiceId).getConnection(); sql = Sql.newInstance(conn); String sqlString ="""{call MY_DEMO_SP(?)}"""; def out_fg_data def parametersList = [Sql.VARCHAR]; sql.call(sqlString, parametersList) {out_json_response -> out_fg_data = out_json_response }; def json = JsonOutput.toJson(out_fg_data) def parser = new JsonSlurper() outputStream.write(parser.parseText(json).getBytes(StandardCharsets.UTF_8)) } as StreamCallback) session.transfer(flowFile, REL_SUCCESS) } catch(Exception e) { log.error('Error during JSON operations', e) session.transfer(flowFile, REL_FAILURE) }     Can you please help what's wrong I am doing here.  ... View more