Member since
06-18-2018
39
Posts
0
Kudos Received
0
Solutions
10-20-2022
06:55 AM
Hi Team, I am creating year date time folder thru groovy ExecuteScript processor and using command execute using creating sh file and writing some content in that.. and passing that sh file to downstream ExecuteStreamCommand processor. Its failing with permission error on sh file. How to give full permission to the file which we create. def commmnd1 = "mkdir -p /shared_path/data/poc/test/1/script" def commmnd3 = "mkdir -p /shared_path/data/poc/test/1/output" def process1 = commmnd1.execute(); def process3 = commmnd3.execute(); def filePath = "${script_dir}/abcd.sh" def file2 = new File(filePath) file2.write "#!/bin/bash\n"; and one more, ExecuteStreamCommand executing sh file and it generating some files in dynamically created by previous ExecuteScript processor and these are files does not have full permission. How to give full permission which we create folder and files dynamically. Please suggest. Thanks in Advance.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
10-12-2022
07:57 PM
Hi Team, I am trying to calling jar utility in Nifi thru Execute Script processor groovy command like below. #1. when i give without double quotes in command java params, its taking java class and executing and giving output. #2. when i try to add double quotes to params inside java command, its not giving any output, its giving null pointer exception and main method not calling. #3. How to escape double quotes inside command and how to call the same in Nifi ? #4. Guys, please suggest me one below use case. flowFile = session.get() if(!flowFile) return try { def command = "java -cp \"/base_dir/poc/lib/dentine/*\" com.abc.xyz.Engine -f=\"/base_dir/input/abc.xlsx\" -o=\"/base_dir/output/\" -av_Log_Directory=\"/base_dir/logs/\" " def process = command.execute() def outputStream = new StringBuffer(); def errStream = new StringBuffer(); process.waitForProcessOutput(outputStream, errStream) flowFile = session.putAttribute(flowFile, "process.exitValue()" , process.exitValue().toString()) session.transfer(flowFile, REL_SUCCESS) } catch(e) { flowFile = session.putAttribute(flowFile, "SCRIPT_EXECUTION", "FAILED") flowFile = session.putAttribute(flowFile, "SCRIPT_ERROR_MESSAGE", e.getMessage()) session.transfer(flowFile, REL_FAILURE) }
... View more
Labels:
- Labels:
-
Apache NiFi
07-25-2022
08:39 PM
Hi All, In Nifi flow, I have below stored procedure need to call in NiFi flow to get auto increment key as flow file attribute. Incoming flow file attribute file_id will have value and same is inserting into table with stored procedure call and return value as auto increment key. In MySQL dB i am able to call this procedure and getting auto increment key as return value. In NIFI, i need to call this stored procedure and get auto increment key as flow file attribute... this procedure call should not affect my existing flow file content. I need help on how to call stored procedure and how to assign return value into flow file attribute. please help me on this. with this ExecuteSQL processor.. i am able to insert data into table.. but not able to get return value and store it into flow file attribute. //table creation CREATE TABLE TRANS_ID_GENERATOR( TRANSACTION_ID INT(11) PRIMARY KEY AUTO_INCREMENT, FILE_ID INT(11) ); //procedure to add file_id into table and get incremented value BEGIN INSERT INTO TRANS_ID_GENERATOR(FILE_ID) VALUES (100); SELECT COUNT(TRANSACTION_ID) INTO OUT_SEQ_NUMBER FROM TRANS_ID_GENERATOR; END Thanks, Rangareddy Y
... View more
Labels:
- Labels:
-
Apache NiFi
07-20-2022
07:48 AM
Thanks @steven-matison its is working now.. import java.nio.charset.StandardCharsets import org.apache.commons.io.IOUtils import java.nio.charset.* import groovy.io.FileType import java.io.* def flowFile = session.get() if(!flowFile) return def flowFiles = [] as List<FlowFile> def list = [] def dir = new File("/zyme_shared/nas_data/poc/zgw/output1/") dir.eachFileRecurse (FileType.FILES) { file -> list << file } list.each{ i -> def flowFile2 = session.create(flowFile) flowFile2 = session.putAttribute(flowFile2, "filename" , i.name) flowFile2 = session.write(flowFile2, { outputStream -> outputStream.write(i.getBytes()) } as OutputStreamCallback) flowFiles << flowFile2 } session.transfer(flowFiles, REL_SUCCESS) session.remove(flowFile) session.commit()
... View more
07-20-2022
06:50 AM
Hi All, Getting error in executing below script thru Execute Script processor. Requirement : upper flow will copy files to some folder.. trying to read files from folder and post them to http processor. For each file i am trying to create a new flowfile and move the same to HTTP processor. But below code failing. Can you please suggest what i am missing. -------------------------------- import java.nio.charset.StandardCharsets def flowFile = session.get() if(!flowFile) return def flowFiles = [] as List<FlowFile> def inputStream = session.read(flowFile) def list = [] def dir = new File("/root/file_data/poc/output/") dir.eachFileRecurse (FileType.FILES) { file -> list << file } list.each{ i -> def newFlowFile = session.create(flowFile) newFlowFile = session.write(newFlowFile, { outputStream -> outputStream.write( i.getBytes(StandardCharsets.UTF_8)) } as OutputStreamCallback) flowFiles << newFlowFile } session. Transfer(flowFiles, REL_SUCCESS) session.remove(flowFile) Thanks in advance!
... View more
Labels:
- Labels:
-
NiFi Registry
06-16-2022
07:44 PM
Hi Team, I just started working on the apache execute script processor. please help me on this. I am getting issues to read files from remote directory of multiple connections and pass all files to next processor. Here I can not use list processor since it is asking static host and port.. so I want to write groovy script to connect remote server and read all files and hand over them to next processor. My seniors suggested to write groovy script to complete this activity. I need help on how to connect remote server and read all files and how to add all files to flow file and how to hand over it to next processor ? Input parameters are 1) username 2) password 3) host 4) port 5) directory path Can you guys suggest me how to get this. Thanks, Rangareddy Y
... View more
Labels:
- Labels:
-
Apache NiFi
08-23-2018
07:01 AM
@ashok.kumar, it's working fine...whatever the folder paths i mentioned in path filter, processor is listing those folder files only. Thanks, but, if any file exist in input directory path /company_shared/ , those file also processor is listing... is there any way we can exclude those files... in my case, we do no keep any files in /compan_shared/ home path. no issue for me. Thanks, Rangareddy Y
... View more
08-22-2018
07:21 AM
@ashok.kumar, can you suggest me how path filter for two folders..i tried path filter, if i give one path it is taking, how to add the other one into path filter.. properties given like below input Directory : /company_shared/nas_data/home/ Path filer : company_shared/nas_data/home/walmart/inbox/ It is reading all the folder files exist on home path. There are other folders in home path, we should not read by listFile processor. Here how to add other path in path filter, please suggest me. Thanks, Rangareddy Y
... View more
08-22-2018
05:41 AM
Hi Team, i want to read files from two different folders. used getFile processor to read files. in properties section i have given one folder path, processor read the files from folder as expected. when i tried to add the other folder name into input directory property it is showing invalid processor. can you please suggest me how to give two folder paths in inptuDirectory property. 2 folder paths: /company_shared/nas_data/home/walmart/inbox/ /company_shared/nas_data/home/target/inbox/ used pipe to concatenate two folder paths. getFile processor: inputDirectory property : /company_shared/nas_data/home/ryemireddy/inbox/|/company_shared/nas_data/home/krishna/inbox/ ERROR: invalid, because directory does not exist. Please suggest me how to give the exact value for inputDirectory property. Thanks, Rangareddy Y
... View more
Labels:
- Labels:
-
Apache NiFi
08-21-2018
11:17 AM
@Steven Matison /homepath/customer_*/inbox/ is giving error. processor is invalid and getting directory does not exist error.
... View more