Support Questions
Find answers, ask questions, and share your expertise

ExecuteScript does not finish subprocess module of python

ExecuteScript does not finish subprocess module of python

Hello,

I am using the ExecuteScript processor to execute a shell command. Therefore I tried several functions of the subprocess library (like check_output, check_call, call, Popen etc.), the os library (os.system(cmd)) and the commands library. But they all have the same behavior: It seems that the ExecuteScript Processor does not wait for the subprocess to finish the command, even if I added the .wait() function to do so...

Here is my code:

import subprocess
import os
import time
import commands
import cmd
flowFile = session.get() 
if (flowFile != None):
    filename = flowFile.getAttribute('filename')
    file_id = filename.split('/')[-1]
    route= flowFile.getAttribute('RouteOnAttribute.Route')
    storePath = '/home/user/input/' + route
    log.info('Filename: ' + filename)
    command= 'aws s3 cp s3://<s3_bucket>/'+filename+ ' ' + storePath 
    flowFile  = session.putAttribute(flowFile,'cmd',command)
    # I tried the following commands:
    #ret = commands.getstatusoutput(cmd)
    #ret = subprocess.Popen(cmd,env=os.environ)
    # p = subprocess.check_output([c for c in cmd.split(' ')])
    # p = subprocess.call([c for c in cmd.split(' ')])
    #ret.wait()
    p = subprocess.check_call([c for c in cmd.split(' ')])
    flowFile = session.putAttribute(flowFile,'storePath',storePath + '/' +file_id)
    #flowFile = session.putAttribute(flowFile,'returncode',str(ret[0]))
    #flowFile = session.putAttribute(flowFile,'ouput',str(ret[1]))
if os.path.isfile(storePath + '/' + file_id):
    session.transfer(flowFile, REL_SUCCESS)
else:
    session.transfer(flowFile, REL_FAILURE)

Is there a way to explicity wait for the subprocess to finish its task? I also copied the the 'cmd' attribute value and executed it on the shell without any problems.

Kind regards,

Jan