Support Questions
Find answers, ask questions, and share your expertise

Script runs in command line, but not in NiFi ExecuteStreamCommand

Script runs in command line, but not in NiFi ExecuteStreamCommand

New Contributor

I am running a Python script that is grabbing a file from S3 and processing it. The script has worked fine until this Tuesday when it started not working with a InvalidToken error (shown below).

ExecuteStreamCommand[id=d6361bf8-3f09-1571-ac80-dcab9236d56e] Transferring flow file StandardFlowFileRecord[uuid=fe47342a-3df4-4867-b535-5a12d459f871,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1559853234093-68, container=default, section=68], offset=944365, length=-1],offset=0,name=penguin/batch1-12/batch1_5M19NR19WN0001/093b7556-5bee-47a3-9877-caeb43ce66a1,size=0] to nonzero status. Executable command /data/penguin_virtualenv/bin/python3 ended in an error: /data/penguin_virtualenv/lib64/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__  return f(*args, **kwds) /data/penguin_virtualenv/lib64/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__  return f(*args, **kwds) Traceback (most recent call last):  File "splitPDF_SeparatorPages_saveS3.py", line 91, in <module>    body = obj.get()['Body'].read()  File "/data/penguin_virtualenv/lib/python3.6/site-packages/boto3/resources/factory.py", line 520, in do_action    response = action(self, *args, **kwargs)  File "/data/penguin_virtualenv/lib/python3.6/site-packages/boto3/resources/action.py", line 83, in __call__    response = getattr(parent.meta.client, operation_name)(**params)  File "/data/penguin_virtualenv/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call    return self._make_api_call(operation_name, kwargs)  File "/data/penguin_virtualenv/lib/python3.6/site-packages/botocore/client.py", line 661, in _make_api_call    raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (InvalidToken) when calling the GetObject operation: The provided token is malformed or otherwise invalid.

Below is the code leading up to the error.

#read in UUID to pull from S3 from stdin
flowFileString = sys.stdin.read()

#Convert the dictionary with filename to just the filename
flowFileFullPath = ast.literal_eval(flowFileString)
flowFileFullPath = flowFileFullPath['filename']

#read in object from the flowFilePath
obj = s3.Object(bucket_name, flowFileFullPath)
    
#read pdf file from object
body = obj.get()['Body'].read()

Running the same input to the script from the command line works perfectly fine though.

109257-1559853932694.png

I've confirmed that the ExecuteStreamCommand processor is using all the correct inputs.

109205-1559853428623.png