Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: AttributeError: 'module' object has no attribute 'DEFAULT_MAX_REDIRECTS

avatar

I'm trying to implement nifi custom processor for google api client via python. However, while importing the libraries, I was getting the Attribute error: @Matt Burgess

ExecuteScript[id=b76c919f-015f-1000-c240-f0d0ade570c2] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: AttributeError: 'module' object has no attribute 'DEFAULT_MAX_REDIRECTS' in <script> at line number 23: javax.script.ScriptException: AttributeError: 'module' object has no attribute 'DEFAULT_MAX_REDIRECTS' in <script> at line number 23
Libaries to Import in the code:
sys.path.append('/usr/local/lib/python3.6/site-packages/googleapiclient') sys.path.append( '/usr/local/lib/python3.6/site-packages/google_api_python_client-1.6.4.dist-info')
sys.path.append( '/usr/local/lib/python3.6/site-packages/google_apputils-0.4.2-py3.6-nspkg.pth') sys.path.append( '/usr/local/lib/python3.6/site-packages/google_apputils-0.4.2.dist-info')
sys.path.append('/usr/local/lib/python3.6/site-packages/oauth2client') sys.path.append( '/usr/local/lib/python3.6/site-packages/oauth2client-4.1.2.dist-info')
sys.path.append('/usr/local/lib/python3.6/site-packages/oauthlib') sys.path.append( '/usr/local/lib/python3.6/site-packages/oauthlib-2.0.6-py3.6.egg-info')

from datetime import datetime, timedelta, date
from googleapiclient.errors import HttpError
from googleapiclient.discovery import build
from oauth2client.client import AccessTokenRefreshError
from oauth2client.service_account import ServiceAccountCredentials
Note:

I'm using the executor script with Python. I understand that is jython based so I added the system path associated with the required libraries. I was getting the same error for oauthclient as well as the googleapiclient libraries. Please let me know what I'm missing. Everything seems to work perfectly locally.

2 REPLIES 2

avatar
Master Guru

You don't need your own sys.path.append calls, you can just put the directories in a comma-separated list in the Module Directory property of ExecuteScript, and it will call sys.path.append for you. However, because it is Jython, if any of the imports (or any of their dependencies) are native CPython modules, then you won't be able to use them in ExecuteScript. All scripts and modules (and dependencies) must be pure Python. For your exact error, I'd have to see the script (where is "module" defined?), but I suspect that one of these libraries is not pure Python.

avatar

Yeah, those libraries are from the google-client-python-api, which is purely based on the python. The error it is giving in terms of Max_Redirects could be the real issue here. It seemed to be the issue within the libraries I specified- Is it the indication that it is reading the libraries.

I got the script working with the execute process (saved lot of pain) but would like to know if the issue is because of the not so pure python libraries or the because of the bug/issue within the libraries.

Further, for me, executeprocess/execute stream seemed so much better as we don't have to bother about using any libraries as the code base grows. The flexibility of controlling the flowfile could be an issue but we can certainly find a workaround using other processors like RouteOnAttribute or something similar as needed. Any comments or inputs on it?