quick question -- do we have any front-end for Python (like we have R Studio for R)?
https://zeppelin.apache.org/ is great for Python and Spark and is included in HDP 2.5
Here is a list of other options:
http://www.pydev.org/ on Eclipse is nice
General text editors like TextWrangle and Sublime Text are good for Python.
I like using VI and Zeppelin for PySpark
Just to follow up on what @Artem Ervits said. Pycharm is good IDE and I have built docker container for it here
Pycharm sits inside the container and the UI will be rendered back to your workstatin. it also have spark client installed allowing you to connect to a spark cluster.