Spark+Pycharm+Pybuilder on Docker

[#1]

How to get a docker image up and running which encapulates a PyCharm IDE integrated with spark and pybuilder. The IDE reside on the docker container and will be display on your laptop/machine. This is to isolate your development enviorment with has spark integrated with spark.

Why? I am a spark developer and spend significant time trying to build a integrated environment. I am spending way too much time on integration before doing what I get paid to do --- Develop! Creating a isolated environment which is integrated with spark and a CIT, easily spun up and down, and repeatable is something which would accelerate my efficiency.

1. Download latest virtualbox from here.
2. To run docker containers or build images a docker machine is required. Download docker machine from here.
3. Download xQuartz to display the IDE on your laptop.
4. View my docker page for information on the docker image here.
5. Clone my PyCharm github repo. You are doing this bootstrap code sample code I have built to your docker container during launch. For example I performed git clone in my /Users/smanjee/docktest
   1. git clone https://github.com/sunileman/pycharm.git
   2. 
6. To start this tutorial start docker machine in a new terminal. For example on my laptop here is the start script
   1. :/Applications/Docker/Docker*app/Contents/Resources/Scripts/start.sh
7. Run docker-machine env to check the IP your machine is assigned (informational only)
8. Pull the image
   1. docker pull sunileman/pycharm
9. Build the image
   1. docker build -t sunileman/pycharm .
10. Open another terminal and start port forwarding
    1. socat TCP-LISTEN:6000,reuseaddr,fork UNIX-CLIENT:\"$DISPLAY\"
11. Get your IP address (not docker machines)
12. Run the image
    1. docker run -it -v /tmp/.X11-unix/:/tmp/.X11-unix/ -v ~/docktest/pycharm/PycharmProjects:/root/PycharmProjects -v ~/docktest/pycharm/.Pycharm40:/root/.PyCharm40 -e DISPLAY=XXX.XX.XX.X:0 --rm sunileman/pycharm
    1. Replace XXX.xx.xx.x with your IP
    2. replace ~/docktest/pycharm/PycharmProjects with your path to pycharm which you downloaded from my github repo
    3. Replace ~/docktest/pycharm/.Pycharm40 with your path to pycharm which you downloaded from my github repo
13. Click on I do not have previous versions
    1. 
14. Click on OK
    1. 
15. Click on OPEN to open the project you mounted to the docker container
    1. 
16. Find the PyCharm project to open
    1. 
17. Now the project has been imported
    1. 
18. So you have the project imported into your IDE which is running within the docker container. To prove the IDE is connected/integrated with spark simply run the python file and you will see spark modules have been imported
    1.