I'm specifically using pyspark and I'm wondering if there is something similar to Pig's "aliases" command that shows all currently available variables. If there is something like that in pyspark, I'm just missing it and I hope someone straightens me out! ;-) I'm not using spark-shell much, but knowing how to do this in that REPL would be useful, too.
The pyspark shell is just Python too. So using dir() should show all existing python variables (although it also shows all imports and a bunch of things you may not be looking for).
If you keep using `globals()` you will eventually get an error, as it keeps adding itself to itself and you eventually get one of the following errors:
RuntimeError: maximum recursion depth exceeded while getting the repr of a list
RuntimeError: dictionary changed size during iteration