While I am running an oozie command : oozie job -oozie http://sandbox.hortonworks.com:11000/oozie -config job.properties -run getting an error as Error: E0803 : E0803: IO error, E0603: SQL error in operation, <openjpa-2.2.2-r422266:1468616 fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. FailedObject: org.apache.oozie.WorkflowJobBean@6336a3e5 Please find my workflow.xml <workflow-app name="WorkFlowForShellActionWithCaptureOutput" xmlns="uri:oozie:workflow:0.4"> <start to="shellAction"/> <action name="shellAction"> <shell xmlns="uri:oozie:shell-action:0.4"> <job-tracker>${JobTracker}</job-tracker> <name-node>${namenode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${QueuName}</value> </property> </configuration> <exec>${recordShellScript}</exec> <file>${recordShScriptPath}#${recordShellScript}</file> <capture-output/> </shell> <ok to="end"/> <error to="killAction"/> </action> <kill name="killAction"> <message>"Killed job due to error"</message> </kill> <end name="end"/> </workflow-app> job.properties as namenode=hdfs://sandbox.hortonworks.com:50070 JobTracker=hdfs://sandbox.hortonworks.com:8050 QueuName=default oozie.libpath=/L301/ oozie.use.system.libpath=true oozie.wf.rerun.failnodes=true appPath=/L301/workfowShell.xml recordShScriptPath=/L301/recordtime.sh r recordShellScript=recordtime.sh ... View more

how to configure multiple broker using python API ... View more

I need to access python modules in pyspark shell, so use python easy-install -m tweepy, and its get downloaded in site-package directory(/usr/lib/python2.6/site-packages/tweepy-3.5.0-py2.6.egg) . And I try to add to the module to the pyspark shell using sc.addPyFile("/usr/lib/python2.6/site-packages/tweepy-3.5.0-py2.6.egg"). But I am getting an error as Traceback (most recent call last): File "", line 1, in File "/usr/hdp/2.5.0.0-1245/spark/python/pyspark/context.py", line 803, in addPyFile self.addFile(path) File "/usr/hdp/2.5.0.0-1245/spark/python/pyspark/context.py", line 786, in addFile self._jsc.sc().addFile(path) File "/usr/hdp/2.5.0.0-1245/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 813, in __call__ File "/usr/hdp/2.5.0.0-1245/spark/python/pyspark/sql/utils.py", line 45, in deco return f(*a, **kw) File "/usr/hdp/2.5.0.0-1245/spark/python/lib/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value py4j.protocol.Py4JJavaError: An error occurred while calling o35.addFile. : java.io.FileNotFoundException: Added file file:/usr/lib/python2.6/site-packages/tweepy-3.5.0-py2.6.egg does not exist. at org.apache.spark.SparkContext.addFile(SparkContext.scala:1388) at org.apache.spark.SparkContext.addFile(SparkContext.scala:1364) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381) at py4j.Gateway.invoke(Gateway.java:259) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:209) at java.lang.Thread.run(Thread.java:745) ... View more