Member since
02-20-2020
2
Posts
0
Kudos Received
0
Solutions
03-11-2020
03:08 AM
Hi All, We are running our pyspark jobs in python 3 conda environment and facing below WARNING logs and leads to more log size ~ 20 MB and difficult to trace the actual ERRORS. Is cloudera ready to support python 3 version (as no more support for python 2) on scripts like /etc/hadoop/conf/topology_script.py etc. May I know solution for python 3 jobs to run in HDP 3.1.5. 20/03/10 18:26:01 WARN ScriptBasedMapping: Exception running /etc/hadoop/conf/topology_script.py 10.101.193.7 ExitCodeException exitCode=1: File "/etc/hadoop/conf/topology_script.py", line 63 print rack ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(rack)? at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008) at org.apache.hadoop.util.Shell.run(Shell.java:901) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResolveCommand(ScriptBasedMapping.java:251) at
... View more
02-20-2020
06:57 AM
I am also getting same issue when running a join between two tables (loaded using hive warehouse connector into data frames and then registered as spark temp tables ) in spark SQL. Could you please let me know if any solution
... View more