Member since
01-07-2020
64
Posts
1
Kudos Received
0
Solutions
04-01-2022
08:14 AM
1 Kudo
I am trying to run an insert statement in hive but it returns me this error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask The same insert runs perfect through impala. What exactly is this error ?
... View more
Labels:
- Labels:
-
Apache Hive
03-10-2022
04:15 AM
Hi,
Does anyone know when CDP Certified Data Developer will be released?
Thanks in advance.
... View more
Labels:
02-21-2022
12:22 AM
I am trying to run a script in oozie and every time I receive the below error regarding impala.dbapi. The module is inserted correctly in the script. Stdoutput Traceback (most recent call last): Stdoutput File "/tmp/sorting_table.py", line 8, in <module> Stdoutput from impala.dbapi import connect Stdoutput ImportError: No module named impala.dbapi Exit code of the Shell command 1 <<< Invocation of Shell command completed <<< java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410) at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55) at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141) Caused by: org.apache.oozie.action.hadoop.LauncherMainException at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:76) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104) at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:63) Script import libraries: from pyspark import SparkContext from pyspark.sql import SparkSession from datetime import datetime,timedelta import ssl from impala.dbapi import connect import thrift_sasl import os
... View more
Labels:
- Labels:
-
Apache Impala
02-15-2022
12:26 AM
Hi, After some actions in a cluster, oozie in hue hasn't got shell action in the ACTIONS tab. It has only HIVE SCRIPT ,HIVESERVER2 SCRIPT, SUB WORKFLOW, SSH, FS, EMAIL, STREAMING, GENERIC, KILL. Is there any way to run my shell script now? My oozie version is 5.1.0-cdh6.3.4
... View more
Labels:
- Labels:
-
Apache Oozie
01-20-2022
06:14 AM
Hi, I want to ask is there any way to make oozie run a WF only if the previous execution has finished?
... View more
Labels:
- Labels:
-
Apache Oozie
12-13-2021
11:21 PM
I have an ETL flow which transfers data from a hive table to another through pyspark. The tables are partitioned. Although I see that in the partition's path in HDFS there are small parquet files. I want to ask: 1)How can I merge these files? 2)Is there any max size or recommended size for hive partitions?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
12-10-2021
02:55 AM
Hi, I want to create a hive table which will store data with orc format and snappy compression. Will power bi be able to read from that table? Also do you suggest any other format/compression for my table?
... View more
Labels:
- Labels:
-
Apache Hive
12-02-2021
01:21 AM
Hi, I have a sqoop jopb in order to transafer data from MySQL to hive with incremental import. Until now it is working fine but I want to ask how can I tell the job to rerun if it will fail? Is that possible? I saw this article: https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#validation but i am not sure. Can someon give an example?
... View more
Labels:
- Labels:
-
Apache Sqoop
11-11-2021
02:58 AM
Hi, After restarting impala I ran some queries and two of them threw an error for to_timestamp function that I have in my view. If I rerun these queries everything runs smoothly. Why is this happening? It is like impala can not read the functions in the first-run of the queries. Thanks in advance
... View more
Labels:
- Labels:
-
Apache Impala
11-10-2021
04:26 AM
Hi @ChethanYM, 1)Unfortunately I haven't keep the full log from the query. 2)Exactly, this is my issue. 3)From impala shell. 3)If QUERY_TIMEOUT_S it is then it has the default value. Regards , Teo
... View more