Member since
09-24-2015
527
Posts
136
Kudos Received
19
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2890 | 06-30-2017 03:15 PM | |
| 4377 | 10-14-2016 10:08 AM | |
| 9598 | 09-07-2016 06:04 AM | |
| 11634 | 08-26-2016 11:27 AM | |
| 1912 | 08-23-2016 02:09 PM |
06-01-2016
12:14 PM
hi: you are right i have received this error: Error: Error while compiling statement: FAILED: SemanticException [Error 10298]: ACID insert, update, delete not supported on tables that are sorted, table journey_oficina (state=42000,code=10298)
so, I have removed the "transactional"="true", and the statement was ok, but now i cant make ACID operaitons, also it have improve the query, but now HOW CAN I UPDATE DE TABLE??
... View more
06-01-2016
10:59 AM
hi: Finally i created it and iam doing ETL of 3 month of data, and i will see if the query have been improved CREATE EXTERNAL TABLE IF NOT EXISTS journey_oficina(
FECHAOPRCNF timestamp,
codnrbeenf string,
CODINTERNO string,
CODTXF string,
FREQ BIGINT
)
CLUSTERED BY (codnrbeenf) INTO 60 BUCKETS
SORTED BY (FECHAOPRCNF)
stored as ORC
LOCATION '/RSI/tables/logs/journey_oficina'
TBLPROPERTIES ("immutable"="false","transactional"="true", "orc.create.index"="true","orc.compress"="ZLIB", "orc.stripe.size"="67108864","orc.row.index.stride"="25000");
its ok???
... View more
06-01-2016
10:19 AM
Hi: if i remove the transactional i cant modifi the table, so like this it will not improve nothing? CREATE EXTERNAL TABLE IF NOT EXISTS journey_oficina_v5(
FECHAOPRCNF timestamp,
codnrbeenf string,
CODINTERNO string,
CODTXF string,
FREQ BIGINT
)
CLUSTERED BY (codnrbeenf)
SORTED BY (FECHAOPRCNF)
INTO 25 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
stored as ORC
LOCATION '/RSI/tables/logs/journey_oficina_v5'
TBLPROPERTIES ("immutable"="false","transactional"="true", "orc.create.index"="true","orc.compress"="ZLIB", "orc.stripe.size"="268435456", "orc.row.index.stride"="10000");
... View more
06-01-2016
09:51 AM
Hi: waht about partitions, or index??? and waht about the grouo by??
... View more
06-01-2016
09:46 AM
Hi: the table i have created is like this, I use bucket because i want to use ACID operations, but there is any other good solution to use ACID operations and improve the query?? CREATE EXTERNAL TABLE IF NOT EXISTS journey_oficina_v4(
FECHAOPRCNF timestamp,
codnrbeenf string,
CODINTERNO string,
CODTXF string,
FREQ BIGINT
)
CLUSTERED BY (codnrbeenf) INTO 25 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
stored as ORC
LOCATION '/RSI/tables/logs/journey_oficina_v4'
TBLPROPERTIES ("immutable"="false","transactional"="true")
... View more
06-01-2016
09:34 AM
hi: actually i have this query very slow, any suggestion to improve that? select codinterno,sum(freq) as freq,codtxf
from journey_oficina_v4
where codnrbeenf = '3008'
and (fechaoprcnf >= '2016-04-01 00:00:00.0'
and fechaoprcnf < '2016-04-02 00:00:00.0' )
and codtxf = 'STR01OON'
group by codinterno, codtxf
order by freq desc
i just have a bucket for codnrbeenf and is not in the query so.... Thanks
... View more
Labels:
- Labels:
-
Apache Hive
06-01-2016
09:07 AM
Hi: finally it work deleting some jars that I put in the hive classpath like apachedrill :(, so i deleted it and i restarted the service. Many thanks all of you.
... View more
05-31-2016
05:02 PM
Hi: i dont have lsof installed, so tomorrow ill tell me OS team to install that and ill try it. is curious that the hadoop classpath is this, its correct?? i dont see hive hadoop classpath
/usr/hdp/2.3.2.0-2950/hadoop/conf:/usr/hdp/2.3.2.0-2950/hadoop/lib/*:/usr/hdp/2.3.2.0-2950/hadoop/.//*:/usr/hdp/2.3.2.0-2950/hadoop-hdfs/./:/usr/hdp/2.3.2.0-2950/hadoop-hdfs/lib/*:/usr/hdp/2.3.2.0-2950/hadoop-hdfs/.//*:/usr/hdp/2.3.2.0-2950/hadoop-yarn/lib/*:/usr/hdp/2.3.2.0-2950/hadoop-yarn/.//*:/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/lib/*:/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/.//*:::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/ojdbc.jar:/usr/hdp/2.3.2.0-2950/tez/*:/usr/hdp/2.3.2.0-2950/tez/lib/*:/usr/hdp/2.3.2.0-2950/tez/conf
... View more
05-31-2016
04:54 PM
hi: I was deleting external and internal tables from job-browser, from this path into hdfs: /apps/hive/warehouse
and the external path
... View more
05-31-2016
04:34 PM
[root@lnxbig05 bin]# grep -iR org.apache.hadoop.hive.shims.HadoopShims /usr/hdp/current/hive-server2/
Binary file /usr/hdp/current/hive-server2/lib/hive-jdbc-1.2.1000.2.4.0.0-169-standalone.jar matches
Binary file /usr/hdp/current/hive-server2/lib/hive-exec-1.2.1000.2.4.0.0-169.jar matches
Binary file /usr/hdp/current/hive-server2/lib/drill-hive-exec-shaded-1.6.0.jar matches
Binary file /usr/hdp/current/hive-server2/lib/hive-exec.jar matches
Binary file /usr/hdp/current/hive-server2/lib/hive-shims-common-1.2.1000.2.4.0.0-169.jar matches
grep: /usr/hdp/current/hive-server2/lib/ojdbc6.jar: No such file or directory
Binary file /usr/hdp/current/hive-server2/lib/hive-shims-common.jar matches
Binary file /usr/hdp/current/hive-server2/lib/hive-jdbc.jar matches
... View more