Cloudera Community
pacosoplas
user rank
Master Collaborator
My Accepted Solutions
Show All

Re: hive updates very slow

by Master Collaborator in Support Questions
‎06-14-2016 06:43 AM
‎06-14-2016 06:43 AM
colud you please type the command example to run a compaction for my table adn bucket in ORC format?? CREATE EXTERNAL TABLE IF NOT EXISTS canal( CODTF string, CANAL string, FECHAOPRCNF string, FRECUENCIA int, CODNRBEENF string ) COMMENT 'A buckered table canal by CODNRBEENF' CLUSTERED BY (CODNRBEENF) SORTED BY (FECHAOPRCNF) INTO 60 BUCKETS stored as ORC LOCATION '/RSI/tables/logs/canal' TBLPROPERTIES ("immutable"="false", "orc.create.index"="true","orc.compress"="ZLIB", "orc.stripe.size"="67108864","orc.row.index.stride"="25000"); ... View more

Re: phoenix on hbase

by Master Collaborator in Support Questions
‎06-14-2016 06:42 AM
‎06-14-2016 06:42 AM
conn <- dbConnect(drv, "jdbc:phoenix:lnxbig04.cajarural.gcr,lnxbig05.cajarural.gcr,lnxbig06.cajarural.gcr:2181:/hbase-unsecure") query <- paste("CREATE TABLE phoenix (a_key VARCHAR PRIMARY KEY, a_col VARCHAR)") B <- dbGetQuery(conn, query) Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", : Unable to retrieve JDBC result set for CREATE TABLE phoenix (a_key VARCHAR PRIMARY KEY, a_col VARCHAR) (ERROR 1103 (XCL03): executeQuery may not be used. Query: CREATE TABLE phoenix (a_key VARCHAR PRIMARY KEY, a_col VARCHAR)) ... View more

phoenix on hbase

by Master Collaborator in Support Questions
‎06-14-2016 06:28 AM
‎06-14-2016 06:28 AM
Hi: Iam trying to connect from rstudio to phonenix hbase but i am receiving this error: drv <- JDBC( "org.apache.phoenix.jdbc.PhoenixDriver", "/usr/hdp/current/phoenix-client/phoenix-client.jar" ) for (l in list.files("/usr/hdp/current/phoenix-client/")) { .jaddClassPath(paste("/usr/hdp/current/phoenix-client/", l, sep = "")) } for (l in list.files("/usr/hdp/current/hbase-client/lib/")) { .jaddClassPath(paste("/usr/hdp/current/hbase-client/lib/", l, sep = "")) } for (l in list.files("/usr/hdp/2.3.2.0-2950/hadoop/lib/")) { .jaddClassPath(paste("/usr/hdp/2.3.2.0-2950/hadoop/lib/", l, sep = "")) } conn <- dbConnect(drv, "jdbc:phoenix:lnxbig04.cajarural.gcr,lnxbig05.cajarural.gcr,lnxbig06.cajarural.gcr:2181:/hbase-unsecure") query <- paste("SELECT * FROM canal") B <- dbGetQuery(conn, query) Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", : Unable to retrieve JDBC result set for SELECT * FROM canal (ERROR 1012 (42M03): Table undefined. tableName=CANAL) but the table is create and has data: ... View more
Labels:

Re: hive updates very slow

by Master Collaborator in Support Questions
‎06-12-2016 05:13 PM
‎06-12-2016 05:13 PM
Ho can i control the map and reduce with hive or tex paralell??? ... View more

Re: pig if error

by Master Collaborator in Support Questions
‎06-12-2016 04:40 PM
3 Kudos
‎06-12-2016 04:40 PM
3 Kudos
Hi: I fixed it like this: B = FOREACH A GENERATE (chararray) (($3 =='01' AND $4=='11') OR ($3 =='01' AND $4=='13') OR ($3 =='01' AND $4=='21') ? 'CM' : '' ) ... View more

pig if error

by Master Collaborator in Support Questions
‎06-11-2016 08:27 AM
‎06-11-2016 08:27 AM
hi: can do this: i am receiving this error (chararray) ( (COD_LINEA =='01' and ID_GRP_PD='11') OR (COD_LINEA =='01' and ID_GRP_PD='13') OR (COD_LINEA =='01' and ID_GRP_PD='21') ? 'CM' : ( (COD_LINEA =='01' and ID_GRP_PD='31') OR (COD_LINEA =='01' and ID_GRP_PD='32') OR (COD_LINEA =='01' and ID_GRP_PD='33') ? 'INGs' : ( (COD_LINEA =='01' and ID_GRP_PD='34') OR (COD_LINEA =='01' and ID_GRP_PD='35') OR (COD_LINEA =='01' and ID_GRP_PD='36') ? 'AXT' : ''))) as PROD; 2016-06-11 10:28:40,658 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Lexical error at line 79, column 9. Encountered: "(" (40), after : "" 2016-06-11 10:28:40,658 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.tools.pigscript.parser.TokenMgrError: Lexical error at line 79, column 9. Encountered: "(" (40), after : "" at org.apache.pig.tools.pigscript.parser.PigScriptParserTokenManager.getNextToken(PigScriptParserTokenManager.java:3370) at org.apache.pig.tools.pigscript.parser.PigScriptParser.jj_ntk(PigScriptParser.java:1449) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:110) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) at org.apache.pig.Main.run(Main.java:631) at org.apache.pig.Main.main(Main.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) ... View more
Labels:

hive compactor

by Master Collaborator in Support Questions
‎06-10-2016 10:12 PM
‎06-10-2016 10:12 PM
HI: After compaction i received this error during update command in hive: subtypes: 1 subtypes: 2 subtypes: 3 subtypes: 4 subtypes: 5 subtypes: 6 fieldNames: "operation" fieldNames: "originalTransaction" fieldNames: "bucket" fieldNames: "rowId" fieldNames: "currentTransaction" fieldNames: "row" , kind: INT , kind: LONG , kind: INT , kind: LONG , kind: LONG , kind: STRUCT subtypes: 7 subtypes: 8 subtypes: 9 subtypes: 10 subtypes: 11 subtypes: 12 subtypes: 13 subtypes: 14 subtypes: 15 subtypes: 16 subtypes: 17 subtypes: 18 subtypes: 19 subtypes: 20 subtypes: 21 subtypes: 22 subtypes: 23 subtypes: 24 fieldNames: "_col1" fieldNames: "_col2" fieldNames: "_col3" fieldNames: "_col4" fieldNames: "_col5" fieldNames: "_col6" fieldNames: "_col7" fieldNames: "_col8" fieldNames: "_col9" fieldNames: "_col10" fieldNames: "_col11" fieldNames: "_col12" fieldNames: "_col13" fieldNames: "_col14" fieldNames: "_col15" fieldNames: "_col16" fieldNames: "_col17" fieldNames: "_col18" , kind: INT , kind: STRING , kind: STRING , kind: STRING , kind: STRING , kind: INT , kind: INT , kind: DOUBLE , kind: INT , kind: INT , kind: DOUBLE , kind: DOUBLE , kind: DOUBLE , kind: INT , kind: INT , kind: DOUBLE , kind: DOUBLE , kind: STRING ] schemaTypes [kind: STRUCT subtypes: 1 subtypes: 2 subtypes: 3 subtypes: 4 subtypes: 5 subtypes: 6 fieldNames: "operation" fieldNames: "originalTransaction" fieldNames: "bucket" fieldNames: "rowId" fieldNames: "currentTransaction" fieldNames: "row" , kind: INT , kind: LONG , kind: INT , kind: LONG , kind: LONG , kind: STRUCT subtypes: 7 subtypes: 8 subtypes: 9 subtypes: 10 subtypes: 11 subtypes: 12 subtypes: 13 subtypes: 14 subtypes: 15 subtypes: 16 subtypes: 17 subtypes: 18 subtypes: 19 subtypes: 20 subtypes: 21 subtypes: 22 subtypes: 23 subtypes: 24 fieldNames: "_col1" fieldNames: "_col2" fieldNames: "_col3" fieldNames: "_col4" fieldNames: "_col5" fieldNames: "_col6" fieldNames: "_col7" fieldNames: "_col8" fieldNames: "_col9" fieldNames: "_col10" fieldNames: "_col11" fieldNames: "_col12" fieldNames: "_col13" fieldNames: "_col14" fieldNames: "_col15" fieldNames: "_col16" fieldNames: "_col17" fieldNames: "_col18" , kind: INT , kind: STRING , kind: STRING , kind: STRING , kind: STRING , kind: INT , kind: INT , kind: DOUBLE , kind: INT , kind: INT , kind: DOUBLE , kind: DOUBLE , kind: DOUBLE , kind: INT , kind: INT , kind: DOUBLE , kind: DOUBLE , kind: STRING ] innerStructSubtype -1 at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.<init>(TreeReaderFactory.java:2056) at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory.createTreeReader(TreeReaderFactory.java:2482) at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.<init>(TreeReaderFactory.java:2062) at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory.createTreeReader(TreeReaderFactory.java:2482) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:219) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.<init>(OrcRawRecordMerger.java:179) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.<init>(OrcRawRecordMerger.java:476) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRawReader(OrcInputFormat.java:1406) at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:569) at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:548) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) ... View more
Labels:

Re: hive updates very slow

by Master Collaborator in Support Questions
‎06-10-2016 09:34 PM
‎06-10-2016 09:34 PM
from hue i can see that totalSize 787795564 ... View more

hive updates very slow

by Master Collaborator in Support Questions
‎06-10-2016 06:44 PM
‎06-10-2016 06:44 PM
hi: iam executing this but its very slow and not all memory ram are using, and just 1 map, iam running with tez, any suggestions?? INFO : Map 1: 1/1 Reducer 2: 59(+1)/60 INFO : Map 1: 1/1 Reducer 2: 59(+1)/60 INFO : Map 1: 1/1 Reducer 2: 59(+1)/60 UPDATE desercion_clientes_temp3 SET PROD='CM' WHERE (COD_LINEA ='01' AND ID_GRP_PD='11') OR (COD_LINEA ='01' AND ID_GRP_PD='13') OR (COD_LINEA ='01' AND ID_GRP_PD='21'); UPDATE desercion_clientes_temp3 SET PROD='INGs' WHERE (COD_LINEA ='01' AND ID_GRP_PD='31') OR (COD_LINEA ='01' AND ID_GRP_PD='32') OR (COD_LINEA ='01' AND ID_GRP_PD='33'); UPDATE desercion_clientes_temp3 SET PROD='AXT' WHERE (COD_LINEA ='01' AND ID_GRP_PD='34') OR (COD_LINEA ='01' AND ID_GRP_PD='35') OR (COD_LINEA ='01' AND ID_GRP_PD='36'); UPDATE desercion_clientes_temp3 SET PROD='CRE' WHERE (COD_LINEA ='01' AND ID_GRP_PD='41'); UPDATE desercion_clientes_temp3 SET PROD='TC' WHERE (COD_LINEA ='01' AND ID_GRP_PD='50') OR (COD_LINEA ='01' AND ID_GRP_PD='51'); UPDATE desercion_clientes_temp3 SET PROD='PRE' WHERE (COD_LINEA ='01' AND ID_GRP_PD='71'); UPDATE desercion_clientes_temp3 SET PROD='VIST' WHERE (COD_LINEA ='03' AND ID_GRP_PD='11') OR (COD_LINEA ='03' AND ID_GRP_PD='21'); UPDATE desercion_clientes_temp3 SET PROD='PLZO' WHERE (COD_LINEA ='03' AND ID_GRP_PD='51'); UPDATE desercion_clientes_temp3 SET PROD='PLZOES' WHERE (COD_LINEA ='03' AND ID_GRP_PD='52'); UPDATE desercion_clientes_temp3 SET PROD='SUBORD' WHERE (COD_LINEA ='03' AND ID_GRP_PD='92') OR (COD_LINEA ='03' AND ID_GRP_PD='93'); UPDATE desercion_clientes_temp3 SET PROD='VAL' WHERE (COD_LINEA ='04' AND ID_GRP_PD='10') OR (COD_LINEA ='04' AND ID_GRP_PD='20') OR (COD_LINEA ='04' AND ID_GRP_PD='40'); UPDATE desercion_clientes_temp3 SET PROD='PPEN' WHERE (COD_LINEA ='04' AND ID_GRP_PD='30') OR (COD_LINEA ='04' AND ID_GRP_PD='34'); UPDATE desercion_clientes_temp3 SET PROD='FONI' WHERE (COD_LINEA ='04' AND ID_GRP_PD='56'); UPDATE desercion_clientes_temp3 SET PROD='SEGAH' WHERE (COD_LINEA ='04' AND ID_GRP_PD='61'); UPDATE desercion_clientes_temp3 SET PROD='REC' WHERE (COD_LINEA ='05' AND ID_GRP_PD='01'); UPDATE desercion_clientes_temp3 SET PROD='PDOM' WHERE (COD_LINEA ='05' AND ID_GRP_PD='02'); UPDATE desercion_clientes_temp3 SET PROD='EMRV' WHERE (COD_LINEA ='05' AND ID_GRP_PD='11'); UPDATE desercion_clientes_temp3 SET PROD='GCOBRO' WHERE (COD_LINEA ='05' AND ID_GRP_PD='12'); UPDATE desercion_clientes_temp3 SET PROD='EMRECDOM' WHERE (COD_LINEA ='05' AND ID_GRP_PD='13'); UPDATE desercion_clientes_temp3 SET PROD='RECVEN' WHERE (COD_LINEA ='05' AND ID_GRP_PD='14'); UPDATE desercion_clientes_temp3 SET PROD='CONFIR' WHERE (COD_LINEA ='05' AND ID_GRP_PD='15'); UPDATE desercion_clientes_temp3 SET PROD='DOMINGREC' WHERE (COD_LINEA ='05' AND ID_GRP_PD='33') OR (COD_LINEA ='05' AND ID_GRP_PD='31'); UPDATE desercion_clientes_temp3 SET PROD='EMISTR' WHERE (COD_LINEA ='05' AND ID_GRP_PD='41'); UPDATE desercion_clientes_temp3 SET PROD='EMISTRMAS' WHERE (COD_LINEA ='05' AND ID_GRP_PD='42'); UPDATE desercion_clientes_temp3 SET PROD='PRVENT' WHERE (COD_LINEA ='05' AND ID_GRP_PD='43'); UPDATE desercion_clientes_temp3 SET PROD='PRDOM' WHERE (COD_LINEA ='05' AND ID_GRP_PD='44'); UPDATE desercion_clientes_temp3 SET PROD='ACPAG' WHERE (COD_LINEA ='05' AND ID_GRP_PD='45'); UPDATE desercion_clientes_temp3 SET PROD='TD' WHERE (COD_LINEA ='05' AND ID_GRP_PD='50') OR (COD_LINEA ='05' AND ID_GRP_PD='51'); UPDATE desercion_clientes_temp3 SET PROD='ACCOMER' WHERE (COD_LINEA ='05' AND ID_GRP_PD='52'); UPDATE desercion_clientes_temp3 SET PROD='TPR' WHERE (COD_LINEA ='05' AND ID_GRP_PD='53'); UPDATE desercion_clientes_temp3 SET PROD='GCEXT' WHERE (COD_LINEA ='05' AND ID_GRP_PD='70') OR (COD_LINEA ='05' AND ID_GRP_PD='71') OR (COD_LINEA ='05' AND ID_GRP_PD='72'); UPDATE desercion_clientes_temp3 SET PROD='SEGVIDA' WHERE (COD_LINEA ='05' AND ID_GRP_PD='81'); UPDATE desercion_clientes_temp3 SET PROD='SEGRIESGO' WHERE (COD_LINEA ='05' AND ID_GRP_PD='82') OR (COD_LINEA ='05' AND ID_GRP_PD='93'); ... View more
Labels:

Re: hive overwrtite table

by Master Collaborator in Support Questions
‎06-10-2016 05:54 PM
1 Kudo
‎06-10-2016 05:54 PM
1 Kudo
Hi: after create the like NO INTERNAL and execute: set hive.enforce.bucketing = true; set map.reduce.tasks = 60; SET hive.exec.parallel=true; SET hive.vectorized.execution.enabled=true; TRUNCATE TABLE desercion_clientes_temp3; INSERT INTO TABLE desercion_clientes_temp3 select * FROM desercion_clientes_temp2; it work!! Many thanks ... View more
Contact Me
Online
Offline
Last Visited
‎11-16-2019 11:43 AM
Community Statistics
Member Since ‎09-24-2015 09:57 AM
Last Visited ‎11-16-2019 11:43 AM
Posts 527
Kudos received 136