Hello, Thank you for your answer. The problem is that clasic Pig scripts (no access to Hive tables, nor to HBase) are running in a distributed way (they have mappers and reducers). However, this one is running only on one node (in Cloudera Manager ->Hosts all namenodes have a Load Average of 0.* and one node has 9.* as load charge) Since you say that normally, even if only mappers are created the script should run in a distributed node, I will post an anonymised version of my script. SET mapreduce.fileoutputcommitter.marksuccessfuljobs false;
SET output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;
SET hbase.zookeeper.quorum '${ZOOKEEPER_QUORUM}';
SET oozie.use.system.libpath true
SET oozie.libpath '${PATH_LIB_OOZIE}'
------------------------------------------------------------
-- hcat
register 'hive-hcatalog-core-0.13.1-cdh5.3.0.jar';
register 'hive-hcatalog-core.jar';
register 'hive-hcatalog-pig-adapter-0.13.1-cdh5.3.0.jar';
register 'hive-hcatalog-pig-adapter.jar';
register 'hive-metastore-0.13.1-cdh5.3.0.jar';
register 'datanucleus-core-3.2.10.jar';
register 'datanucleus-api-jdo-3.2.6.jar';
register 'datanucleus-rdbms-3.2.9.jar';
register 'commons-dbcp-1.4.jar';
register 'commons-pool-1.5.4.jar';
register 'jdo-api-3.0.1.jar';
-- UDF
REGISTER 'MyStoreUDF-0.3.8.jar';
------------------------------------------------------------------------------------------------------------
----------------------------------------------- input data -------------------------------------------------
var_a= LOAD 'my_database.my_table' USING org.apache.hcatalog.pig.HCatLoader() as
(
a:chararray ,
b:chararray,
c:chararray,
d:chararray,
e:chararray,
f:long,
g:chararray,
h:chararray,
i:long,
j:chararray,
k:bag{((name:chararray,value:chararray))},
l:chararray,
m:chararray );
var_a_filtered= FILTER sessions BY (a== 'abcd' );
var_a_proj= FOREACH var_a_filteredGENERATE
a,
b,
c,
d;
STORE var_a_proj INTO 'hbaseTableName'
USING MyStoreUDF('-hbaseTableName1 hbaseTableName1 -hbaseTableName2 -hbaseTableName2 '); Thank you! Alina GHERMAN
... View more
