hi experts! in HDFS serivices there is tool called balancer, which purposed to ensure even distribution of blocks across cluster. my question is how frequently it kicks in to check is cluster imbalanced or not? is there any way to change this frequency?   thanks! ... View more

hi dear experts!   i do have a challenge.  i do have a dynamuc service pool, let's say root.marketing. Many users, who belong to this pool is submitting jobs on it (Bob, Alice, Tom). i want to know resource consumption for each of the users. like for the last day Bob used in average 33 cores, Alice 12, Tom 118... or something like this. in other words want to know who consume what within the same pool   thanks! ... View more

Hi dear experts!   i'm trying to load data with ImportTSV tool , like this: hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dmapreduce.job.reduces=1000 -Dimporttsv.columns="data:SS_SOLD_DATE_SK, HBASE_ROW_KEY" -Dimporttsv.separator="|" -Dimporttsv.bulk.output=/tmp/store_sales_hbase store_sales /user/root/benchmarks/bigbench/data/store_sales/*, but have only one reducer (despite on -Dmapreduce.job.reduces=1000 setting ). i even set  mapreduce.job.reduces=1000 on the Cluster wide, but still have only one reducer.   Could anybody hint how to resolve this? thank you in advance for any input! ... View more

Hi dear experts! i'm trying to load data from CSV format on HDFS to HBase with ImportTSV (importtsv). it works perfectly fine in case when HBASE_ROW_KEY is the single CSV column. but i don't know how to create composite HBASE_ROW_KEY (from two columns). for example, i have CSV with 3 columns:   row1, 1, abc row1, 2, dd row2, 1, iop row3, 1, kk and row could be uniqly identified by first two columns.   any inputs will be highly appreciated! ... View more

Hi dear expert! i'm trying to export data with sqoop.export.records.per.statement parameter. But for some reasons sqoop doesn't recognize it:   sqoop export --direct --connect jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds --table orcl_dpi --export-dir /tmp/dpi --input-fields-terminated-by ',' --lines-terminated-by '\n' -m 70 --batch -Dsqoop.export.records.per.statement=10000 -Dsqoop.export.statements.per.transaction=100 Warning: /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/01/22 19:59:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.1 16/01/22 19:59:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Error parsing arguments for export: 16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.records.per.statement=10000 16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.statements.per.transaction=100  i've tried to remove --direct key (target DB is Oracle), but it also doesn't help: sqoop export --connect jdbc:oracle:thin:@host:1521:orcl --username user --password pass --table orcl_dpi --export-dir /tmp/dpi --input-fields-terminated-by ',' --lines-terminated-by '\n' -m 70 --batch -Dsqoop.export.records.per.statement=10000 -Dsqoop.export.statements.per.transaction=100 Warning: /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/01/22 20:00:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.1 16/01/22 20:00:29 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Error parsing arguments for export: 16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.records.per.statement=10000 16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.statements.per.transaction=100 thank you! ... View more