Member since
04-22-2016
931
Posts
46
Kudos Received
26
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1860 | 10-11-2018 01:38 AM | |
| 2220 | 09-26-2018 02:24 AM | |
| 2252 | 06-29-2018 02:35 PM | |
| 2932 | 06-29-2018 02:34 PM | |
| 6105 | 06-20-2018 04:30 PM |
04-25-2017
08:11 PM
I don't see anything on Hortonworks page if this exam is open book ? are we allowed internet access during exam or not? thanks
... View more
Labels:
- Labels:
-
Apache Hadoop
04-23-2017
01:41 PM
I asked this simple question before but got no response , trying again . if I set these values for the scheduler in YARN config , it breaks YARN , what am I doing wrong here? yarn.scheduler.capacity.maximum-am-resource-percent=0.2
yarn.scheduler.capacity.maximum-applications=10000
yarn.scheduler.capacity.node-locality-delay=40
yarn.scheduler.capacity.root.accessible-node-labels=*
yarn.scheduler.capacity.root.acl_administer_queue=*
yarn.scheduler.capacity.root.capacity=100
yarn.scheduler.capacity.root.queues=Engineering,Marketing,Support
yarn.scheduler.capacity.root.Engineering.acl_administer_jobs=*
yarn.scheduler.capacity.root.Engineering.acl_submit_applications=*
yarn.scheduler.capacity.root.Engineering.capacity=100
yarn.scheduler.capacity.root.Engineering.maximum-capacity=100
yarn.scheduler.capacity.root.Engineering.state=RUNNING
yarn.scheduler.capacity.root.Engineering.user-limit-factor=1
yarn.scheduler.capacity.Engineering.minimum-user-limit-percent=100
yarn.scheduler.capacity.root.Marketing.acl_administer_jobs=*
yarn.scheduler.capacity.root.Marketing.acl_submit_applications=*
yarn.scheduler.capacity.root.Marketing.capacity=100
yarn.scheduler.capacity.root.Marketing.maximum-capacity=100
yarn.scheduler.capacity.root.Marketing.state=RUNNING
yarn.scheduler.capacity.root.Marketing.user-limit-factor=1
yarn.scheduler.capacity.Marketing.minimum-user-limit-percent=100
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
Apache YARN
04-20-2017
04:09 AM
on Hortonworks website there is a PDF which lists the exam tasks which are far more than whats listed on the amazon AWS practice exam site ? there is a an "Exam" folder there under which there is an html listing the tasks , and they are only 6 tasks where as the PDF document on the Horton works site lists at least dozen more tasks. which one is true? tasks.jpg
... View more
Labels:
- Labels:
-
Apache Hadoop
03-21-2017
06:15 PM
errorlog.txtI have created a hbase table and I am populating it as follows but its throwing exception (see attached file) fdaf hbase(main):002:0> create 'besthbase' ,'f1','f2','f3','f4','f5','f6','f7','f8'
0 row(s) in 1.2670 seconds
[hbase@hadoop1 ~]$ hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator="," -Dimporttsv.columns="HBASE_ROW_KEY,id,f1:c1,f2:c2,f3:c3,f4:c4,f5:c5,f6:c6,f7:c7,f8:c8" besthbase hdfs:/user/hbase/BESTPASS_PLATE.csv
fasd 2017-03-21 14:06:45,345 INFO [main] mapreduce.Job: Task Id : attempt_1490118831477_0002_m_000000_0, Status : FAILED
Error: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1940 actions: org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family id does not exist in region besthbase,,1490117043490.f816b4ccfbd445cbb56f827df5213d1d. in table 'besthbase', {NAME => 'f1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f3', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f4', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f5', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f6', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f7', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f8', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:722)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:677)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
... View more
Labels:
03-21-2017
02:32 PM
I solved this problem using the following method , but I do want to know why would one want to use the SERDE method and not this one? [hbase@hadoop1 ~]$ more a.csv
5842,50,30,4,240,340
5843,52,32,5,250,360
5844,56,31,2,248,333
[hbase@hadoop1 ~]$
create table test3(Id int, lowT string, highT string,vib int, lowP string,highP string) ROW FORMAT DELIMITED FIELDS TERMINATED BY "," STORED AS TEXTFILE
TBLPROPERTIES("skip.header.line.count"="1");
load data inpath '/user/hbase/a.csv' OVERWRITE INTO TABLE test3;
Loading data to table default.test3
Table default.test3 stats: [numFiles=1, numRows=0, totalSize=63, rawDataSize=0]
OK
Time taken: 0.668 seconds
... View more
03-20-2017
09:22 PM
ok I created the table in hive n hbase as follows , how do I run those "A = " and "STORE A INTO" commands. . as hive ? hbase? pig?
create table test2(id int, lowT string, highT string, vib int, lowP string,highP string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES('hbase.columns.mapping'=':key,temperature:lowT,temperature:highT,vibration:vib, pressure:lowP, pressure:highP')
TBLPROPERTIES('hbase.table.name'='test1') ;
... View more
03-20-2017
07:07 PM
hi pbarna can you try your create table command and see if it works for you? its not working for me. thanks FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 6 elements while hbase.columns.mapping has 7 elements (counting the key if implicit))
[hbase@hadoop1 ~]$
... View more
03-20-2017
06:58 PM
but I am stuck even before this step. .. please see the error below
... View more
03-20-2017
06:17 PM
also if I try your command I get this error : [hbase@hadoop1 ~]$ hcat -f a.ddl
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 6 elements while hbase.columns.mapping has 7 elements (counting the key if implicit))
... View more
03-20-2017
06:12 PM
I want to make columns as "TEMP" "VIBRATION" "PRESSSURE" , The TEMP and PRESSURE have two columns "high" and "low" , what will be the command syntax then ?
... View more