@vamsi valiveti bq. 1)what is the meaning of COUNTER VALUE = 23 i can see cf1:no3 is incremented to 3 but i did not what is meant by COUNTER VALUE = 23? COUNTER VALUE printed is latest value after incrementing the value to specified number. In your case the column value before incrementing might be 20. => Hbase::Table - test hbase(main):013:0> incr 'test', 'spam', 'foo:bar', 1 COUNTER VALUE = 1 0 row(s) in 0.0080 seconds hbase(main):014:0> incr 'test', 'spam', 'foo:bar', 1 COUNTER VALUE = 2 0 row(s) in 0.0060 seconds hbase(main):015:0> incr 'test', 'spam', 'foo:bar', 1 COUNTER VALUE = 3 0 row(s) in 0.0040 seconds bq. 2)How to identify no of rows[3 row(s)]?Is it no of distinct keys present in HBASE table? Yes. It's distinct row keys in the table. bq. 3)The below command is showing following error.can not we use incr on existing column in HBASE? Yes we cannot use incr on existing column. The column value should be initialized with incr only. ... View more

The class is from hbase-server.jar. You can add /usr/hdp/2.4.0.0-169/phoenix/lib/hbase-server.jar to classpath and try. ... View more

CREATE TABLE ddl statement automatically creates HBase table. http://phoenix.apache.org/language/index.html#create_table Similary DROP and ALTER TABLE statements drop, alter HBase tables. Here is the full list of queries supported by Phoenix. http://phoenix.apache.org/language/index.html ... View more

@mayki wogno There is a chance that the master might be went down while creating the znode or system catalog table data might be deleted from hdfs You can connect to zookeeper and remove the znode from zookeeper and create connection. The znode path is {zookeeper.znode.parent}/table/SYSTEM.CATALOG. Connect to zookeeper with this command ./hbase zkcli Remove znode from zk client: rmr /hbase-unsecure/table/SYSTEM.CATLOG ... View more

@SBandaru, For HBase tuning you can refer following links: http://www.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok https://community.hortonworks.com/questions/1277/what-is-the-best-consolidated-guide-for-hbase-tuni.html For Phoenix tuning following link helps: http://phoenix.apache.org/tuning.html Use http://phoenix.apache.org/update_statistics.html for more parallelization and better performance. For more phoenix level optimizations You can refer Optimization sections in http://www.slideshare.net/je2451/apache-phoenix-and-apache-hbase-an-enterprise-grade-data-warehouse ... View more

@Ramy Mansour You can directly create table in phoenix and load data using CsvBulkLoadTool. http://phoenix.apache.org/bulk_dataload.html#Loading_via_MapReduce Currently with your data there will be 1000's of mappers running. The number of reducers depending on the number regions so increase the parallelization you can presplit the table by providing split points in DDL statement. You can also compress the table to reduce IO or shuffle data during bulkload tool. http://phoenix.apache.org/language/index.html#create_table Or else you can directly use ImportTsv and completeBulkload bulkload tools for loading data into HBase table directly. https://hbase.apache.org/book.html#importtsv https://hbase.apache.org/book.html#completebulkload Here some more configurations can be provided to mapred-site.xml to improve the job performance. <property> <name>mapreduce.map.output.compress</name> <value>true</value> </property> <property> <name>mapred.map.output.compress.codec</name> <value>org.apache.hadoop.io.compress.SnappyCodec</value> </property> ... View more

You can get more info here: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration http://hortonworks.com/blog/hbase-via-hive-part-1/ ... View more

Here you get the more details: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_Sys_Admin_Guides/content/ch_clust_capacity.html ... View more

@Dheeraj Madan After altering the table you can run major compaction on the table from hbase shell so that existing data will be automatically compressed. > major_compact 'SNAPPY_TABLE' ... View more

Got the logs from @nshetty and checked somehow the hbase:acl table is not created because earlier hbase:acl znode is present but the table is not present. Once after deleting the hbase:acl table znode and restarting the service it's working fine. ... View more

It seems that PDI is not supporting Apache Phoenix yet. While loading data PDI generating insert queries but Phoenix needs upsert queries. http://jira.pentaho.com/browse/PDI-14038 https://mail-archives.apache.org/mod_mbox/phoenix-user/201509.mbox/%3CCAB3fahz1wacofQHDTyMMO-W_nCQ0gAcL0LpXdeMmZMxVXwwWOg@mail.gmail.com%3E It would be better to contact Pentaho PDI community. ... View more

HDP 2.4 we are using phoenix version 4.4.0 which doesn't have transactions feature so there is no support for tephra in it. ... View more

Hi @Sasikumar Natarajan You can include timestamp also part of primary key so you will have 1440 rows and also when you search by giving device id and date you can get all the 1440 records as usual. It will be fast also because it's going to be range query. create table devicedata (deviceId integer not null, day date not null, ts timestamp, val double CONSTRAINT my_pk PRIMARY KEY (deviceId, day, timestamp)) Since it's time series data your regions might be bottle neck because continuous writes might go to single region. Then you you can use salt buckets to avoid it. create table devicedata (deviceId integer not null, day date not null, ts timestamp, val double CONSTRAINT my_pk PRIMARY KEY (deviceId, day, timestamp)) SALT_BUCKETS=N. ... View more

Here you can find more detailed information: The FileNotFoundExceptions is coming from the split daughters not being able to find the files of the parent region. The parent regions' files might already been deleted at this point. HBCK has a flag to fix this, but if it is a handful of regions/files affected, I usually prefer to manually move the reference files out of the hbase root directory. For reference, here is the high level flow: Go to region servers log, and find the file name for FileNotFoundException, copy the file name Check hdfs to see whether the file is really not there. Figure out whether this is an actual hfile or a reference file. HFiles are named like<region_name>/<column_family>/<UUID> while reference files are named like<region_name>/<column_family>/<UUID>.<parent_region_name>. If the missing file does not belong to the region which is throwing the exception, then it is due to the reference file referring to the missing file. So we should find and move the reference file (which should be very small) out of the daughter regions directory. Notice that the reference file name should contain the actual UUID of the referred file and the parent regions name. ... View more

Hi @Michael Dennis "MD" Uanang This should help you https://community.hortonworks.com/questions/2349/tip-when-you-get-a-message-in-job-log-user-dr-who.html ... View more

There is time difference of more than half minute between master and region servers that's why getting clock is out of sync exception. You can set same time for master and regionservers machines or install ntp to maintain time sync among all the machines. ... View more

1) Yes Bulk CSV data loading supports local indexes as well. You can happily go ahead with it. 2) Asynchronous index population supports local indexes but it would be better go by normal create index for local indexes because it's faster than async index population. 3) Only thing we can do is every time after loading data from HBase we need to rebuild the index but it will be costly. ... View more