Member since
12-14-2015
8
Posts
4
Kudos Received
0
Solutions
02-03-2016
03:42 PM
I haven't got a working solution yet. It may be worth looking at Apache Phoenix and using that through Knox. I'm no longer working on my PoC due to other commitments. I'll try and get back to finishing it but until then, I think the solution will existing in editing Knox's .class files to change it's behaviour when querying HBase.
... View more
01-04-2016
05:51 PM
1 Kudo
I would like to host an API from within Hadoop to be queried from Knox. I am able to query API's from around the web, but how would I go about hosting my own API and placing it withing Hadoop. As a test, I have built a basic temperature units converter API in Eclipse and I'm hosting it on my Tomcat. I would like to host this in Hadoop to query it from Knox (the Knox query part I can do). From reading documentation I think I will need to host on a Jetty Server, but I have no clue if it's possible and if so, where to host it. Any help would be much appreciated.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Knox
12-18-2015
03:22 PM
1 Kudo
I would like to alter who has access to different Knox Services. For example, I can currently access all the services using guest:guest-password or admin:admin-password but would like to change that so only admin can access certain services. I believe it will mean changing something in the default.xml file in the Knox topologies. I have tried adding this parameter to the default.xml, but I can still access HBase as a guest. <param>
<name>webhbase.acl</name>
<value>admin</value>
</param>
It's not just HBase I would like to change but custom services too so a more general answer would be very much appreciated.
... View more
Labels:
- Labels:
-
Apache Knox
12-17-2015
10:17 AM
HI @Aidan Condron, If you're not bulk loading, you can upload to HBase through Hive. Head to Hive through Ambari. You can upload your .csv files to HDFS, I use the tmp folder. Then use the following in Hive, create table MyTable (col_value STRING);
LOAD DATA INPATH '/tmp/MyData.csv' OVERWRITE INTO TABLE MyTable;
CREATE TABLE MyHiveTable (FirstName STRING, LastName STRING);
insert overwrite table MyHiveTable
SELECT
regexp_extract(col_value, '^(?:([^,]*)\,?){1}', 1) FirstName,
regexp_extract(col_value, '^(?:([^,]*)\,?){2}', 1) LastName
from MyTable;
CREATE TABLE MyHBaseTable(firstname STRING, lastname STRING)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key, f:c1')
TBLPROPERTIES ('hbase.table.name' = 'MyNamesTable');
FROM MyHiveTable INSERT INTO TABLE MyHBaseTable
Select MyHiveTable.*; It's not a fast method, but the Regex and intermediary stages are useful if you need to additional control over your data before it goes into HBase
... View more
12-16-2015
09:38 AM
When I cURL the API either using the APPID in the blog or my own, I get the following,
[root@sandbox ~]# curl 'http://api.openweathermap.org/data/2.5/weather?zip=95054,us&appid=2de143494c0b295cca9337e1e96b00e0'
curl: (6) Couldn't resolve host 'api.openweathermap.org'
[root@sandbox ~]# curl 'http://api.openweathermap.org/data/2.5/weather?zip=95054,us&appid=a457898615afb99ea5e7fc9aa10bbb98'
curl: (6) Couldn't resolve host 'api.openweathermap.org' The same goes for any open web API. However, I am able to ping IP addresses and cURL API's from my command line. Does HDP 2.3.2 have a DNS server?
... View more
12-15-2015
12:12 PM
Thanks. I've used the Developer's Guide and tried to follow your blog. However, I cannot connect to the weather API (or any web hosted API for that matter) within the sandbox. I've got my own APPID. Will your guide/blog work in a Sandbox environment?
... View more
12-14-2015
12:49 PM
2 Kudos
I've created a custom Knox service with the intention of using custom .class files to be able to query multiple HBase tables and/or return multiple HBase table rows The work is a PoC to extent the capabilities of Knox through a custom service. Is this possible? I am still unable to get the new service to even begin calling the custom .class files I've uploaded to replace HBaseDispatch. I am doing this within the HDP 2.3.2 Sandbox on Windows.
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Knox