Member since
05-30-2018
1322
Posts
715
Kudos Received
148
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4025 | 08-20-2018 08:26 PM | |
| 1930 | 08-15-2018 01:59 PM | |
| 2361 | 08-13-2018 02:20 PM | |
| 4077 | 07-23-2018 04:37 PM | |
| 4993 | 07-19-2018 12:52 PM |
07-09-2016
04:26 AM
Duplicate question answered here https://community.hortonworks.com/questions/44208/hdfs-heterogeneous-storage-using-aws-s3-as-storage.html
... View more
07-08-2016
05:39 PM
2 Kudos
@Faisal Hussain I do not think there is one but definitely there is plan to develop one. https://cwiki.apache.org/confluence/display/NIFI/First-class+Avro+Support You can convert Avro to JSON (ConvertAvroToJSON) and then call a script (lots of example of scripts in the internet for converting JSON to csv using java/javascript/perl/bash/awk etc and etc) in the ExecuteStreamCommand processor. This would work. Let us know if you need more details.
... View more
07-08-2016
07:58 AM
It does mean, you have to use the tool the generate an app, and fill the result on the form what you attached.
... View more
07-06-2016
10:28 PM
5 Kudos
This tutorial will show how to export data out of hbase table into csv format. We will use airport data from american statical association available here. Assume you have a sandbox up and running lets start. First ssh into your sandbox and switch user to hdfs sudo su - hdfs Then grab the airport data by issues a wget wget http://stat-computing.org/dataexpo/2009/airports.csv For my example the file is located /home/hdfs/airports.csv Now lets create a hbase table called "airports" with column family "info". Do this in hbase shell Now that the table is created lets load it. Get out of hbase shell. as user hdfs run the following to load the table hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=, -Dimporttsv.columns="HBASE_ROW_KEY,info:iata,info:airport,info:city,info:country,info:lat,info:long" airports hdfs://sandbox.hortonworks.com:/tmp/airports.csv That will kick off map reduce job to load airport table in hbase. once that is done you can do a quick verify in hbase shell by running counts 'airports' You should see 3368 records in the table. Now lets log into pig shell. We will create a variable called airport_data which we will load our hbase table into by issuing: airport_data = LOAD 'hbase://airports'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
'info:iata,info:airport,info:city,info:country,info:lat,info:long', '-loadKey true')
AS (iata,airport,city,country,lat,long); Now that we have our data in a variable lets dump it to hdfs using csv format by issuing: store airport_data into 'airportData/export' using PigStorage(','); So we have dumped the export into hdfs directory airportData/export. Lets go view it And there you go. We have loaded data into hbase table. Exported data from the table using pig in csv format. Happy pigging.
... View more
Labels:
07-08-2016
01:28 AM
@slachterman Good catch. fixed.
... View more
03-29-2017
07:05 AM
I meet this question after upgrading from 2.3 to 2.5, then now it's solved by this workarround.
... View more
07-06-2016
05:06 PM
@Ali Bajwa yes /etc/hosts works. also I enabled bridge network to get a ip.
... View more
10-16-2017
09:18 PM
Hi, how did it work for you?
... View more
10-11-2016
08:25 PM
@Jasper please try here https://s3-us-west-1.amazonaws.com/hdb2-on-hdp/Hortonworks_May_2016.vmwarevm.7z
... View more