val ebayds = sc.textFile("/user/spark/xbox.csv") case class Auction(auctionid: String, bid: Float, bidtime: Float, bidder: String, bidderrate: Int, openbid: Float, price: Float) val ebay = ebayds.map(a=>a.split(",")).map(p=>Auction(p(0),p(1).toFloat,p(2).toFloat,p(3),p(4).toInt,p(5).toFloat,p(6).toFloat)).toDF() ebay.select("auctionid").distinct.count am getting error For input string: "" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ... View more

Hi , am joining two relations(withoutschema) in pig and want to pick particular columns from both relations. A = load 'data1' using PigStorage(',');($0,$1...$8) B = load 'data2' using PigStoarge(',');($0..$4); C = foreach(join A by($1,$2),B by($1,$2)) generate $0,$1,$4,$5,(how to select B relation columns) ... View more

am trying to import data from mysql to hive/hdfs but getting error. 1. QueryDatabaseTable ---mysql data ConvertAvroToJson --- output [{"emp_id": 467260, "emp_name": "Rob", "emp_age": 32}, {"emp_id": 467261, "emp_name": "Vijay", "emp_age": 32}, {"emp_id": 467258, "emp_name": "Jayaprakash", "emp_age": 26}, {"emp_id": 467259, "emp_name": "Kalyan", "emp_age": 32}, {"emp_id": 467262, "emp_name": "Andy", "emp_age": 20}, {"emp_id": 467263, "emp_name": "Ashley", "emp_age": 24}, {"emp_id": 467264, "emp_name": "Mounika", "emp_age": 24}] splitjson -- how to split json file into single flow files ... View more