Hi, I've been working with Hadoop and testing a lot of components of it ecossystem. Now I'm doing a small project that consists in two phases: a) Data Cleansing b) KPIs defintion The step a) I already do in Apache PIG. Now I load the data to Apache Hive. And thus, as in all other projects that I work I only see Apache Hive as data repository. Basically, I just used the Hive to load the data after data cleansing step and the use it as regular data source, nothing more. Since 'm very new in Big Data/Hadoop world, I would like to know what kind of jobs/activities are normal to do using Apache Hive. Sorry for the ignorance :) Thanks! ... View more

Hi experts, I've four .CSV (three dimensions and one Fact Table) in my HDFS. I already do some data cleansing in Apache PIG and I want to put them into Hive. My question is: There is a good idea creates the start schema in Hive or is a better idea to create one big table? I didn't find any good article that explains which is the better way to apply data modeling in Big Data. Many thanks! ... View more

Hi experts, I've the following part of script in Apache Pig: .... A = foreach Source_Data generate (int) ID, ToString( ToDate((long) Time), 'yyyy-MM-dd hh:ss:mm') as date, (int) Code; Store A into '.../newfile'; ... Now I want to create a new Script using Python UDF to guarantee that in my newfile on column Date (#1) I only have String in the format 'yyyy-MM-dd hh:ss:mm'. Is possible to do that? Many thanks! ... View more

Hi experts, I've a dataset with 4 columns and want to know if the column B only have numbers, if the job detect some non numeric value I want to put that value into null. Could I do this in PIG or must be Python embed pig? Many thanks! ... View more

Hi experts, I've the following field: 1388481000000 as the number of milliseconds elapsed from the Unix Epoch (1970-01-01 UTC) How can I convert to Unix TimeStamp? I'm trying to use ToUnixTime(1388481000000,'dd/MM/yyyyHH:mm:ss','GMT') but it gives me error... How can I convert into Unix Timestamp? Many thanks! ... View more