I've the following code: Source = LOAD '.../MyTextFiles' using PigStorage(' '); Data = FOREACH Source GENERATE (chararray)$1 AS Detail_ID, (chararray)$2 AS Code_Time, (chararray)$3 AS BreakLine; Transform = FOREACH Data GENERATE $1, ToUnixTime($2,'dd/MM/yyyyHH:mm:ss','GMT'), $3; SPLIT Transform INTO Src31 IF ToDate($2,'yyyy-MM-dd')==ToDate('2013-12-31', 'yyyy-MM-dd'), Src01 IF ToDate($2,'yyyy-MM-dd')==ToDate('2014-01-01', 'yyyy-MM-dd'); STORE Src31 INTO '.../31_06_2016' using PigStorage(' '); STORE Src01 INTO '.../01_07_2016' using PigStorage(' '); If run the code without the STORE Statement it fives me successfully but If I try to introduce the new data into a new tables it gives me error... anyone knows why? Many thanks! ... View more

Hi experts, I've multiple files distributed by different directories (according to date) into my HDFS. All this files follows the same schema and only the first column (which represents the date) allows us to differentiate each text file. I wanna to merge all this directories into one table in Hive. I don't know which column I can put as partitioned column. My text files have this schema: - Date - ID - Investment - Country - City The column date of each file column is what allows us to differentiate each file. Each directory aggregate multiple files from one date and have the day as name. Which is the PARTITIONED column in this case? ... View more

Hi experts, I'm a beginner using Hadoop and was reading a book that talks about Storage Format. My source data are some text files and my question is: I need/can transform my files into Sequence File, Avro, Parquet or Optimized Row Columnar? There I take some advantage using it instead of text files? Many thanks!! ... View more

Hello, I want to load some .csv files to HDFS. I already decide that I want to do, in next step, some data transformation with Spark. My question is: I've some advantage to use PIG instead Spark for load data into HDFS? Many thanks! ... View more