Hi All, I am also facing the same issue. I am logged into Ambari with admin user and trying to create /move files from /to under /user/root/satish/ . Here are the details on the folder permissions. [root@sandbox sat]# hadoop fs -ls /user/root/satish/Found 3 itemsdrwxr-xr-x - root hdfs0 2017-06-14 00:54 /user/root/satish/inputdrwxr-xr-x - root hdfs0 2017-06-14 00:55 /user/root/satish/outputdrwxr-xr-x - root hdfs0 2017-06-14 00:55 /user/root/satish/scripts Even I tried with different path too, i am getting the same error. Please let me know if I am missing anything here. Error: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask ... View more

@Gobi Subramani Here are the details on when to use MR vs Spark. Other useful links: http://www.infoworld.com/article/3014440/big-data/five-things-you-need-to-know-about-hadoop-v-apache-spark.html https://www.xplenty.com/blog/2014/11/apache-spark-vs-hadoop-mapreduce/ If this information is useful, please accept this answer. ... View more

@Girish Chaudhari Note, when you change the location of the file by using alter command, the old data file is not moved to new location. On your issue, 1) do you have any data files in the mentioned path? 2) Did you get any warnings / errors while you executed this ALTER command? ... View more

@pratik vagyani Best and easy way is through Ambari File view upload option. ... View more

@John ClevelandThis is what i did, changed the script to dump all data , truck_events = LOAD '/user/satu/test.csv' USING PigStorage(',') AS (driverId:int, truckId:int, eventTime:chararray, eventType:chararray, longitude:double, latitude:double, eventKey:chararray, correlationId:long, driverName:chararray, routeId:long,routeName:chararray,eventDate:chararray); --DESCRIBE truck_events; DUMP truck_events; --truck_events_subset = LIMIT truck_events 100; --DESCRIBE truck_events_subset; --DUMP truck_events_subset; Job completed in 62 seconds. ... View more

@ozac Can you try without renaming the directory from a1 to a2? Create a separate directory with name as a2 and then alter the table. ... View more

If the target table is Hive table then you can use -hive-overwrite ... View more

@ozac When you alter the location for hive table, it will not move the data from old location to new location (if any data present). It will only load the data from the new location. The error shows that the required data file is not present. Make sure that the data file is also present in the new location. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. cannot find dir =hdfs://sandbox.hortonworks.com:8020/tmp/a2/000000_0_copy_1 in pathToPartitionInfo: [hdfs:/tmp/a2] ... View more

@Amey Shirke I don't think you can use insert into command in Pig script. Other way is to use sqoop to load this data that was the output from pig script to load into any RDBMS tables. ... View more

@Stinger It won't give you the file size, but it will give you the path where those files are stored, so you can refer that path for details like file size, permissions, etc. ... View more

@John Cleveland Your script looks correct and I see below error in your log file. Not sure why it's not able to setup the load function. 2017-06-06 19:14:44,999 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2081: Unable to setup the load function. I ran the same script with same dataset and my script completed within a minute. PFA for the screenshots. ... View more

Access the path where these files are stored to find the size of these split files. You can do a DESCRIBE EXTENDED /FORMATTED Tablename to find the exact path of the files. ... View more

@John Cleveland Could you please post the complete script? I don't see where the relational join_data is defined in your script. Also, post the log details. ... View more