@ammu ch Can you share the output of this command? /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p ******* get <hivehosts> <cluster> hive-site |grep dir And sudo -u hdfs hdfs crypto -listZones|grep <usename> ... View more

@henryon wen Can you please check if you have below properties?. <property> <name>hadoop.proxyuser.httpfs.groups</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.httpfs.hosts</name> <value>*</value> </property> See this doc http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_hadoop-ha/content/ha-nn-deploy-hue.html ... View more

@Phoncy Joseph You can set the input record size in hive to a higher value to reduce the number of mappers but you might need to increase the mapper heap size also. set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat; set mapred.min.split.size=100000000; Or Try using hadoop har file achieve to small file into single file. https://hadoop.apache.org/docs/r1.2.1/hadoop_archives.html#Looking+Up+Files ... View more

@Zack Riesland You can put it directly through "hdfs fs -put /tablepath s3://bucket/hivetable. If you have partitions in hive table and you can run this command for each partition directory in concurrent mode through a small shell script just to increase the data ingestion speed. And same S3 data can be used again in hive external table. CREATE EXTERNAL TABLE mydata (key STRING, value INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LOCATION 's3n://mysbucket/'; ... View more