it is possible, I see people sqoop into manged table then alter table to be external, sqoop continues to work http://stackoverflow.com/questions/27991258/how-to-create-external-table-in-hive-using-sqoop-need-suggestions#29602510 Another option is to manually create table in hive as external and sqoop reference to that table, see section on incremental_table http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_dataintegration/content/incrementally-updating-hive-table-with-sqoop-and-ext-table.html For sqoop to create an external table outright, you might want to open an enhancement jira ... View more

Try to add the rest of the hdfs ports to port forwarding https://ambari.apache.org/1.2.3/installing-hadoop-using-ambari/content/reference_chap2_1.html and if you are on HDP 2.5 sandbox follow this tutorial to port forward https://community.hortonworks.com/articles/65914/how-to-add-ports-to-the-hdp-25-virtualbox-sandbox.html ... View more

Here you go https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md ... View more

are you asking about balancing HDFS so that all nodes are evenly distributed? In that case you need to run HDFS balancer and it will spread that node's data across nodes, otherwise it defeats the point of balancing out HDFS. You can use -include -f hostsfile to hint which nodes to run balancer against but idea is you need more than one node there, is meant to be across multiple datanodes not single datanode. https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer If your question is to balance data across all disks on one node, you can use disk balancer, sadly it's a new feature in Hadoop 3. https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSDiskbalancer.html ... View more

Interesting question, I should try it out. That said if you can't figure out on your own or won't get other responses, then try creating a hive schema on top of your avro files, then run the following statement in hive to create a new table in orc or text format from your original avro table. Then you should be able to find examples of exporting with sqoop from orc or text create table newtable as select * from avrotable --sample avro backed hive table CREATE DATABASE DEMO; USE DEMO; CREATE EXTERNAL TABLE mailinglist COMMENT "just drop the schema right into the HQL" ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' LOCATION '/tmp/rdbms' TBLPROPERTIES ( 'avro.schema.literal'='{ "type": "record", "name": "mailinglist", "namespace": "any.data", "fields": [ { "name": "id", "type": [ "null", "int" ] }, { "name": "first_name", "type": [ "null", "string" ] }, { "name": "last_name", "type": [ "null", "string" ] }, { "name": "email", "type": [ "null", "string" ] }, { "name": "gender", "type": [ "null", "string" ] } ] }'); ... View more

why would you want to do that? Try HDC to spin up an HDP cluster with ease. http://hortonworks.com/products/cloud/aws/ If you need more than just ETL, Data Science, Machine Learning or Bi then try Cloudbreak. http://hortonworks.com/apache/cloudbreak/ ... View more