The suggestion here is not to use S3 to populate HDFS - as you have gathered, that would involve moving data back and forth. It isn't necessary for your use-case, though some folks do indeed do just that if their raw data is in S3 and they need to use HDFS for latency requirements. However, the suggestion here is to have your processing jobs (MR, Impala, etc.) run directly off of S3. See this blogpost as an example: http://blog.cloudera.com/blog/2016/08/analytics-and-bi-on-amazon-s3-with-apache-impala-incubating/ The "off the beaten track" was referring to using the bootstrap script to mount EBS volumes. This is extending Director's existing functionality, which should come out of the box soon. But until then, I'm not aware of anyone else who has written such a bootstrap script. Hence the warning. Hope this helps.
... View more
