With your original approach, each query can filter out whole partitions of the table based on the WHERE clauses (that is if your table is partitioned and at least some of the columns in the clause use those partitions). However, if your WHERE clauses are pretty different/unique, then you will be scanning big portion of the table for every one of your 100+ queries.   With the suggested approach, there is only one scan of the table, but there is more processing that is happening for each row. The best way to see if performance is better is just to test it and go with the winner. ... View more

The error likely indicates that some AWS resources were not reachable from CDP control plane. Double check your security policy settings and any proxy settings. Reach out to support as they will be able to better assist by being able to look at your particular environment setup.   Regarding the logs, if the CM instance was stood up in your Data Lake, you can search the logs by clicking "Command logs" or "Service logs" in the Data Lake tab of your Data Lake environment.  ... View more

Try adding these options in your sqoop command: --relaxed-isolation --metadata-transaction-isolation-level TRANSACTION_READ_UNCOMMITTED More info here: https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.3/bk_data-movement-and-integration/content/controlling_trans_isol.html and here: https://www.tutorialspoint.com/java-connection-settransactionisolation-method-with-example   Hope this helps.   Regards, Alex     ... View more