I got the following responses from Cloudera Certification. Regarding Question #1, the FAQ page has the most the up-to-date information. So right now I'd better hold off purchasing the exam until the DE575 is relaunched. Regarding Question #2, the course is the "Spark and Hadoop Developer" training course is the one I should take for preparing DE575. Regarding Question #3, the environment for the exam is fixed and only available on CDH. Candidates do not have the option to take the exam in an HDP environment. The skills tested are applicable to HDP development as well, it is in the developer track, so it should have nothing to do with the environment that it is running in. It is primarily interested in transforming data that sits on the cluster. ... View more

Finally, I figured out what is going on.  The root cause is that, I only set up testuser on edge nodes, not the name node. I looked into this page, https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-common/GroupsMapping.html, which shows that "For HDFS, the mapping of users to groups is performed on the NameNode. Thus, the host system configuration of the NameNode determines the group mappings for the users."  After I created the user on the NameNode and ran the command hdfs dfsadmin -refreshUserToGroupsMappings the copy is successful and there is no permission-denied error. ... View more

I am happy to see you sorted this out. The community has a responsive design so it adjusts based on the size of the window it is in. At a certain point the menu items collapse into a pull down on the upper left side of the page like you would see on a mobile device. Click on the three bars and the pull down appears, select community and then "ask a question is at the bottom of the list.    ... View more

@Seaport  Integration with Ranger is for Atlas Security and in case you want to use tag based policies which is one of the most common use cases for Ranger + Atlas. But is not a requirement, see following link: https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/configuring-atlas/content/additional_requirements_for_atlas_with_ranger_and_kerberos.html   This is also mentioned in our documentation see more in section: Additional Requirements for Atlas with Kerberos without Ranger   Finally, to install Atlas you are only required to have installed Infra Solr, Kafka and Hbase.    Regards, Felix ... View more

My mistake. For some reason, the reboot of HDP did not restart YARN NodeManagers and I missed it from the Ambari UI. After I manually started each NodeManager, the MR job runs without an issue.  ... View more