Hello.   The error clearly states that you don't have valid cloud storage path. Can you verify the S3 bucket path and its permissions? ... View more

So does the storageadmin has all the required permissions on the bucket ? ... View more

The user with which you are trying to access the bucket has all the appropriate permissions to read,write on that particular bucket ? ... View more

Hello  Yes, Can you upgrade the CDH  5.14 to 6.3.x    The below document should be helpful for you to upgrade. https://docs.cloudera.com/documentation/enterprise/upgrade/topics/ug_cdh_upgrade.html   But i would highly recommend you to connect with the Cloudera professional team for assistance on the production upgrades. ... View more

Hello ,   Can you check and increase the below parameters?   --conf spark.executor.memory=XXX increasing number of executors Also, See below doc for tuning your spark jobs. https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/ ... View more

@vidanimegh I have done some research and there does not appear to be a way to disable the HTTP Options method from the Ambari Web UI. An RMP has been raised for this feature (RMP-10941) but it is not currently available. In terms of security, these options are made available because Ambari can be administered via curl API calls but each call is authenticated. In terms of other ways to secure the WebUI, SSL can be enabled to ensure any response is encoded and Kerberos can be enabled to further secure the cluster. ... View more

Hello @vidanimegh  ,  Do you have any CVE number for the vulnerability you are facing by the internal team? Do you want to disable HTTP? That logically means you want to enable HTTPS. Correct me if my understanding is wrong. ... View more

Hello Megh,   Yes, You can surely do it using the command  #ambari-server setup-security. This will make changes in the ambari.properties   You can refer to the below doc for complete steps by steps setup to disable HTTP and enable HTTPS. https://docs.cloudera.com/HDPDocuments/Ambari-2.2.0.0/bk_Ambari_Security_Guide/content/_optional_set_up_ssl_for_ambari.html ... View more

The script looks good, Can you check the permission for this script and provide 755 for the same? Also, verify the ownership. ... View more

Then this could be an issue with the file permissions or the ownership. Can you share the  /opt/cloudera/parcels/Anaconda-5.3.1/bin/python2.7 setup.py file contents ? Attaching it could be a good idea. ... View more