@Artem Ervits, @Cassandra Spencer, @Geoffrey Shelton Okot - We've updated our KB articles with multiple options for connecting to the environment, including static IP's. https://hortonworks.my.salesforce.com/articles/en_US/How_To/SmartSense-Gateway-setup https://hortonworks.my.salesforce.com/articles/en_US/How_To/Uploading-SmartSense-Bundles ... View more

Hi @thomas sandidge - Right now the Gateway requires a direct SFTP connection to Hortonworks. We're working on an authenticated HTTPS proxy solution for our next release. Would that alternative work for your security team? ... View more

@Sean Roberts - the answer to that question really depends on the version of Ambari that you're using. There are multiple tools that can be used to calculate the appropriate memory settings for Hive/Tez/YARN/MR and I would treat them in this order of preference based on two things: 1. the frequency of update, and 2. the amount of context that the tool has for its calculations: SmartSense - The memory-related rules are updated very frequently, at least once a quarter and have the most context as it has all of the SmartSense diagnostic information at its disposal and can take into account actual use versus configured use of services (cores, spindles, memory, other services running on that machine, other 3rd party utilities being run on that machine) Stack Advisor - Updated frequently, but tied to Ambari releases so depends on if the customer is using Ambari and if they are which specific version of Ambari and how up to date it is 1.7 vs 2.0 vs 2.1, etc.. HDP Configuration Utility - Most basic and least frequently updated, but if the customer does not have Ambari or SmartSense and is manually deploying HDP is better than nothing. ... View more

From a SmartSense perspective it's about 80% ext4, and 20% XFS. We recommend using either and have specific mount options for each type of filesystem. ... View more

You can script it using the following python script [delete_users.py]: import sys,json from string import Template password='admin' ambari_host='revo5.hortonworks.local' request = Template("echo 'Removing $user_name'\ncurl --insecure -u admin:$password -H 'X-Requested-By: ambari' -X DELETE http://$ambari_host:8080/api/v1/users/$user_name") payload = json.load(sys.stdin) for user in payload["items"]: if user["Users"]["ldap_user"]: print request.substitute(password=password,ambari_host=ambari_host,user_name=user["Users"]["user_name"]) $ curl -u admin:admin 'http://revo5.hortonworks.local:8080/api/v1/users/?Users/user_name.matches%28.*.*%29&fields=*' > users.json $ cat users.json | python delete_users.py > delete_users.sh The python script will output an echo statement for the username and a curl request delete each LDAP user. You can review that shell script as it will have all of the individual curl commands to delete the LDAP users that have been sync'd into Ambari. Once you're comfortable with the list you can run it with bash and observe the process. $ bash delete_users.sh ... View more

@kkane - this is the Ambari JIRA that introduced the change: https://issues.apache.org/jira/browse/AMBARI-10270 A method was introduced that checks if the JDK name (jdk.name) and JDBC database (server.jdbc.database) properties are set to determine if setup has already been run: +def check_setup_already_done(): + properties = get_ambari_properties() + if properties == -1: + print_error_msg("Error getting ambari properties") + return -1 + + return properties.get_property(JDK_NAME_PROPERTY) and properties.get_property(JDBC_DATABASE_PROPERTY) + ... View more

@ccook make sure that the Ambari Server trusts the certificate that the LDAP server is using. One quick way to get that certificate directly is to use openssl to retrieve that certificate from the LDAP server, and then explicitly add it to a new keystore: $ openssl s_client -showcerts -connect ldapserver.domain.com:636 You'll see the certificate printed in STDOUT, just look for BEGIN CERTIFICATE. You will need to grab the entire certificate including the ----BEGIN and END ---- text, and save it to a file. In this case we'll call it ldap.cert. Once this has been done you can follow 1.2.(1-3) steps in the doc to create a new JKS keystore and import that certificate to ensure that it's trusted by Ambari: http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Ambari_Security_Guide/content/_configure_ambari_to_use_ldap_server.html Now you've got a JKS keystore with that certificate in it, you can tell Ambari to use that when connecting to your LDAP server using SSL by re-running the ambari-server setup-ldap. Just make sure you answer correctly for: Use SSL=true TrustStore type=jks Path to TrustStore file=/etc/ambari-server/keys/ldaps-keystore.jks Password for TrustStore={{ what you typed in step 1.2.3 }} ... View more