Re: HDP on AWS

ewalk · ‎07-26-2016

@ARUN Cloudbreak is the tool that will allow you to automate this using Ambari Blueprint.

ewalk · ‎07-05-2016

That sounds right given the error message.

ewalk · ‎07-01-2016

LDAP 49 errors can mean a lot of different things. Can you please check the logs from the LDAP server for the specific error cause? This is very likely a bad password error... Atlassian has a good explanation of the different error codes: https://confluence.atlassian.com/kb/common-user-management-errors-820119309.html

ewalk · ‎06-23-2016

To explain why you use different commands for the two nodes: The idea here is that you don't want both nodes to attempt the rollback and risk becoming inconsistent. In a perfect world you could probably do the rollback on both, but the world isn't perfect. The best way to guarantee consistency is to have one node do the rollback and then rebootstrap the second node to the first (i.e. overwrite node 2's state information with that from the rolled back node 1).

ewalk · ‎06-21-2016

AD is most definitely the easiest answer, unless you're morally opposed to it ;). You get integrated LDAP and KRB with nice user management tools. IPA does have some nice ootb features, though, around self service, etc.

ewalk · ‎06-21-2016

I generally recommend letting DNS handle this. The latest versions of the KRB client will default to resolving the KDC from SRV records in the DNS for the realm. This should be configured by default if you use Microsoft Active Directory (or AWS Simple AD). If you want it explicit in your krb5.conf file, you can use DNS round robin with the A/AAA/CNAME and reference that name in krb5.conf. Further, you could have multiple "kdc" entries for a realm in krb5.conf and a master_kdc entry which is only used when there are certain kinds of issues. You can always manage the krb5.conf from Ambari inside the Kerberos component configs.

ewalk · ‎06-20-2016

@Shishir Saxena In approach a, lack of involvement from enterprise security teams is not a positive thing. When dealing with a large enterprise, security is paramount and we should never be recommending that HDP administrators be permitted to manage authentication systems. Separation of duties is a core security principle and should not be taken lightly.

ewalk · ‎06-20-2016

For a general enterprise scenario I'd recommend approaches b and c. Depending on the security administrators, they will agree to one of these. In general it is preferable to reduce the number of sources of identity within an organization to allow for easily managed, secure control. I would very strongly advise against the stand-alone KDC approach in any real production environment.

ewalk · ‎05-11-2016

@Chris McGuire, that is probably the case, I'm not very familiar with the way spark is configured. I do know that, generally speaking, unless you explicitly say insert or use hive streaming you don't have deltas and don't need to worry about compaction. The partition append merging is a whole different story...

ewalk · ‎05-11-2016

@mbalakrishnan, do you think it might be missing it because they are originating from spark, not map, mapred or tez?

ewalk · ‎05-11-2016

@Chris McGuire, I'm not sure you're using the Hive Streaming API, then. I'm not sure how Spark Streaming is setup to write out to hive, so it could be behaving correctly.

ewalk · ‎05-11-2016

Hi @Chris McGuire, Can you please provide an "hdfs dfs -ls -R <table-folder>" Compaction only operates on tables with delta directories. I suspect that the method you're using (SaveMode.Append) is just appending to the existing partition (or adding a new partition) and not actually creating deltas. Best, Eric

ewalk · ‎01-25-2016

I would generally say that if you're talking SSO for ODBC on Windows, the easiest option client side is Kerberos. As long as the user is logged into his workstation as an AD user that has rights on the Hive tables, the Kerberos ticket the OS gets at login will be sufficient for authentication to Hive over the SASL transport. You just need to make sure you've kerberized your cluster using one of your Active Directory Domain Controllers as the KDC (http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_Ambari_Security_Guide/content/ch_configuring_amb_hdp_for_kerberos.html). At that point just configure the ODBC driver for kerberos. Again, the only client side requirement is that the machine is joined to the domain and the user logs in with an authorized account.

ewalk · ‎01-11-2016

A few things to double check: 1. Is there any message on the Hiveserver2 that correlates to the Beeline error? 2. Can you double check that you have the Unlimited Strength Key JCE Policy installed correctly and that alternatives is pointing at the copy of java that has this poilcy (I'm noticing that your KRB Ticket only supports AES256) 3. Set system property sun.security.krb5.debug to true on your JVM and see if you can get any details from the debug logs.

ewalk · ‎12-31-2015

In the short run you can always look at the metastore database (assuming you're using the db txn manager) and try to clear them manually from the tables there.

ewalk · ‎12-22-2015

@Darpan Patel I would double check those host names and that the ports are open.

ewalk · ‎12-22-2015

@Darpan Patel Haven't tried setting that up in a NameNodeHA environment yet, but it seems that it is trying to resolve the reference to the NN Service Name in DNS and failing. As for the Hive error, I'd suggest stopping ambari-server, doing a kdestroy for the user as which ambari-server runs and a kinit as the ambari-server user before starting it again.

ewalk · ‎12-21-2015

No worries, I hope some of these things have been fixed since I went through this back in September (#4 should be resolved in Ambari 2.1.2). The Kdestroy/Kinit thing was definitely strange, never did work out why that was needed.

ewalk · ‎12-21-2015

So I had a bunch of trouble with these, here are some of the things to note: When creating the view in Ambari don't use the "Local Ambari Managed Cluster" option, always use the custom when you have a kerberized cluster. Definitely read the instructions carefully (i.e. this one: http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_ambari_views_guide/content/section_pig_view_kerberos_config.html) per @Neeraj Sabharwal. Stop Ambari Server, do a kdestroy for the user ambari-server run as, do a kinit for the ambari user using it's proper keytab as the ambari linux user, then start ambari-server again. Do this procedure each time you restart Ambari Server. For the pig view, there was a known issue where you needed to add: ,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar to your templeton.libjars for WebHCat (https://issues.apache.org/jira/browse/AMBARI-13096). Check your Ambari version...

ewalk · ‎12-18-2015

If you wish to reference a file in S3 from a pig script you might do something like this: set fs.s3n.awsSecretAccessKey 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxx'; set fs.s3n.awsAccessKeyId 'xxxxxxxxxxxxxxxxxxxxx'; A = load 's3n://<bucket>/<path-to-file>' USING TextLoader; If you're on HDP 2.2.6, you'll likely see this error: Error: java.io.Exception, no filesystem for scheme: s3n The following steps resolve this issue: In core-site.xml add: <property> <name>fs.s3n.impl</name> <value>org.apache.hadoop.fs.s3native.NativeS3FileSystem</value> <description>The FileSystem for s3n: (Native S3) uris.</description> </property> Then add to the MR2 and/or TEZ class path(s): /usr/hdp/${hdp.version}/hadoop-mapreduce/* These configs ensure 2 things: That the worker YARN containers spawned by pig have access to the hadoop-aws.jar file That the worker YARN containers know which class implements the file system type identified by "s3n://" References: Apache Mailing Lists Topic from Legacy Hortonworks Forums

ewalk · ‎12-17-2015

The root user's umask is 0027

ewalk · ‎12-17-2015

Yeah, I've thought about that, I don't like it... it's one more thing that's hacked together manually and has to be tracked and maintained. I'd much prefer a fix to Ambari's config...

ewalk · ‎12-15-2015

This is HDP 2.2.6, Ambari 2.1.2.1

ewalk · ‎12-15-2015

Ambari sets up set-hdfs-plugin-env.sh with root:hadoop 700 permissions when it restarts HDFS, this causes ranger integration to break as the hdfs user cannot execute this script when the namenode starts. I can fix the problem if I restart the namenode manually, but that means to deploy config changes I need to restart from Ambari, then correct permissions and restart manually. How

Online	Offline
Last Visited	‎09-06-2016 06:29 PM

Date Registered	‎12-08-2015 06:44 PM
Last Visited	‎09-06-2016 06:29 PM
Total Messages Posted	34
Total Kudos Received	19

Re: HDP on AWS

Re: Kerberos KDC secondary

Re: Hive compactions on External table

Re: HDP on AWS

Re: issue with openldap/kerberos

Re: issue with openldap/kerberos

Re: What's the difference of -bootstrapstandby and...

Re: Looking for an automated integration of HDP/Am...

Re: Kerberos KDC secondary

Re: Choosing Kerberos approach for Hadoop cluster ...

Re: Keberos Implementation Approach: is it recomme...

Re: Hive compactions on External table

Re: Hive compactions on External table

Re: Hive compactions on External table

Re: Hive compactions on External table

Re: Windows authentication (Active Directory) to H...

Re: beeline and kerberos

Re: Hive Acid: How to kill Locks & transaction ?

Re: Configuring ambari views on Kerberized Cluster

Re: Configuring ambari views on Kerberized Cluster

Re: Configuring ambari views on Kerberized Cluster

Re: Configuring ambari views on Kerberized Cluster

HDP 2.2 Configuration required for S3

Re: How can I fix incorrect file permissions for s...

Re: How can I fix incorrect file permissions for s...

Re: How can I fix incorrect file permissions for s...

How can I fix incorrect file permissions for set-h...