In Nifi with the PutS3Object, we get an error using AWS Credentials Provider service. AwsCredentialsProviderControlerService is configured to use IAM roles as follows: Use Default Credentials = True Use Anonymous Credentials = False Assume Role ARN = arn:aws:iam::ahjhdiauisjkk:role/role-test Assume Role Session Name = nifitest (*arbitary name*) Session time = 3600 No other values are set in the AwsCredentialsProviderControlerService We are using IAM roles because of organizational policies. The error is loosely transcribe here (it may contain typos): 13:40:46 EDT - All Nodes - ERROR PutS3Object[id=asdfasdfasdfasdf] Failed to put StandardFlowFileRecord[uuid=xxxxxxxx,claim=StandardContentClaim[resourceClaim=StandardResourceClaim[id=11111111, container=default,section=1], offset=0,length=222222],offset=1,name=test3,size=33333] to Amazon S2 due to com.amazonaws.services.securitytokenmodel.AWSSEcurityToeknServiceException: User: arn:aws:sts::7777777:assumed-role/role-hdf-node/i-03333330000 is not authorized to perform: sts:Assumerole on resource: arn:aws:sts::7777777:role/role-hdf-node (Service: AWSSecurityToeknService;Status Code: 403; Error Code: AccessDenied; Request ID: aaaaaaaaaaaa) com.amazonaws.services.securitytokenmodel.AWSSEcurityToeknServiceException: User: arn:aws:sts::7777777:assumed-role/role-hdf-node/i-03333330000 is not authorized to perform: sts:Assumerole on resource: arn:aws:sts::7777777:role/role-hdf-node (Service: AWSSecurityToeknService;Status Code: 403; Error Code: AccessDenied; Request ID: aaaaaaaaaaaa) Thanks for any help. Jim ... View more

I am trying to configure LDAP authentication with Ambari and other components (e.g. Zeppelin, Ranger and Knox). This is a complex environment with Centrify where users have multiple identities which resolve to a single ID in Linux. On Linux, my id is "u1101". I can execute "id u1101" or "id james-jones" and both result in the same Linux ID -- e.g. uid=60012(u1101) gid=5001(user). On a Windows system, only james-jones works. There are two domains: company.com and corp.company.int. I want to configure LDAP authentication using the Linux ID rather than the windows sAMAccountName, but it only works with the sAMAccountName. I have configured the correct settings in Ambari for userObjectClass, baseDn, userAttributeName, primaryUrl, managerDn, etc. I don't know if the problem is with Centrify or an LDAP server hosting the Linux accounts. I decided to conduct a test, and it seems that using dn bind account is prohibited in one case but not the other. Since Ambari LDAP authentication essentially does two queries, one with the bind user to find the person logging in, and then a second query with the person's DN and the password they entered, I tested two ldapsearch queries. One with the Windows DN and the second using the Linux DN. Using ldapsearch, this query succeeds using my Windows domain DN ldapsearch -D "CN=1101737,OU=NY,OU=my_company,OU=CO_Users,DC=corp,DC=company,DC=int" -W \ -b "dc=corp,dc=company,dc=int" \ "sAMAccountName=james-jones" This query fails on authentication using my UNIX domain DN ldapsearch -D "CN=james-jones@corp.company.int,CN=Users,CN=Global,CN=Zones,OU=Centrify,OU=Enterprise Systems,DC=corp,DC=company,DC=int" -W \ -b "DC=corp,DC=company,DC=int" \ "sAMAccountName=james-jones" There are two different LDIFs for these two ids (u1101 and james-jones): sAMAccountName=james-jones objectClass=person dn=CN=1101737,OU=NY,OU=my_company,OU=CO_Users,DC=corp,DC=company,DC=int distinguishedName==CN=1101737,OU=NY,OU=my_company,OU=CO_Users,DC=corp,DC=company,DC=int The other looks like non-AD LDAP and I think openLdap may be behind it with some centrify magic going on. uid=u1101 objectClass=posixAccount dn=CN=james-jones@corp.company.int,CN=Users,CN=Global,CN=Zones,OU=Centrify,OU=Enterprise Systems,DC=corp,DC=company,DC=int distinguishedName=CN=james-jones@corp.company.int,CN=Users,CN=Global,CN=Zones,OU=Centrify,OU=Enterprise Systems,DC=corp,DC=company,DC=int The reason this matters is that when I use Ambari views, my account needs to match the Unix accounts. Hopefully that made sense. It's a bit long and complicated. Any help is very appreciated. ... View more

Ambari 2.5.0.3 complains about openblas-Rblas x86_64 0.2.19-4.el7 installed when installing HDP 2.6. This was in the checks done during service installation. What version of R is HDP happy with? If we remove openblasRblas, it will also remove R 3.3.3 and we want to replace it with the correct version. ... View more

In Ambari 2.5.0.3 I noticed the following options under "Misc" when I want to specify an custom service account name: Skip group modifications Have Ambari manage UIDs Whether to skip creating users and groups in a sysprepped cluster Previously I only remember seeing "Skip group modifications", but may be the UID option was there. The last option definitely seems new to me. Anyway, is there documentation or an HCC article explaining exactly what these do and when I would want them? I can take a guess based on the names, but it would be nice to have a clear understanding. Our present need is to have service accounts and groups pre-created in AD/Centrify and I assume not defined on the local OS at all. In this case it seems like checking all three options would be safe, but option 3 seems like it might imply options 1 and 2 are selected. I would like advice on how to proceed with my current scenario, but any additional insight into what these actually do is appreciated. Thanks. Incidentally, for those that care - you can set these three options above in blueprints. See my previous question: https://community.hortonworks.com/questions/103647/how-to-use-blueprints-with-pre-created-accounts-fo.html ... View more

Is there a way to use blueprints assuming we have pre-created service accounts (created in AD/Centrify), and if so, which I could use some help. I assume that in the blueprints I could add properties something like this: "spark_user" : "svcspark", "spark_group" : "svcspark" Part 1 of this question -- Will that cause Spark to run as svcspark? Also, related to this, I noticed there are now 3 options (checkboxes) in Ambari 2.5.0.3 which may relate to pre-created accounts. These are found under Misc when adding a service: Skip group modifications Have Ambari manage UIDs Whether to skip creating users and groups in a sysprepped cluster Part 2 of this question -- Can these be set in blueprints and if so, how? Our plan is to use blueprints but we need to create all service accounts in AD/Centrify before Ambari or services are installed. Thanks ... View more