About mbigelow

mbigelow · ‎07-28-2017

Ok, this means that the CSD file isn't in place and/or Cloudera Manager server was not restarted after it was put in place. Please check both. After the CM server restart the service should be available to add to an existing or new cluster.

mbigelow · ‎07-27-2017

The other time I have seen this the issue was that Spark2 had not been added as a service to the cluster and therefor the Spark2 gateway was not in place. This would include environmental and configuration settings for clients like spark2-shell. At a cluster screen click the Action menu and select 'Add a Service'. Follow the wizard and add Spark2 from the list of services available.

mbigelow · ‎07-27-2017

You can leave the password blank, it will then prompt the user for the password. curl -u user: https;//..

mbigelow · ‎07-27-2017

It is possible that AWS is throwing an error that leads Director to state that it can't get the instance in time. The minimum required is only 1. You could test quickly by launching the same size in the same region to see if there is an issue on the AWS side.

mbigelow · ‎07-27-2017

Hmm I don't have an answer but a comment. I am using CDH 5.8.2. I get this same behavior from HUE. Some connections are not closed when the tab is closed. I don't recall the exact state, but we have the idle_query_timeout and idle_session_timeout set to 1 hour and these are closed after that time. So if idle_query isn't working try idle_session. If that doesn't work, there maybe something going one with your specific setup.

mbigelow · ‎07-27-2017

What instance type are you using? Are they on-demand or reserved? What region and zone is this in? Did you use a Placement group? It could be that you are requesting an instance size, like the d2.8xl, and there just isn't enough room in the region/zone to allocate them in time, 20 mins. It could also be that AWS black magik isn't fast enough in moving instances around if you used a Placement group.

mbigelow · ‎07-27-2017

Lets take a step back. You mentioned that the cluster is Kerberized and therefor Navigator needs some configs along those lines to interact with the cluster but what is its auth mechanism. I believe it can be different. Yes, it is. The External auth options for Nav are AD (LDAP), OpenLDAP, and SAML. If you do not have one of these set up then it will use the Cloudera Manager accounts (which could be internal or external). If it is internal, then you will need to use one of those accounts, with the proper Nav role/group assigned, in the -u cmuser:cmpass switch. You won't need a Kerberos ticket in this case.

mbigelow · ‎07-26-2017

It should work with both --negotiate and -u. curl -v --negotiate -u "username" http://www.blah.com

mbigelow · ‎07-26-2017

Did you restart CM and CMS? If not, then it will not pickup the csd file and it will not be available as a service to install. If you have, for the cluster with the parcels distributed and activated, choose 'Add a Service' from the cluster action menu. Is it available in that list of services?

mbigelow · ‎07-26-2017

Hive has the limitation that it cannot set these values at runtime. They have to either be in the core-site.xml or the table definition (not positive on this one though). The former will require admin access to the CM UI. The latter can be done by the person creating the table, you. Set the location as such, s3a://ACCESS_KEY:SECRET_KEY/path/. This will not work if there is a '/' unless you have this patch. https://issues.apache.org/jira/browse/HADOOP-3733

Online	Offline
Last Visited	‎03-25-2019 05:55 PM

Member Since	‎08-16-2016 08:51 PM
Last Visited	‎03-25-2019 05:55 PM
Posts	642
Kudos received	129

Cloudera Community

Re: Configuring the HDFS superuser in Kerberos

Re: Hive process crash

Re: Upgrade from CDH 5.11 Express to Enterprise

Re: Adding user to Cloudera Manager using REST AP...

Re: Running in non-interactive mode, and data appe...

Re: Spark 2.1 didn't work in CDH 5.12

Re: Spark 2.1 didn't work in CDH 5.12

Re: Kerberos enabled cluster - Curl command for Cl...

Re: Director cannot create EC2 - Insufficient numb...

Re: Query Cancel and idle_query_timeout is not wor...

Re: Director cannot create EC2 - Insufficient numb...

Re: Kerberos enabled cluster - Curl command for Cl...

Re: Kerberos enabled cluster - Curl command for Cl...

Re: Having Spark 1.6.0 and 2.1 in the same CDH

Re: Unable to configure AWS credentials on Oozie h...