About kkane

kkane · ‎01-05-2016

I use a detailed Ambari blueprint to install a HDP cluster. When a new version of Ambari is released my blueprint upgrade technique is to manually install a similar cluster, export the new blueprint, diff old and new, then update the new blueprint for anything missed. This process is time consuming and prone to error. Is there a better way to maintain blueprints across Ambari versions?

kkane · ‎01-04-2016

Thanks Enis!

kkane · ‎12-30-2015

Would a short CF TTL of 30 minutes on 2 to 36 million row table be detrimental to performance? This HBase table would be queried or inserted from Storm at a goal rate of 20k entries per second at peak performance and would like the current entries to expire after 30 minutes.

kkane · ‎12-28-2015

I now see where the agent is calling initSchema... had missed it before. It fails when run from Ambari but succeeded when run from the cli. I'll accept an answer and drive on.

kkane · ‎12-28-2015

Ambari 2.1.2 - HDP 2.3.2 Startup of the Hive metastore fails because schema tool can't validate the the hive.VERSION table. According to this, http://docs.hortonworks.com/HDPDocuments/Ambari-2.... Ambari is supposed to create the schema automatically with an HDP stack of 2.2 (I'm assuming later versions as well) Kris

kkane · ‎12-04-2015

For anyone that is interested in why I want this: People use Oozie Shell actions to orchestrate work because they are familiar/comfortable with bash. When using the Hive CLI from a YARN container context and secure cluster the engine specification must be done in an early stage. For the Hive CLI this means users must change their existing HQL from hive -e 'set tez.credentials.path=${HADOOP_TOKEN_FILE_LOCATION}' to --hiveconf tez.credentials.path=${HADOOP_TOKEN_FILE_LOCATION} For Sqoop, there is no way to affect this change from configuration and the user must ship a custom hive-site.xml in the distributed cache of the Oozie-shell. This hive-site.xml must set the engine to MR If this is not done, Hive and Sqoop will launch a Tez AM that may never even be used but which fails due to missing the delegation token. These changes are small from from this explanation but rather large for organizations that already have established HQL and the cluster upgrade includes a default engine change from MR to Tez.

kkane · ‎12-04-2015

Looks like the answer is no: https://issues.apache.org/jira/browse/HIVE-9602

kkane · ‎12-04-2015

The desired end result is that I want no YARN containers automatically started when I enter the Hive CLI and the engine default is Tez. hive.prewarm prevents default work containers but I need to stop the Tez AM from launching.

kkane · ‎12-04-2015

Since my question pertains to containers, prewarm is the correct answer but it is not the correct question so I will ask again.

kkane · ‎12-04-2015

So, I've tog hive.prewarm.enabled=false and still get a tez session from the Hive CLI.

Online	Offline
Last Visited	‎07-01-2016 12:57 AM

Member Since	‎09-26-2015 12:45 AM
Last Visited	‎07-01-2016 12:57 AM
Posts	30
Kudos received	10

Cloudera Community

Re: Is there a way to prevent the start of the Tez...

Re: Can I use the token files present in Oozie to ...

What is the best way to upgrade Ambari blueprints ...

Re: When is HBase CF TTL too short?

When is HBase CF TTL too short?

Re: When does Ambari create the Hive schema when u...

When does Ambari create the Hive schema when using...

Re: Is there a way to prevent the start of the Tez...

Re: Is there a way to prevent the start of the Tez...

Is there a way to prevent the start of the Tez AM ...

Re: How to prevent default Tez containers.

Re: How to prevent default Tez containers.