About jknulst

jknulst · ‎06-01-2017

@Predrag Minovic @Balint Molnar thanks for your input. I thought of the option to do the minimal (manual) kafka install on the node, but the overhead to manually sync the kafka config from there onwards did not look that appealing to me. So instead, I installed a Ambari managed Kafka broker on the node. I prevent that broker from ever having any topics by introducing a special CG for that node with auto.create.topics.enable set to 'false'.

jknulst · ‎05-29-2017

Hi, Ambari has no option to install a 'Kafka Client'. So if you want to have the Kafka shell scripts kafka-topics.sh etc plus Ambari managed kafka configs there seems to be no other option then to install a full Kafka broker on the node. Is there a way to have a Kafka client or broker that is fully synced with Kafka config by Ambari, but without it ever receiving Kafka topics on disk?

jknulst · ‎05-09-2017

BEWARE !! In the connection string below: beeline -u "jdbc:hive2://rjk-hdp25-s-01:2181,rjk-hdp25-s-02:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/rjk-hdp25-m-02@FIELD.HORTONWORKS.COM" the principal hive/rjk-hdp25-m-02@FIELD.HORTONWORKS.COM is actually the one running the Hiveserver2 AND NOT the user that you have just used to kinit'ed with yourself just before you fired the beeline command !!! Then it works. It is very very confusing, so beware! To survive a hiveserver2 HA failover, the syntax: hive/_HOST@FIELD.HORTONWORKS.COM also works

jknulst · ‎05-09-2017

I have a solution for option 1.) now. If you add a custom yarn-site.xml property on Ambari like below, the SSL nodemanager WebUI starts on port 8042 yarn.nodemanager.webapp.https.address 0.0.0.0:8042

jknulst · ‎05-08-2017

@Vaisakh Sundaram I tried your approach, but after the server restart it does not stop the alert to still call on port 8042 instead of 8044.

jknulst · ‎05-05-2017

Hi, I have all YARN WebUI's running on SSL. This results in the nodemanagers webUI running on port 8044 instead of the default, which is 8042. The NM WebUI is available at 8044. Now I see lots of Ambari alerts, since Ambari still calls port 8042 for performing YARN health checks. There are 2 options here: 1.) Make NM start its webUI on 8042 still (with SSL). The default yarn-site.xml has this yarn.nodemanager.webapp.address ${yarn.nodemanager.hostname}:8042 but apparently this is changed into 8044 (maybe in Yarn source code, maybe by Ambari yarn config templates (??) or 2.) Notify Ambari somehow to do the YARN health check on the correct port; 8044 So, far I could make neither work for me. Please help

jknulst · ‎01-29-2017

@Camille Bernardez Please mark the question as answered, if it is answered

jknulst · ‎01-27-2017

@Camille Bernardez Not really, if you have a couple of Regionservers and the key space is distributed evenly. Once again this is the sweetspot of HBase, high volume random read/write workloads.

jknulst · ‎01-27-2017

@Camille Bernardez That would be absolutely no problem. It is exactly what Hbase was designed for; handling many, many concurrent insert (put) and retrieve (get) requests. So it will handle two or more concurrent jobs, even if they handle the same keys.

jknulst · ‎01-27-2017

@Reddy The commands below work: Hive does not accept subquery in that DDL clause, but this works: ALTER TABLE myTable DROP PARTITION (date < 'date1') , PARTITION (date >'date2'); It needs literals for 'date1' and 'date2'. If you need these to be dynamic then you can use ' --hivevar date1=xxxxx ' for it. More on that here

Online	Offline
Last Visited	‎12-05-2024 10:10 AM

Member Since	‎08-15-2016 02:39 PM
Last Visited	‎12-05-2024 10:10 AM
Posts	189
Kudos received	63

Cloudera Community

Re: Metron Connection failed [Errno 111] error on ...

Re: How do I verify all of my data from enrichment...

Re: Metron profiler client unbale to load native h...

Re: Kafka ConsumerGroupCommand Error

Re: Metron Parser Fail to start with Ambari 2.5 an...

Re: How to install a Ambari managed Kafka client

How to install a Ambari managed Kafka client

Re: beeline and kerberos

Re: Yarn-site template for SSL

Re: Yarn-site template for SSL

Yarn-site template for SSL

Re: Concurrent Put in Hbase

Re: Concurrent Put in Hbase

Re: Concurrent Put in Hbase

Re: Need to drop a hive partition based on date ra...