Member since
05-19-2016
25
Posts
18
Kudos Received
0
Solutions
10-19-2018
07:42 AM
Update: The tag propagation feature (ATLAS-1821) was released as part of HDP 3.0.0.
... View more
10-01-2018
11:26 AM
HBase, like Hadoop, uses the Simple Authentication and Security Layer (SASL) framework implementation in the
Java SE libraries to provide authentication and message integrity and privacy services.
When a cluster is secured with Kerberos and
hbase.rpc.protection=privacy , SASL uses the GSS-API (Java-GSS) framework to initiate
Kerberos authentication between client and server, which results in the sharing of a (symmetric) cryptographic key between client and (Region) server which can be used for message (HBase payload) encryption.
With the JCE libraries installed, the cipher/encryption type used should default to
aes256-cts-hmac-sha1-96 .
... View more
09-18-2018
04:05 AM
The HBase Book says this about enabling wire-encryption in HBase: Once HBase is configured for secure RPC it is possible to optionally configure encrypted communication.
To do so, add the following to the hbase-site.xml file on every client:
<property>
<name>hbase.rpc.protection</name>
<value>privacy</value>
</property> What is/are the encryption algorithm(s) used when HBase wire-encryption is enabled?
... View more
Labels:
- Labels:
-
Apache HBase
10-10-2017
02:24 AM
Thank you Jay. This worked.
... View more
10-09-2017
03:01 AM
I'd like to automate a cluster build using FreeIPA as the identity/authentication infrastructure and I'm looking for a way to set the enableIpa flag in Ambari's experimental feature set via the REST API. Is this possible?
... View more
Labels:
- Labels:
-
Apache Ambari
02-03-2017
12:48 AM
2 Kudos
Thanks @Jay SenSharma. Basically, there is no official process for doing this, and the best thing would have been to ensure you do not ever choose the wrong database technology when you are first installing Ambari! If you can get an experienced DBA to carry this out, that would be the safest way to approach it. The conversion process documented hadoopadmin.co.in is useful for a high-level procedure, however with Ambari 2.2.2 I found there are a significant number of changes that needed to be made to the SQL source before it was compatible with MySQL, including: Commenting out SET statements at the top of the dump file. Changing the case of some table names - e.g. qrtz_blob_triggers in PostGreSQL becomes QRTZ_BLOB_TRIGGERS in MySQL Pre-creating some tables that are created during Ambari run-time with names such as ds_jobimpl_6 Definitely worth having a test environment for this operation if you cannot afford to lose your cluster and all the data in it. If you're not deeply familiar with PostGreSQL/MySQL SQL differences then the conversion tool suggestion is a good one.
... View more
01-24-2017
07:43 AM
2 Kudos
We have a few clusters that were installed using Ambari running with a PostGreSQL back-end (the default) but need to standardise on a DBA-managed MySQL set-up. What is the recommended method to convert Ambari to use MySQL? The clusters are Kerberised and all components are running in HA mode.
... View more
Labels:
- Labels:
-
Apache Ambari
12-15-2016
12:47 PM
3 Kudos
For HDP 2.5+ the answer is yes. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_security/content/configure_knox_for_ha.html
... View more
12-06-2016
09:27 AM
2 Kudos
Thank you for writing this up @Jonas Straub and thanks for the extra details @Michael Young. In case it helps anyone else, here is the entire "add-unknown-fields-to-the-schema" section. Not only did I have to apply the classes as @Michael Young has pointed out, but I also had to ensure that both these new processors were added at the top of the updateRequestProcessorChain. This was using a stand-alone SolrCloud instance that was installed as part of Ambari 2.2.2 with HDP 2.4.2. For that install, I found that I had to re-run the install SolrCloud install steps per the HDP Ranger Audits to SolrCloud documentation, consisting of: Running add_ranger_audits_conf_to_zk.sh Starting solr (start_solr.sh) Running create_ranger_audits_collection.sh <updateRequestProcessorChain name="add-unknown-fields-to-the-schema">
<processor class="solr.DefaultValueUpdateProcessorFactory">
<str name="fieldName">_ttl_</str>
<str name="value">+14DAYS</str>
</processor>
<processor class="solr.processor.DocExpirationUpdateProcessorFactory">
<int name="autoDeletePeriodSeconds">300</int>
<str name="ttlFieldName">_ttl_</str>
<str name="expirationFieldName">expire_at</str>
</processor>
<processor class="solr.RemoveBlankFieldUpdateProcessorFactory"/>
<processor class="solr.ParseBooleanFieldUpdateProcessorFactory"/>
<processor class="solr.ParseLongFieldUpdateProcessorFactory"/>
<processor class="solr.ParseDoubleFieldUpdateProcessorFactory"/>
<processor class="solr.ParseDateFieldUpdateProcessorFactory">
<arr name="format">
<str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str>
<str>yyyy-MM-dd'T'HH:mm:ss,SSSZ</str>
<str>yyyy-MM-dd'T'HH:mm:ss.SSS</str>
<str>yyyy-MM-dd'T'HH:mm:ss,SSS</str>
<str>yyyy-MM-dd'T'HH:mm:ssZ</str>
<str>yyyy-MM-dd'T'HH:mm:ss</str>
<str>yyyy-MM-dd'T'HH:mmZ</str>
<str>yyyy-MM-dd'T'HH:mm</str>
<str>yyyy-MM-dd HH:mm:ss.SSSZ</str>
<str>yyyy-MM-dd HH:mm:ss,SSSZ</str>
<str>yyyy-MM-dd HH:mm:ss.SSS</str>
<str>yyyy-MM-dd HH:mm:ss,SSS</str>
<str>yyyy-MM-dd HH:mm:ssZ</str>
<str>yyyy-MM-dd HH:mm:ss</str>
<str>yyyy-MM-dd HH:mmZ</str>
<str>yyyy-MM-dd HH:mm</str>
<str>yyyy-MM-dd</str>
</arr>
</processor>
<processor class="solr.AddSchemaFieldsUpdateProcessorFactory">
<str name="defaultFieldType">text_general</str>
<lst name="typeMapping">
<str name="valueClass">java.lang.Boolean</str>
<str name="fieldType">booleans</str>
</lst>
<lst name="typeMapping">
<str name="valueClass">java.util.Date</str>
<str name="fieldType">tdates</str>
</lst>
<lst name="typeMapping">
<str name="valueClass">java.lang.Long</str>
<str name="valueClass">java.lang.Integer</str>
<str name="fieldType">tlongs</str>
</lst>
<lst name="typeMapping">
<str name="valueClass">java.lang.Number</str>
<str name="fieldType">tdoubles</str>
</lst>
</processor>
<processor class="solr.LogUpdateProcessorFactory"/>
<processor class="solr.RunUpdateProcessorFactory"/>
</updateRequestProcessorChain>
... View more