Member since
05-02-2019
19
Posts
15
Kudos Received
0
Solutions
08-22-2018
08:13 AM
@Vivek Somani : Can you please let me know if Hive LLAP is supported by OneFS in new release? If yes please share the version . Thanks in advance for your help .
... View more
06-30-2017
03:24 AM
5 Kudos
The latest and greatest version of Hortonworks
HDB i.e version 2.2.0.0 addresses many of the concerns from customers/prospects who have been
considering Hortonworks HDB for their production data analytics workloads. Below is the list of major improvements available
in Hortonworks HDB 2.2.0.0 over its predecessor.
Ranger support (in BETA mode) Lack of support
for Ranger has been a deal breaker in field in past. Hortonworks HDB 2.2.0.0 is step in right direction and includs Beta support for Apache Ranger. Hortonworks
HDB 2.2.0.0 now include a RESTful service that provides integration between HDB and
Ranger policy management. Ranger
integration enables you to use Apache Ranger to authorize user access to Hortonworks
HDB resources. Using Ranger enables you to manage all of your Hadoop
components’ authorization policies using the same user interface, policy store,
and auditing stores. For more details on how to use Ranger for authorization in Hortonworks
HDB 2.2.0.0, please refer to http://hdb.docs.pivotal.io/220/hawq/ranger/ranger-overview.html PXF ORC with Pivotal HDB
ORC is a columnar
file format that provides a highly efficient way to both store and access HDFS
data. ORC files offer improvements over text and RCFile formats in terms of
both compression and performance.ORC support is
provided by a new profile “HiveORC “ exposed in the PXF Hive plug-in. The “HiveORC” profile supports access to data
stored in ORC format only through a Hive mapped table Optimized Row Columnar file format, which had been Beta in prior versions, is now fully supported by Hortonworks HDB 2.2.0.0
Support for CentOS 7 / RHEL 7 Hortonworks
HDB 2.2 now has downloads for Red Hat Enterprise Linux 7 and CentOS 7 operating
systems Compatibility with Hortonworks Data Platform ecosystem
Hortonworks HDB 2.2 is certified to work
with Hortonworks Data Platform (DHP) 2.5.3 and Ambari 2.4.2 The support for HDP 2.6 is currently
undergoing testing and is tentatively planned to be generally available in Q3
2017
... View more
01-12-2017
07:49 PM
3 Kudos
Thanks to deep collaboration between Hortonworks and Isilon engineering teams, we have come up with the below table that shows the status of Ambari Views Through Ambari 2.2.2
Ambari
Version
Kerberos
Enabled
WebHDFS*
Enabled
Ambari View
Works?
Remarks
Through Ambari
2.2.2
Y
Y
No
Does
not work with Kerberos enabled because OneFS implementation of WebHDFS does not support chunked encoding.
However we believe that if we use HTTPFS in front ,
it should work. This has not been officially tested though.
Through Ambari
2.2.2
Y
N
No
Does not work in this combination . This is default behavior of Ambari views which does not support Kerberos without WebHDFS.
Through
Ambari 2.2.2
N
Y
Yes
If
the default fs.defaultfs is set to "hdfs" in core-site.xml, then it
works.
The default value in Ambari is: hdfs:// <rpc
address, so it works by default.
if you need to use custom Hive View Cluster Configuration,
be sure to use hdfs:// protocol and port 8020 for the WebHDFS FileSystem URI.
Through
Ambari 2.2.2
N
N
Yes
This should work without any change
if you need to use custom Hive View Cluster Configuration,
be sure to use hdfs:// protocol and port 8020 for the WebHDFS FileSystem URI.
* - OneFS implementation of WebHDFS. With Ambari 2.4
Ambari
Version
Kerberos
Enabled
WebHDFS*
Enabled
Ambari View
Works?
Remarks
Ambari 2.4
Y
Y
No
Does not work with Kerberos enabled because OneFS implementation of WebHDFS does not support chunked encoding.
However we believe that if we use HTTPFS in front, it should work. This has not been officially tested though
Ambari 2.4
Y
N
No
Does not work in this combination . This is default behavior of Ambari views which does not support Kerberos without WebHDFS.
Ambari 2.4
N
Y
Default Hive does
not work
(use a workaround)
Hive
Views do not work with default settings.
File View, Tez View, Zeppelin works
Workaround : Only way to use Hive
View is to use custom Hive View Cluster Configuration, be sure to use hdfs://
protocol and port 8020 for the WebHDFS FileSystem URI.
Ambari 2.4
N
N
Yes
We
believe this should work without any change
if you need to use custom Hive View Cluster Configuration,
be sure to use hdfs:// protocol and port 8020 for the WebHDFS FileSystem URI.
* - OneFS implementation of WebHDFS.
... View more
Labels:
11-04-2016
10:22 PM
With 801 FS, there is support for encryption of data-in-flight as well as data-at-rest . Data-at-rest encryption is achieved by using Self-Encrypting-Drives (SED)s . In-flight data encryption is through new features that leverages Kerberos infrastructure. To know details on how to set up encryption on OneFS 801, visit https://community.emc.com/community/products/isilon/blog/2016/11/04/enhanced-hadoop-security-with-onefs-801-and-hortonworks-hdp
... View more
12-10-2015
09:46 PM
1 Kudo
While storage space is absolutely critical as @Neeraj Sabharwal and @Ali Bajwa wrote in their post we just "discovered" that also CPU is a key point. When HWX released AMS we began to deploy Ambari and AMS on the same machine, but soon the understood that for a production environment it could be a good practice to use one VM for Ambari and another VM for AMS, so the really high impact on computation resources of AMS didn't impact Ambari (sometimes, during the aggregation phase we got 16 CPU at 90% for 10/15 minutes).
... View more
11-10-2015
01:59 AM
@Neeraj Should We keep this answer or remove it. Looks like @vsomani@hortonworks.com changed the question. I have created an article out of it. http://community.hortonworks.com/articles/3131/replacing-disk-on-datanode-hosts.html
... View more
11-06-2015
08:05 PM
Dan, partner has limitations on storage capacity (currently they have SDDs with 400 GB). And the SDDs will be JBOD and not in RAID.
... View more
10-13-2015
07:02 PM
1 Kudo
This is a bad idea and will be flagged during an architecture review. Master servers must have RAID for production deployments, and we recommend RAID 5 or, more often, 10. No amount of software HA and failover will address a situation when an OS or primary disk is lost if care wasn't taken of the data in advance. Imagine if HA was a solid solution, but client configuration for the HA mode was saved on a disk which failed, without any replica or a redundant drive. Master nodes must have RAID arrays for production.
... View more