Member since
06-09-2016
185
Posts
22
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2187 | 04-21-2017 07:57 AM | |
1365 | 04-18-2017 07:07 AM | |
3247 | 02-27-2017 05:41 AM | |
905 | 12-09-2016 11:05 AM | |
1270 | 11-24-2016 11:20 AM |
01-23-2017
06:14 AM
Hi All, to install/run HDP using Ambari, there are many ports such as 50070 needs to be open However on cloud platforms keeping these ports open creates risks Is there a way to keep them accessible from the services however blocked from outside internet. Thanks, Avijeet
... View more
Labels:
01-11-2017
03:40 PM
Thanks @mqureshi, sorry about the wild estimate numbers. Is it possible to share any sample reference from any real-life scenario, how much storage required proportionately for one large big table. The solution we are looking at needs both HBASE and SOLR as there is a need or real time read/write as well as full text search. So need to plan 2 separate clusters. Also I was told Solr is better without HDFS https://community.hortonworks.com/questions/71032/hdp-search.html#answer-71041
... View more
01-11-2017
06:07 AM
1 Kudo
Hi All, I am looking for ballpark estimation for SOLR storage requirement Let's say I have 100TB raw data (structured relational data) if I keep it in HDFS - I would need 200TB+ storage considering compression factor 2.3 and replication factor 3 (100/2.3 * 4) if I keep in HBASE - I would need around 1000TB storage considering HBASE needs 5 times storage of HDFS if I keep in SOLR - I would need around 100TB (close to the raw data size) - no hdfs Can you pls validate these assumptions, please provide if you have rough estimates, that will help to do a rough sizing. Thanks, Avijeet
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Solr
01-11-2017
05:58 AM
Thanks @Cassandra Targett, so is it right to say, there are 3 modes 1. Solr standalone mode 2. master/slave 3. SolrCloud I assume the configuration is different for each type of setup.
... View more
01-10-2017
09:54 AM
2 Kudos
Hi All, I am bit confused with Solr and Solrcloud Why would anyone use Solr and not Solrcloud, as Solrcloud gives it all the HA functionality Also as it is recommended to run Solr clusters separately from HDP clusters, does this same applies to Solrcloud also. SolrCloud works with HDFS and not Solr, if we run it in separate cluster then is there any use of using HDFS? Thanks, Avijeet
... View more
Labels:
- Labels:
-
Apache Solr
01-09-2017
07:03 AM
Hi All, I am looking for what is the recommendation around setting up HIVE and HBASE clusters, specifically is it recommended to have separate HIVE and HBASE clusters or put both in same clusters. just not from a capacity perspective but overall - what would be a overall design philosophy perspective. As many HDP components use HBASE now such as Atlas, is there a recommendation to set up a small HBASE cluster within a HIVE cluster. Thanks, Avijeet
... View more
Labels:
- Labels:
-
Apache Hive
01-04-2017
05:27 AM
Hi @dvillarreal I have kerberos enabled, I have a directory d--------- - hr1 hr 0 2016-09-21 09:49 /hr-zone in Ranger I have given access to /hr-zone to only hr1 user When I try to see the file as hdfs user I can see it [root@securityLab01 keytabs]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: hdfs-securityLab@MYDOMAIN.LOCAL Valid starting Expires Service principal
01/04/2017 05:22:05 01/04/2017 15:22:05 krbtgt/MYDOMAIN.LOCAL@MYDOMAIN.LOCAL
renew until 01/11/2017 05:22:05 [root@securityLab01 keytabs]# hadoop fs -ls /hr-zone Found 2 items -rw-r--r-- 3 hr1 hr 46878 2016-09-19 07:33 /hr-zone/sample_07.csv -rw-r--r-- 3 hr1 hr 46892 2016-09-19 07:33 /hr-zone/sample_08.csv
... View more
01-02-2017
05:46 AM
Hi, I was checking the security requirements for SEC 17a-4 and it seems one requirement is Root account should not be able to access a directory In Ranger - even if a directory is protected for a user/group - hdfs can always access it. However for HBASE I see that hdfs cannot access a table without permissions. Does it show that SEC 17a-4 cannot be achieved with Ranger-HDFS however can work with Ranger-HBASE? Thanks, Avijeet
... View more
Labels:
- Labels:
-
Apache Ranger
01-02-2017
05:42 AM
@Divakar Annapureddy I checked the document Eliminates the root account and replaces it with a compliance administrator account that executes commands with sudo This requirement doesn't seem to be supported by Ranger - hdfs can access folders protected by Ranger
... View more
12-28-2016
03:04 PM
@Divakar Annapureddy This is awesome! Thanks a ton. let me go through. I will keep the question open for any other inputs.
... View more