Member since
09-11-2015
21
Posts
20
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2252 | 09-11-2016 09:44 AM | |
1603 | 08-21-2016 06:18 AM | |
1261 | 08-18-2016 06:21 AM | |
2206 | 08-17-2016 02:13 PM |
03-18-2017
04:23 PM
1 Kudo
hadoop fault injection framework - how to inject fault incrementally and explore all code flow area in a deterministic way (not hitting the same call flow/exception again) I gone throw the below topic, it is based on probability model and it requires lot of iterations (due to probability it hits the same call flow/exception again) to cover the complete code flow. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/FaultInjectFramework.html is there any other framework or technique using that we can inject fault incrementally and explore all code flow area in a deterministic way
... View more
Labels:
- Labels:
-
Apache Hadoop
09-11-2016
09:44 AM
1 Kudo
Finally after exploring all the metrics/statistics info from the Solr admin overview page, below information helped to find the total size: 1) document of solr are composed of indexes & files compressed 2) the size of your solr core per node is to get in solr admin page 3) From the admin page corresponding core overview section provides Num of Docs & Total Size (including index & stored files) 4) Similarly get the size info from all Solr Nodes
... View more
08-29-2016
04:54 AM
2 Kudos
We have logsearch with 6 node solr cluster, and we want to measure the total memory & disk space consumption by one bundle ID logs in logsearch cluster (each bundle ID maps to one cluster logs of 7 nodes) please suggest a way to measure this, based that we want to plan & manage the number of cluster logs steaming to log-search & its archival strategy effectively.
... View more
Labels:
- Labels:
-
Apache Solr
08-21-2016
06:18 AM
4 Kudos
Hi Ravi, The below doc provides all the HDP components port details, I think this might help you.
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_Reference_Guide/content/reference_chap2.html
... View more
08-18-2016
06:21 AM
1 Kudo
reduce tasks "can run anywhere in the cluster" means on any of the node which has "Node Manager" installed on it
... View more
08-17-2016
02:49 PM
This link might be helpful https://community.hortonworks.com/questions/1635/instructions-to-setup-wasb-as-storage-for-hdp-on-a.html seems the below properties need to be verified on the data node which is failing: The following is a list of configurations that should be modified to configure WASB:
fs.defaultFS wasb://<containername>@<accountname>.blob.core.windows.net
fs.AbstractFileSystem.wasb.impl org.apache.hadoop.fs.azure.Wasb
fs.azure.account.key. . blob.core.windows.net <storage_access_key>
Even though WASB will be set as the fs.defaultFS, you still need to define DataNode directories for HDFS. As the intent here is to use WASB as the primary FS, you can set the HDFS datanode directories to the temporary /mnt/resource mount point that is provided with Azure compute servers if you only plan to use HDFS for temporary job files. DataNode Directories /mnt/resource/Hadoop/hdfs/data
... View more
08-17-2016
02:13 PM
2 Kudos
we can do this by writing client https://cwiki.apache.org/confluence/display/Hive/HiveClient to have multiple connections and use those connections to run queries. Connection con = DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", "");
... View more
08-17-2016
01:38 PM
2 Kudos
Increasing the 'tickTime' value of zk helps to reduce ConnectionLoss due to delay/missing of heartbeats, basically it increases the session timeout. the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.
... View more
08-17-2016
01:27 PM
Hi Subhash, Using hive shell or beeline instance we will be able to connect only one session at a time, but we can run multiple instances of beeline which can connect to different servers.
... View more