About shiremath

shiremath · ‎03-18-2017

hadoop fault injection framework - how to inject fault incrementally and explore all code flow area in a deterministic way (not hitting the same call flow/exception again) I gone throw the below topic, it is based on probability model and it requires lot of iterations (due to probability it hits the same call flow/exception again) to cover the complete code flow. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/FaultInjectFramework.html is there any other framework or technique using that we can inject fault incrementally and explore all code flow area in a deterministic way

shiremath · ‎09-11-2016

Finally after exploring all the metrics/statistics info from the Solr admin overview page, below information helped to find the total size: 1) document of solr are composed of indexes & files compressed 2) the size of your solr core per node is to get in solr admin page 3) From the admin page corresponding core overview section provides Num of Docs & Total Size (including index & stored files) 4) Similarly get the size info from all Solr Nodes

shiremath · ‎08-29-2016

We have logsearch with 6 node solr cluster, and we want to measure the total memory & disk space consumption by one bundle ID logs in logsearch cluster (each bundle ID maps to one cluster logs of 7 nodes) please suggest a way to measure this, based that we want to plan & manage the number of cluster logs steaming to log-search & its archival strategy effectively.

shiremath · ‎08-21-2016

Hi Ravi, The below doc provides all the HDP components port details, I think this might help you. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_Reference_Guide/content/reference_chap2.html

shiremath · ‎08-18-2016

reduce tasks "can run anywhere in the cluster" means on any of the node which has "Node Manager" installed on it

shiremath · ‎08-17-2016

This link might be helpful https://community.hortonworks.com/questions/1635/instructions-to-setup-wasb-as-storage-for-hdp-on-a.html seems the below properties need to be verified on the data node which is failing: The following is a list of configurations that should be modified to configure WASB: fs.defaultFS wasb://<containername>@<accountname>.blob.core.windows.net fs.AbstractFileSystem.wasb.impl org.apache.hadoop.fs.azure.Wasb fs.azure.account.key. . blob.core.windows.net <storage_access_key> Even though WASB will be set as the fs.defaultFS, you still need to define DataNode directories for HDFS. As the intent here is to use WASB as the primary FS, you can set the HDFS datanode directories to the temporary /mnt/resource mount point that is provided with Azure compute servers if you only plan to use HDFS for temporary job files. DataNode Directories /mnt/resource/Hadoop/hdfs/data

shiremath · ‎08-17-2016

we can do this by writing client https://cwiki.apache.org/confluence/display/Hive/HiveClient to have multiple connections and use those connections to run queries. Connection con = DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", "");

shiremath · ‎08-17-2016

Increasing the 'tickTime' value of zk helps to reduce ConnectionLoss due to delay/missing of heartbeats, basically it increases the session timeout. the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.

shiremath · ‎08-17-2016

Hi Subhash, Using hive shell or beeline instance we will be able to connect only one session at a time, but we can run multiple instances of beeline which can connect to different servers.

Online	Offline
Last Visited	‎06-29-2017 05:49 PM

Member Since	‎09-11-2015 05:16 AM
Last Visited	‎06-29-2017 05:49 PM
Posts	21
Kudos received	20

Cloudera Community

Re: How to get the total memory & disk space consu...

Re: HDP required/opened ports & as well as the def...

Re: Reduce program hosting location

Re: Hive Shell (Client) is not running muliple ser...

how to inject fault incrementally and explore all ...

Re: How to get the total memory & disk space consu...

How to get the total memory & disk space consumpti...

Re: HDP required/opened ports & as well as the def...

Re: Reduce program hosting location

Re: data node cant start because the root user

Re: Hive Shell (Client) is not running muliple ser...

Re: hiveserver 2 zookeeper discovery connection ti...

Re: Hive Shell (Client) is not running muliple ser...