Member since
06-10-2015
87
Posts
5
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1951 | 04-05-2019 08:46 AM | |
10084 | 01-11-2019 10:51 AM | |
2962 | 10-22-2018 02:58 PM | |
3938 | 04-08-2018 11:14 AM | |
21349 | 06-20-2017 07:30 AM |
04-05-2019
08:46 AM
1 Kudo
Hi Esteban, Our recommendation is to place [username] pools under a parent pool, such as root.users.[username]. This allows you to control overall usage of [username] queues relative to other pools under the root. This way, if you have 3 root level pools: root.production root.adhoc root.users ...you can define appropriate weights at amonst these 3 pools. If you configured root.[username], each user pool will be added with a default share of 1. So, for example, if your initial configuration was: Weight Pool 10 root.production 10 root.adhoc ... root.[username] placement rule In the beginning, root.production and root.adhoc will each have 50% of cluster resources. When the first user runs a job, their subpool is created with a default weight of 1. Weight Pool 10 root.production 10 root.adhoc 1 root.user1 Now imagine that 50 users run a job and 50 new pools are created at the root level with weight of 1. All of the sudden you have weighted resources heavily to the user pools: Weight Pool 10 root.production 10 root.adhoc 50 root.[username] generated pools (with 50 users) So in summary, you may prefer to do something that would limit all users, no matter how many, to a fixed ratio of cluster resources: Weight Pool 10 root.production 10 root.adhoc 10 root.users -> root.users.[username] placement rule Thanks, Nick
... View more
02-01-2019
01:13 AM
Hi Nick, That would be great! Thank you so much for your time and helping me in this matter, I really appreciate it 🙂
... View more
01-21-2019
10:29 PM
Problem is due to python version you have in your node. incompatibility between the Python 3 version and the Python 2 version. The default Solr commands use the python2 version, so here we need to remove the Python global environment variables, not the python3 global environment variables. Thanks & Regards, J.Ganesh Kumar.
... View more
10-22-2018
02:58 PM
The console output is not stored anywhere besides in the CDSW Web UI itself. If the user is executing YARN applications (Spark/Pyspark) then you would have the logs from any YARN containers stored as they normally would be and you could access them via the JobHistory Server Web UI. Any Spark application run in CDSW will be in `--deploy-mode client` which means that the Spark Driver output is in the Web UI only - you have to manually copy it out of there and save in a file for future reference. Nick
... View more
04-08-2018
11:42 PM
Thanks Nick for the update.
... View more
03-15-2018
09:02 AM
I got this to work as follows (but there's a twist when dealing with Eclipse from Windows): System.setProperty("javax.net.ssl.trustStorePassword","xxxxxx"); System.setProperty("java.security.auth.login.config", "C:\\my_jaas.conf"); String zkHosts = "zookeeper1.com:2181,zookeeper2.ghp.com:2181,zookeeper3.ghp.com:2181/solr"; Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder(); SolrHttpClientBuilder kb = krbBuild.getBuilder(); HttpClientUtil.setHttpClientBuilder(kb); CloudSolrClient solrServer = new CloudSolrClient.Builder().withZkHost(zkHosts).build(); String collectionName = "test_col"; solrServer.setDefaultCollection(collectionName); solrServer.connect(); SolrQuery parameters = new SolrQuery(); parameters.set("q", "*:*"); try { solrServer.commit(); QueryResponse response = solrServer.query(parameters); for (SolrDocument solrDocument : response.getResults()) { System.out.println(" " + solrDocument.toString()); } krbBuild.close(); solrServer.close(); } catch (SolrServerException | IOException e) { e.printStackTrace(); } If you are working from a Windows Environment and Windows is generating your TGT Kerberos keys and Kerberos is hooked into AD (that is, not using MIT Kerberos), then you need to update the registry as follows (thanks to Cloudera Systems Engineer Tom Roach): On windows, to use your kerberos tickets from AD in your Java program, you need to setup this Windows registry. HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Kerberos\Parameters Value Name: AllowTgtSessionKey Value Type: REG_DWORD Value: 0x01 jaas.conf: Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=false useTicketCache=true principal="myuser@MYDOMAIN.COM"; };
... View more