Member since
07-31-2013
1924
Posts
462
Kudos Received
311
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1980 | 07-09-2019 12:53 AM | |
| 11934 | 06-23-2019 08:37 PM | |
| 9193 | 06-18-2019 11:28 PM | |
| 10186 | 05-23-2019 08:46 PM | |
| 4604 | 05-20-2019 01:14 AM |
06-06-2016
09:22 AM
On the host where you are invoking this command, do you have proper client configs (gateway roles if you use CM) properly deployed? Please also look at our DistCp matrix to learn how/where to invoke the DIstCp command depending on your situation: http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_admin_distcp_data_cluster_migrate.html#concept_hfx_tqr_rp_section
... View more
06-06-2016
08:42 AM
2 Kudos
This is certainly interesting and I can reproduce it. I do find that if I query the GATEWAY role directly it shows up the MR2 configs. Given that MR2 is a purely GATEWAY + JOBHISTORYSERVER aspect in YARN, this would make sense, but I agree the service-level exhaustive config ought to show it too, and I'll report this internally to see what's up and if it can be changed to show the values in the future. Something such as the below would work in the meantime: curl -k -iv -X GET 'http://admin:admin@HOST:7180/api/v12/clusters/cluster/services/yarn/roles/YARN-GATEWAY-1/config?view=full' You can grab your gateway rolename from the services/yarn/roles endpoint first, in case its something custom. In my example above the rolename was "YARN-GATEWAY-1".
... View more
06-06-2016
04:00 AM
2 Kudos
HDFS does not auto-create user home directories, and there is no such feature to leverage. However, if you add users in Hue it presents an option to perform this for you in its user add wizard screen.
... View more
06-03-2016
01:42 AM
Always better to have one Q per topic, for easy searchability/context. But in any case, YARN service in CM also includes MR2 properties. The page at http://www.cloudera.com/documentation/enterprise/latest/topics/cm_props_cdh570_yarn_mr2included_.html lists all available CM configs for YARN+MR2 service. You are looking for the config in CM named "mapreduce_application_classpath" with default value '$HADOOP_MAPRED_HOME/*, $HADOOP_MAPRED_HOME/lib/*, $MR2_CLASSPATH'?
... View more
06-02-2016
05:08 PM
Apologies if the post was ambiguous. I was speaking on the API lines, in that you need to add it in XML form into that quoted field name, rather than try to add a new property name that it won't accept (your original curl). You require to do something like this (notice the name of the config being the safety valve described earlier, and that the value is in XML form with the contents of what you are trying to add in): curl -iv -X PUT -H "Content-Type:application/json" -H "Accept:application/json" -d '{"items":[{ "name": "core_site_safety_valve","value": "<property><name>hadoop.proxyuser.ztsps.users</name><value>*</value></property><property><name>hadoop.proxyuser.ztsps.groups</name><value>*</value></property>"}]}' http://admin:admin@10.1.0.1:7180/api/v12/clusters/cluster/services/hdfs/config
... View more
06-02-2016
04:49 PM
There's no direct way to filter that out via a simple statement. You can either use JDBC to fetch catalog data and iterate through the descriptors, or run iterative "DESCRIBE (FORMATTED|EXTENDED) <TABLENAME>" and grep out the Storage and Format related information. If this is a one-off, you can also explore querying the underlying HMS DB, but do not make something persistently reliant on that as the schema is subject to change between releases.
... View more
06-02-2016
12:49 AM
Any non-defined property would need to be added verbatim (in XML property tag form) to the Advanced Configuration Snippets. In your case you need to add the entry into the HDFS config fields named "Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml", i.e. CM name "core_site_safety_valve" [1]. [1] - http://www.cloudera.com/documentation/enterprise/latest/topics/cm_props_cdh570_hdfs.html
... View more
06-01-2016
04:04 PM
Most Hadoop and HBase roles at least have a configuration servlet that you can visit over at their Web Port's /conf page, for ex. http://NNHOST:50070/conf, which would show you their loaded configuration.
... View more
06-01-2016
03:12 PM
NameNode by default would wait upto 10.5 minutes before declaring a non-heartbeating DataNode as dead and processing its block list as under-replicated. P.s. Its better to open a new topic per question, it helps others searching for specific Q&A.
... View more
06-01-2016
02:31 AM
Your configuration is written for an agent named "tier1" but your agent is instead starting with name "agent". Thereby it does not pick up any configs cause the configs are for another agent. Are you using CM to manager Flume agents? You can specify per-agent names in its CM -> Flume -> Configuration page if so, or alter the config of the agent to use "agent" in place of "tier1". If you're not using CM, then please read: http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_flume_files.html#topic_12_8, i.e. edit the file /etc/default/flume-ng-agent to change the agent's name (currently "agent") to match the config "tier1", or conversely edit the configuration under /etc/flume-ng/conf/flume.conf to use the agent's name "agent" in place of "tier1".
... View more