Member since
06-23-2021
2
Posts
0
Kudos Received
0
Solutions
06-23-2021
06:33 PM
Hi All, We are looking for some best practice guidelines on setting limits for users in the limits.conf. We don't use the "*" for username in the limits.conf, they must be defined. CDP is the only thing on our clusters and would like to know what you suggest for each of the service users (i.e. hive/spark/hdfs/oozie, yarn, httpfs, clouder-scm, sqoop, knox, livy, flume, hue, zookeeper, mapred, druid, ranger, solr, etc, essentially a breakdown of all users that can be added as part of the CDP environment. I have looked through the documentation and haven't found a definitive answer to this question. Any help deeply appreciated, Thanks in advance,
... View more
06-23-2021
03:45 PM
Hi All, I'm very new to cloudera cdp, I have just been moved over from MapR. I'm looking to find out, i've looked quite a bit but haven't found anything thus far, if there is a cluster wide tool for doing things in the cluster. i.e. in a mapr cluster, if i wanted to get a list of all hdfs volumes with loads of other info on the volumes, i could type: "maprcli volume list" and it will give me all the below information. This is just one example, the maprcli interface into MapR has the following commands (below), i'm just wondering if there is anything similar for CDP so that I could work this into home grown scripts that could easily span clusters. Thanks full list of mapr commands: acerole acerole [validate] -role [role_name] acl acl [show|set|edit] -type [cluster|volume] -name <volume name> alarm alarm [list|raise|clear|clearall|mute|unmute|config|names|group] audit audit [data|cluster|info] blacklist blacklist [user|listusers] cluster cluster [mapreduce [get|set] | gateway | feature [enable|list] | info getminmax] config config [load|save] dashboard dashboard [info] -[version|multi_cluster_info] debugdb debugdb cdscan|bmap|checkTablet|rawScan|multiOp dialhome dialhome [metrics|enable|status|ackdial|lastdialed] disk disk [list|listall|add|remove] dump dump [containerinfo|containers|cldbmetainfo|volumeinfo|volumenodes|replicationmanagerinfo|replicationmanagerqueueinfo|rereplicationinfo|rereplicationmetrics|balancerinfo|balancermetrics|rolebalancerinfo|rolebalancermetrics|assignvouchers|fileserverworkinfo|zkinfo|supportdump|cldbnodes]snapshotsizeupdateinfo entity entity [list|info|modify|remove] fid fid [dump|stat] file file [offload|recall|tierjobabort|tierjobstatus|tierstatus] job job [kill|changepriority|linklogs|status] license license [add|remove|list|apps|showid|addcrl|listcrl] nagios nagios [generate] nfs4mgmt nfs4mgmt [add-export|remove-export|list-exports] nfsmgmt nfsmgmt [refreshexports|refreshgidcache] node node [list|move|modify|allow-into-cluster|services|topo|remove|heatmap|listcldbs|listzookeepers|cldbmaster|canremovesp|maintenance|metrics|show|listcldbzks|refill] rlimit rlimit [set|get] -resource <resource> schedule schedule [create|modify|list|remove] security security [genkey|genticket|getmapruserticket] service service [list|info] setloglevel setloglevel [cldb|fileserver|nfs|jobtracker|tasktracker|hbmaster|hbregionserver] stream stream [create|edit|info|delete|topic|cursor|replica|upstream|listrecent] table table [create|edit|delete|listrecent|info|cf|region|replica|upstream|index|changelog] task task [killattempt|failattempt] tier tier [create|modify|remove|info|list|rule] trace trace [dump|setmode|setlevel|resize|reset|info|print|setslowopsmillis] -host x.x.x.x -port <port> urls getting service webserver link virtualip virtualip [add|remove|edit|move|list] volume volume [create|modify|remove|mount|unmount|list|snapshot|mirror|dump|rename|upgradeformat|move|link|info|showmounts|fixmountpath|offload|recall|compact|tierjobabort|tierjobstatus|tierstats] maprcli volume list data retrieved: AdvisoryQuotaExceededAlarm AlmostFullTopologyAlarm BecomeMasterStuckAlarm CannotMirrorAlarm CompactionFailureAlarm ContainersNonLocalAlarm CriticallyDegradedEcStripesAlarm DataUnavailableAlarm DataUnderReplicatedAlarm DegradedEcStripesAlarm EcDataUnavailableAlarm FullTopologyAlarm InodesExceededAlarm LargeRowWarning MirrorFailureAlarm NoNodesInTopologyAlarm OffloadRecallFailureAlarm QuotaExceededAlarm ReplTypeConversionInProgress SnapshotFailureAlarm TableIndexEncodingErrorAlarm TableIndexErrorAlarm TableIndexLagHighAlarm TableReplicationAsyncAlarm TableReplicationErrorAlarm TableReplicationLagHighAlarm VOLUME_ALARM_ADVISORY_QUOTA_EXCEEDED VOLUME_ALARM_BECOME_MASTER_STUCK VOLUME_ALARM_CANNOT_MIRROR VOLUME_ALARM_COMPACTION_FAILURE VOLUME_ALARM_CRITICALLY_DEGRADED_EC_STRIPES VOLUME_ALARM_DATA_CONTAINERS_NONLOCAL VOLUME_ALARM_DATA_UNAVAILABLE VOLUME_ALARM_DATA_UNDER_REPLICATED VOLUME_ALARM_DEGRADED_EC_STRIPES VOLUME_ALARM_EC_DATA_UNAVAILABLE VOLUME_ALARM_INODES_EXCEEDED VOLUME_ALARM_MIRROR_FAILURE VOLUME_ALARM_NO_NODES_IN_TOPOLOGY VOLUME_ALARM_OFFLOAD_RECALL_FAILURE VOLUME_ALARM_QUOTA_EXCEEDED VOLUME_ALARM_SNAPSHOT_FAILURE VOLUME_ALARM_TOPOLOGY_ALMOST_FULL VOLUME_ALARM_TOPOLOGY_FULL accesstime actualreplication advisoryquota aename aetype allowGrant allowReadForExecute auditVolume audited coalesceInterval containerAllocationFactor creationTime creator creatorcontainerid creatorvolumeuuid criticalReReplTimeOutSec dareEnabled data-size-mirrored-mb data-size-to-mirror-mb dbindexlagsecalarmthresh dbrepllagsecalarmthresh disableddataauditoperations enableddataauditoperations enforceMinReplicationForIO fixCreatorId forceAudit lastSuccessfulMirrorTime limitspread localpath logicalUsed maxinodesalarmthreshold maxnssizembalarmthreshold metricsEnabled minreplicas mirror-percent-complete mirrorDataSrcCluster mirrorDataSrcVolume mirrorDataSrcVolumeId mirrorId mirrorSrcCluster mirrorSrcVolume mirrorSrcVolumeId mirrorscheduleid mirrorstatus mirrorthrottle mirrortype mountdir mounted nameContainerDataThresholdMB nameContainerSizeMB needsGfsck nextMirrorId nsMinReplicas nsNumReplicas numactivecgcontainers numcontainers numreplicas partlyOutOfTopology quota rackpath reReplTimeOutSec readonly replicatedlogicalused replicatedtotalused replicationtype scheduleid schedulename skipWireSecuityForTierInternalOps snapshotcount snapshotused tierenable totalused used volumeid volumename volumetype wireSecurity
... View more
Labels:
- Labels:
-
Cloudera Data Platform (CDP)