Hi @Daggers    You can write simple script using yarn rest api to fetch only completed applications [month/daywise] and copy only those applications from hdfs to local. Please check below link -   https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html   ... View more

Hi @Daggers    I think you can try this -   1. Below properties decides the path for storing yarn logs in hdfs - Belos is sample example from my cluster - yarn.nodemanager.remote-app-log-dir = /app-logs yarn.nodemanager.remote-app-log-dir-suffix = logs-ifile 2. You can do "hadoop dfs -copyToLocal" for above path which will copy all applications to local and then you can pass to splunk ?   Do you think that can work for you? Let me know if you have more questions on above. ... View more

Hi @Peruvian81  there is no such option in ambari UI   You can either check from Namenode UI --> datanode tab and see if the block counts are increasing. ... View more

Hi @Peruvian81    Once you add new datanode to cluster and if the replication starts you can see messages somehow like below in datanode logs - which signifies that new node is finalizing blocks written as well as receiving blocks from source node within replication.   DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1490)) - src: /<IPADDRESS>:45858, dest: /<IPADDRESS>:1019, bytes: 7526, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-646394656_1, offset: 0, srvID: 973c1ebc-7c88-4163-aea3-8c2e0f4f4975, blockid: BP-826310834-<IPADDRESS>-1480602348927:blk_1237811292_164146312, duration: 9130002 datanode.DataNode (DataXceiver.java:writeBlock(669)) - Receiving BP-826310834-<IPADDRESS>-1480602348927:blk_1237811295_164146315 src: /<IPADDRESS>:36930 dest: /<IPADDRESS>:1019   ... View more