Member since 
    
	
		
		
		01-09-2019
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                29
            
            
                Posts
            
        
                3
            
            
                Kudos Received
            
        
                3
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1877 | 06-03-2020 12:34 AM | |
| 1461 | 06-03-2020 12:24 AM | |
| 1283 | 05-28-2020 02:37 AM | 
			
    
	
		
		
		01-07-2021
	
		
		07:40 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Dear @janvanbesien ,  Were you able to migrate to a supported version of JDK 11, please?  Kind regards,  Julius 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-05-2021
	
		
		05:36 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello Boenu,  Were you able to resolve this issue, please? We upgraded our NiFi deployment today and are facing the same issue (tested in Firefox and Chrome).  Kind regards,  Julius  EDIT: found the related JIRA issues, seems like there is no fix for it, just a workaround 😞  https://issues.apache.org/jira/browse/NIFI-7849  https://issues.apache.org/jira/browse/NIFI-7870 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-26-2020
	
		
		02:49 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi SherKhan,  Based on the screenshot, you have two conditions configured for the warning trigger. The first one will trigger if the free space falls below 5GB, the second one will trigger once the disk utilization is higher than 50%.  The disk belonging to master.ndctech.net has 55.53% of it's capacity used, and so the warning is triggered. If you consider this to be a false alarm, you should raise the warning threshold from 50% to 70% perhaps.  Kind regards,  Gyuszi 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-30-2020
	
		
		03:51 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @rbloughThank you for the continued support.     2) The command is being run as the hdfs user.     1) The detailed output showed that there are 603,723 blocks in total. Looking at the HDFS UI, the Datanodes report having 586,426 blocks each.     3) hdfs fsck / -openforwrite says that there are 506,549 blocks in total.     The discrepancy in block count seems to be there still. Below are the summaries of the different fsck outputs.     hdfs fsck / -files -blocks -locations -includeSnapshots  Status: HEALTHY  Number of data-nodes: 3  Number of racks: 1  Total dirs: 64389  Total symlinks: 0    Replicated Blocks:  Total size: 330079817503 B (Total open files size: 235302 B)  Total files: 625308 (Files currently being written: 129)  Total blocks (validated): 603723 (avg. block size 546740 B) (Total open file blocks (not validated): 122)  Minimally replicated blocks: 603723 (100.0 %)  Over-replicated blocks: 0 (0.0 %)  Under-replicated blocks: 0 (0.0 %)  Mis-replicated blocks: 0 (0.0 %)  Default replication factor: 3  Average block replication: 3.0  Missing blocks: 0  Corrupt blocks: 0  Missing replicas: 0 (0.0 %)  Blocks queued for replication: 0    Erasure Coded Block Groups:  Total size: 0 B  Total files: 0  Total block groups (validated): 0  Minimally erasure-coded block groups: 0  Over-erasure-coded block groups: 0  Under-erasure-coded block groups: 0  Unsatisfactory placement block groups: 0  Average block group size: 0.0  Missing block groups: 0  Corrupt block groups: 0  Missing internal blocks: 0  Blocks queued for replication: 0  FSCK ended at Wed Sep 30 12:23:06 CEST 2020 in 23305 milliseconds  hdfs fsck / -openforwrite  Status: HEALTHY  Number of data-nodes: 3  Number of racks: 1  Total dirs: 63922  Total symlinks: 0    Replicated Blocks:  Total size: 329765860325 B  Total files: 528144  Total blocks (validated): 506549 (avg. block size 651004 B)  Minimally replicated blocks: 506427 (99.975914 %)  Over-replicated blocks: 0 (0.0 %)  Under-replicated blocks: 0 (0.0 %)  Mis-replicated blocks: 0 (0.0 %)  Default replication factor: 3  Average block replication: 2.9992774  Missing blocks: 0  Corrupt blocks: 0  Missing replicas: 0 (0.0 %)  Blocks queued for replication: 0    Erasure Coded Block Groups:  Total size: 0 B  Total files: 0  Total block groups (validated): 0  Minimally erasure-coded block groups: 0  Over-erasure-coded block groups: 0  Under-erasure-coded block groups: 0  Unsatisfactory placement block groups: 0  Average block group size: 0.0  Missing block groups: 0  Corrupt block groups: 0  Missing internal blocks: 0  Blocks queued for replication: 0  FSCK ended at Wed Sep 30 12:28:06 CEST 2020 in 11227 milliseconds     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-22-2020
	
		
		11:23 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @Madhur ,     Thank you for your prompt response. Unfortunately I do not have a subscription with Cloudera, I am unable to access the Knowledge Base. So far we have managed to get by with the free version of CDH 6.3 and the help of the community on and off these forums 🙂     Kind regards,  Gyuszi Kovacs 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-22-2020
	
		
		11:17 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @rblough- thank you very much for your prompt reply.     1) I got the block numbers from the HDFS status page in Cloudera Manager. Based on your question I checked the numbers on the NameNode UI. For the DEV environment all three DataNodes are online, showing 1,613,019 blocks on each (CM shows 1,613,104). For the PROD environment the NameNodeUI shows 477,464 blocks on each of the three DataNodes.     2) Yes, all of the DataNodes are showing as online. dfsadmin -report confirms this, so does the NameNodeUI and CM. Coincidentally, the report did not include the total block count, just the number of missing or under replicated blocks - everything sits at zero.     3) The replication factor is set to 3 in both environments.     Kind regards,  Gyuszi Kovacs 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-22-2020
	
		
		07:01 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Greetings, I would like to clear up my understanding of how the block count is measured for the cluster.  A bit of background information - we started receiving high block count warnings on Cloudera Manager (6.3), this lead to some investigating and cleanup. Currently I am trying to lower the block count on our DEV environment, but I am a bit confused.  On Cloudera Manager, when I navigate to the HDFS service, and look at the Health Tests I see that there are "...1,608,301 total blocks in the cluster." However, when I run:   sudo -u hdfs hdfs fsck / -files -blocks -locations   the summary at the end states that:  Replicated Blocks:  Total size: 53194573887 B (Total open files size: 1557592 B)  Total files: 569244 (Files currently being written: 202)  Total blocks (validated): 553524 (avg. block size 96101 B) (Total open file blocks (not validated): 193)  Minimally replicated blocks: 553524 (100.0 %)  Now this is fine, from what I understand, the fsck command shows the block count without the replication factor of 3, so the numbers add up roughly. However, when I perform the same comparison on our PROD environment, I get the following. On Cloudera Manager I see that there are "... 449,966 total blocks in the cluster.", yet the fsck command returns:  Replicated Blocks:  Total size: 317645298827 B (Total open files size: 2529389 B)  Total files: 389375 (Files currently being written: 142)  Total blocks (validated): 368223 (avg. block size 862643 B) (Total open file blocks (not validated): 130)  Minimally replicated blocks: 368223 (100.0 %)  Could someone please explain the discrepancy between the numbers in this case?  Thank you, kind regards,  Gyuszi 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Cloudera Manager
- 
						
							
		
			HDFS
			
    
	
		
		
		08-11-2020
	
		
		08:31 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 More than a year later and there is still no open source version available, from what I can tell. Am I missing something? :S     @Cloudera -  know that Covid shook the world, but an updated roadmap would be appreciated. Cheers! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-03-2020
	
		
		12:34 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @Maria_pl ,  Does the send-mail processor produce any errors when the message fails to send? Also, please check the NiFi logs (nifi-app.log should be the one that might give you a hint as to what is happening in the background, but while there you might want to check the rest of the NiFi logs).  I see that you are using the latest version of NiFi. We are currently on version 1.10.0 and PutEmail works fine.   Kind regards,  Julius 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-03-2020
	
		
		12:24 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello,  This topic is described at length in the official documentation - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_ports.html   Kind regards,  Julius 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













