Member since 
    
	
		
		
		09-02-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                523
            
            
                Posts
            
        
                89
            
            
                Kudos Received
            
        
                42
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 2723 | 08-28-2018 02:00 AM | |
| 2695 | 07-31-2018 06:55 AM | |
| 5677 | 07-26-2018 03:02 AM | |
| 2978 | 07-19-2018 02:30 AM | |
| 6459 | 05-21-2018 03:42 AM | 
			
    
	
		
		
		02-10-2018
	
		
		09:06 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Cloudera learning     I don't think any formula aviable to estimate the durtaion ( at least to my knowledge)     But I tried the data node decommission few times for various reasons and this is my experience     for a 40 TB node, it took between 45 mins and 60 mins approx (i hope it should be less than 5 or 10 mins for you)     you can monitor this from CM -> HDFS -> WebUI -> Namenode Web UI -> a new window will open, 'Datanodes' menu -> scroll down to Decommissioning (keep refresh this page to get the progress) 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-03-2018
	
		
		07:59 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @axie     Have you created sqoop job before list? (or) looking for ad hoc sqoop command which are currently running?     If you have not created sqoop job then please try this before list  step1:   sqoop job --create myjob \--import \--connect jdbc:mysql://localhost/db \
--username root \--table employee --m 1    step 2:   sqoop job --exec myjob    step 3:  sqoop job --list    To get help: sqoop job --help    For more details: https://www.tutorialspoint.com/sqoop/sqoop_job.htm     If you are running sqoop in ad hoc basis without job, it may not be listed. Ex:  sqoop import \--connect jdbc:mysql://localhost/db \
--username root \--table employee --m 1    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-01-2018
	
		
		01:26 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @NewBee22     For your first question, Go to Cloudera Manager -> HDFS -> Configuration -> search for 'Log Directory' -> change the dir path wherever applicable     For your second question, Go to Cloudera Manager -> HDFS -> Configuration -> search for 'maximum log' -> Here you can change both Max log file size & Number of log files to retain.  Ex:  Maximum Audit Log File Size - 100 Mib (You can reduce the size)  Number of Audit Logs to Retain - 10 (you can reduce how many logs to retain)        Finally, you can do this for all the applicable services like yarn, etc 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-25-2018
	
		
		07:26 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Venkatesh_Kumar     Pls try this options  https://github.com/rcongiu/Hive-JSON-Serde#jsonserde---a-readwrite-serde-for-json-data    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-23-2018
	
		
		11:03 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @ludof     Please run the below command, it will show you which java version being used currently     echo $JAVA_HOME       
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-19-2018
	
		
		01:05 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @ludof     If my understanding is correct, you want to add few new parameters which are not already present in configuration. If so, in general cloudera provides this option     Ex: Cloudera manager -> Hive -> Configuration -> search for 'Advanced Configuration' -> identify the correct file & click on + symbol and add the corresponding name and value      if you need specific to Oozie, search for 'Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml' under Oozie -> Configuration and add name values as needed     also pls make sure to restart the corresponding service, so that your changes will be activated 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-19-2018
	
		
		07:53 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @ludof     If you have cloudera managed cluster then it is recomended to manage your configuration via cloudera manager.      please do not edit the configuration file manually unless you are very familiar as the same copy of configuration file will be maintained in different nodes and in different locations (in the same node) for various reasons  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-19-2018
	
		
		07:34 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @ludof     The table 'titles' will be in hdfs, so the user root cannot access it unless it has permission to read/write (root is not an admin for hdfs). Either you can set the permission to the root user in hdfs for the table or use the user who has already permission to read/write 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-16-2018
	
		
		08:45 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @vratmuri     It is not a problem, it says "Test is disabled"      Pls chk this,     Go to Cloudera Management Service -> Configuration -> search for "Embedded Database"    You will find "Suppress Health Test: Embedded Database Free Space" option as disabled and this is the default option.     This option is disabled by default because Embedded db is only for initial stage and it is not recommended for bigger clusters and If the embedded database is not in use (according to cloudera recommendation), this has no effect.     If you need you can enable it and restart only Cloudera management service 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-12-2018
	
		
		11:23 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							    @cjervis  - This is the third time i'm posting the below message in the same thread as it get deleted automatically everytime when I edit the below message. In summary, I made few typos and tried to correct it by editing the message but found the entire post is missing after edit. Please take care!!     @ludof     You have mentioned that you are importing data from MySQL and you have already installed MySQL Connector. So sqoop suppose to use MySQL Connector but it is still using "Generic JDBC Connector". I believe the below three reasons will be the reason for your issue     1. You might have made some mistake in the MySQL Connector configuration/installation    (or)     2. The MySQL Connector version that you are using is not Suitable for your MySQL Version.  Ex: If you are using MySQL 5.6, it requires a driver version 5.1.26 or higher    (or)     3. May be you would get the latest mysql connector version in /usr/share/java, still it may create trouble if you have old version and new version in the same folder and mysql-connector-java.jar is still pointing to the older version  Ex: Consider you have below old & new version in /usr/share/java and mysql-connector-java.jar is pointing to older version  mysql-connector-java-5.1.25-bin.jar -- old version  mysql-connector-java-5.1.40-bin.jar -- new version    then remove the link mysql-connector-java.jar and make it refer to the new version as follows  rm mysql-connector-java.jar  ln -s /usr/share/java/mysql-connector-java-5.1.40-bin.jar mysql-connector-java.jar  mysql-connector-java.jar -> /usr/share/java/mysql-connector-java-5.1.40-bin.jar    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













