Member since
09-02-2016
523
Posts
89
Kudos Received
42
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2310 | 08-28-2018 02:00 AM | |
2162 | 07-31-2018 06:55 AM | |
5074 | 07-26-2018 03:02 AM | |
2436 | 07-19-2018 02:30 AM | |
5867 | 05-21-2018 03:42 AM |
02-10-2018
09:06 AM
@Cloudera learning I don't think any formula aviable to estimate the durtaion ( at least to my knowledge) But I tried the data node decommission few times for various reasons and this is my experience for a 40 TB node, it took between 45 mins and 60 mins approx (i hope it should be less than 5 or 10 mins for you) you can monitor this from CM -> HDFS -> WebUI -> Namenode Web UI -> a new window will open, 'Datanodes' menu -> scroll down to Decommissioning (keep refresh this page to get the progress)
... View more
02-03-2018
07:59 PM
@axie Have you created sqoop job before list? (or) looking for ad hoc sqoop command which are currently running? If you have not created sqoop job then please try this before list step1: sqoop job --create myjob \--import \--connect jdbc:mysql://localhost/db \
--username root \--table employee --m 1 step 2: sqoop job --exec myjob step 3: sqoop job --list To get help: sqoop job --help For more details: https://www.tutorialspoint.com/sqoop/sqoop_job.htm If you are running sqoop in ad hoc basis without job, it may not be listed. Ex: sqoop import \--connect jdbc:mysql://localhost/db \
--username root \--table employee --m 1
... View more
02-01-2018
01:26 PM
@NewBee22 For your first question, Go to Cloudera Manager -> HDFS -> Configuration -> search for 'Log Directory' -> change the dir path wherever applicable For your second question, Go to Cloudera Manager -> HDFS -> Configuration -> search for 'maximum log' -> Here you can change both Max log file size & Number of log files to retain. Ex: Maximum Audit Log File Size - 100 Mib (You can reduce the size) Number of Audit Logs to Retain - 10 (you can reduce how many logs to retain) Finally, you can do this for all the applicable services like yarn, etc
... View more
01-25-2018
07:26 AM
@Venkatesh_Kumar Pls try this options https://github.com/rcongiu/Hive-JSON-Serde#jsonserde---a-readwrite-serde-for-json-data
... View more
01-23-2018
11:03 AM
1 Kudo
@ludof Please run the below command, it will show you which java version being used currently echo $JAVA_HOME
... View more
01-19-2018
01:05 PM
1 Kudo
@ludof If my understanding is correct, you want to add few new parameters which are not already present in configuration. If so, in general cloudera provides this option Ex: Cloudera manager -> Hive -> Configuration -> search for 'Advanced Configuration' -> identify the correct file & click on + symbol and add the corresponding name and value if you need specific to Oozie, search for 'Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml' under Oozie -> Configuration and add name values as needed also pls make sure to restart the corresponding service, so that your changes will be activated
... View more
01-19-2018
07:53 AM
1 Kudo
@ludof If you have cloudera managed cluster then it is recomended to manage your configuration via cloudera manager. please do not edit the configuration file manually unless you are very familiar as the same copy of configuration file will be maintained in different nodes and in different locations (in the same node) for various reasons
... View more
01-19-2018
07:34 AM
1 Kudo
@ludof The table 'titles' will be in hdfs, so the user root cannot access it unless it has permission to read/write (root is not an admin for hdfs). Either you can set the permission to the root user in hdfs for the table or use the user who has already permission to read/write
... View more
01-16-2018
08:45 AM
@vratmuri It is not a problem, it says "Test is disabled" Pls chk this, Go to Cloudera Management Service -> Configuration -> search for "Embedded Database" You will find "Suppress Health Test: Embedded Database Free Space" option as disabled and this is the default option. This option is disabled by default because Embedded db is only for initial stage and it is not recommended for bigger clusters and If the embedded database is not in use (according to cloudera recommendation), this has no effect. If you need you can enable it and restart only Cloudera management service
... View more
01-12-2018
11:23 AM
@cjervis - This is the third time i'm posting the below message in the same thread as it get deleted automatically everytime when I edit the below message. In summary, I made few typos and tried to correct it by editing the message but found the entire post is missing after edit. Please take care!! @ludof You have mentioned that you are importing data from MySQL and you have already installed MySQL Connector. So sqoop suppose to use MySQL Connector but it is still using "Generic JDBC Connector". I believe the below three reasons will be the reason for your issue 1. You might have made some mistake in the MySQL Connector configuration/installation (or) 2. The MySQL Connector version that you are using is not Suitable for your MySQL Version. Ex: If you are using MySQL 5.6, it requires a driver version 5.1.26 or higher (or) 3. May be you would get the latest mysql connector version in /usr/share/java, still it may create trouble if you have old version and new version in the same folder and mysql-connector-java.jar is still pointing to the older version Ex: Consider you have below old & new version in /usr/share/java and mysql-connector-java.jar is pointing to older version mysql-connector-java-5.1.25-bin.jar -- old version mysql-connector-java-5.1.40-bin.jar -- new version then remove the link mysql-connector-java.jar and make it refer to the new version as follows rm mysql-connector-java.jar ln -s /usr/share/java/mysql-connector-java-5.1.40-bin.jar mysql-connector-java.jar mysql-connector-java.jar -> /usr/share/java/mysql-connector-java-5.1.40-bin.jar
... View more