About saranvisa

saranvisa · ‎07-28-2017

@alexe yes, you can login with pyspark (or) spark-shell and run your commands one by one to get the result. All they need is your result and code that you have used to generate the result. So it is not necessary to create .jar from your code and execute (also it will take additional time)

saranvisa · ‎07-25-2017

@ponypony No, it will not support. The below doc refers to CDH 5.11.x, also it refers to the features from relational databases or Hive are not available in Impala. In which Indexing is one among them https://www.cloudera.com/documentation/enterprise/5-11-x/topics/impala_faq.html#faq_features__faq_unsupported

saranvisa · ‎07-25-2017

@syamsri how much you have increased? so i think in your case it is not sufficient, you have to increase more and try your update & delete operations

saranvisa · ‎07-24-2017

@cllearner There could be many reasons 1. CM -> hosts -> check the hosts are in green 2. Login as root and start ntpd service if it is not started automatically service ntpd status service ntpd start chkconfig --list ntpd chkconfig ntpd on; wait for few mins and check the hosts status again. post more logs if you are still facing issue

saranvisa · ‎07-23-2017

@syamsri Go to Yarn -> Configuration -> search for "yarn.nodemanager.resource.memory-mb". If it is 1 GB by default, increase it to 2 GB Save it and restart YARN (Sometime, may need to restart HUE as well, need to chk) try again now

saranvisa · ‎07-22-2017

@SKumar2000 There could be multiple reasons for this issue 1. MySQL jdbc connector/driver is missing - you can ignore this as your first command is working 2. your sqoop command. Pls specify the target database, table in your import command as follows and try again sqoop import \ --connect "jdbc:mysql://host:3306/dbname" \ --username uid \ --password pwd \ --table table_name \ --delete-target-dir \ --hive-database hive_db_name \ --hive-table hive_tablename \ --split-by col1 \ --target-dir 'dir_path'

saranvisa · ‎07-20-2017

@ponypony This is due to java heap space issue. Pls try the below steps 1. Check the current mapreduce.map.memory.mb and mapreduce.reduce.memory.mb (there are different ways to check - you can use either Cloudera manager -> Yarn -> configuration (or) from hive/beeline CLI (or) mapred-site.xml or yarn-site.xml ) 2. Increase (1 or 2 GB) java heap space temporarly, i've already shared the details in the below link. NOTE: The below link refers to different issue but you can use this soluation for your issue too as sqoop uses MR http://community.cloudera.com/t5/Hadoop-101-Training-Quickstart/Map-and-Reduce-Error-Java-heap-space/m-p/47023#M4622 3. Try the sqoop now, if the issue fixed then work with your admin and increase it permanently

saranvisa · ‎07-19-2017

@keeblerh In general, the namenode port # is 50070. We can also customize, so You can double check that in CM -> HDFS -> Configuration. so you need to use namenode1:50070 instead of namenode1:8020 (in both source & target) also i saw that you have mentioned CDH 5, but make sure both source & target are same version (including sub versions like 5.2, 5.3, 5.7, etc). If they are not same version then you need to use different commands, you can get more details in the below link https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_admin_distcp_data_cluster_migrate.html#topic_7_2

saranvisa · ‎07-16-2017

@vikash145 when do you get this error? when you try to launch spark shell (or) when you start the history servicer (or) in a different situation? how did you start all other services? using the same user also i don't see any keyword called error in your log, it says only warning

saranvisa · ‎07-16-2017

@vikash145 According to this log, your user id doesn't have permission to start the server, login as root and try again additionally you can check the 'service <service name> status>' and try to start it also check the config by running the below command and on it chkconfig --list

Online	Offline
Last Visited	‎08-10-2019 05:12 PM

Member Since	‎09-02-2016 11:35 AM
Last Visited	‎08-10-2019 05:12 PM
Posts	523
Kudos received	96

Cloudera Community

Re: Promoting Metadata

Re: Mix on premise and cloud nodes

Re: impala-shell

Re: How do I see user usage stats by table in Impa...

Re: Replica Not FoundException

Re: How to run Spark Scala script in the CCA175 ex...

Re: hive index on impala

Re: Update and Delete are not working in Hive ?

Re: cloudera not starting after rebooting machine

Re: Update and Delete are not working in Hive ?

Re: Sqoop Import not working with Mysql table data

Re: sqoop stuck at the end

Re: UnresolvedAddressException trying to distcp be...

Re: Spark History Server failed to start

Re: Spark History Server failed to start