Member since
09-02-2016
523
Posts
89
Kudos Received
42
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2035 | 08-28-2018 02:00 AM | |
1713 | 07-31-2018 06:55 AM | |
4509 | 07-26-2018 03:02 AM | |
1949 | 07-19-2018 02:30 AM | |
5187 | 05-21-2018 03:42 AM |
07-05-2017
07:54 AM
@littlewolf Syntax: SELECT [metric expression] WHERE [predicate] To start with tsquery, you can click on any existing charts in CM, it will show you the output based on tsquery. In general, it won't allow you to alter the existing chart and it is not recommended. So you can go to CM -> Charts -> Chart builder -> follow the above syntax, it will automatically pop-up metric expression and predicate. Ex: SELECT physical_memory_used https://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_dg_tsquery.html
... View more
07-03-2017
08:42 AM
@hadoopSparkZen Try this, it will work (Note: Login to hive and make sure the table is exists) import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.sql.hive.HiveContext
val conf = new SparkConf().setAppName("Test").setMaster("yarn-client")
val sc = new SparkContext(conf)
val sqlContext = new HiveContext(sc)
import sqlContext.implicits._
val resultDF = sqlContext.sql("select * from test.emp where empid=100")
... View more
07-02-2017
07:33 PM
@Freakabhi You can consider few more points before choose one of the approach, like... 1. Number of records: approach 1 is fine for very huge records and approach 2 is ok for the less records 2. How to handle the issue if something goes wrong? : The 4th step in approach 2 deletes base table and recreate with new data. Consider you have noticed an issue with data after couple of days, how do you get deleted base_table? if you have answer then go for approach 2 3. Approach 3: You are choosing approach 1 because Hbase supports updates but hive does not support updates (I guess this is your understanding). Yes your understand was correct with old hive version. But Update is available in starting Hive 0.14 https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Update
... View more
07-01-2017
02:10 PM
@yueyang Pls try to add the below principal in your connection string, it 'may' help you ";principal=impala/hostname@REALM" hostname - hostname where you have configured impala/hive REALM - for your kerberos
... View more
07-01-2017
02:03 PM
@HMC The default uid/password is admin/admin. Below are few recommendations where you can focus to fix this issue 1. Adding required userid/pwd in hue.ini may help to fix this issue 2. Make sure you have created the hue user & granted required access on your meta database (mysql, postgresql, etc) Ex: create user 'hue' identified by 'hadoop1'; grant all on hue.* to hue; 3. if you have configured Hue with LDAP then the above credential will not work until you add admin user in your ldap 4. some configuration issue with hue
... View more
06-30-2017
07:04 AM
@hadoopSparkZen You have to declare the variable sqlContext before you import as follows.. But you are using hiveObj instead... Once you are done with the below steps, you can use sqlContext to interact with Hive val sqlContext = new HiveContext(sc) import sqlContext.implicits._
... View more
06-23-2017
11:44 AM
1 Kudo
@VincentSF oh ok got it... Go to CM -> Yarn -> Configuration -> search for "yarn.nodemanager.resource.memory-mb" it will show you the memory restriction that you set for each node (it will get configuration from yarn-site.xml) You can tweak this 'little'. Note: 1. The memory is common for all the services. so you cannot use all the memory for Yarn alone. Also don't increase the memroy for the above setting too much because it may create memory overlap issue accross the services. So may be you can set aprox 50% of total memory but again it is depends upon the memory utilization by other services. Since you have 183 nodes, the 50% is not common for all the nodes, it will change case by case 2. Also when you increase your memory on each node, it is not recommended to increase more than yarn.scheduler.maximum-allocation-mb Hope this will give some idea
... View more
06-22-2017
01:18 PM
@VincentSF Is it a typo? are you using TB by mistake instead of GB? I never heard a RAM capacity with 18 TBs I am asking this because you are repeatedly using Terabyte (TB).. am I missing something?
... View more
06-22-2017
12:57 PM
@VincentSF What do you mean by "Yarn is restricting itself at 18 tb" ? I hope you are referring to disk space... Is your problem related to disk space or memory or something else?
... View more
06-22-2017
11:25 AM
@VincentSF After you configure roles/services to each node, based on your node capacity it will allocate resouces to each node. To get your current allocation of CPU, memory etc, go to CM -> each host one by one -> Resources
... View more