Member since 
    
	
		
		
		02-21-2017
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                34
            
            
                Posts
            
        
                0
            
            
                Kudos Received
            
        
                1
            
            
                Solution
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 10193 | 02-20-2017 03:51 AM | 
			
    
	
		
		
		02-28-2017
	
		
		06:17 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 1) metatool -listFSRoot  email.input  employee.input  extrnl  recharge.input  txns1.txt
aruna@aruna:~/hive_demo$ metatool -listFSRoot
Initializing HiveMetaTool..
17/02/28 14:07:46 INFO metastore.ObjectStore: ObjectStore, initialize called
17/02/28 14:07:46 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
17/02/28 14:07:46 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
17/02/28 14:07:58 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/02/28 14:08:00 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/02/28 14:08:00 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/02/28 14:08:08 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/02/28 14:08:08 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/02/28 14:08:10 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/02/28 14:08:10 INFO metastore.ObjectStore: Initialized ObjectStore
Listing FS Roots..
  2) hdfs getconf -nnRpcAddresses  aruna@aruna:~/hive_demo$ hdfs getconf -nnRpcAddresses
localhost:9000
 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-24-2017
	
		
		05:57 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I installed Hbase and start the Hbase. But when i check the runn deamons using "sudo jps" it shoes me only below.  aruna@aruna:~/hbase-1.2.4$ sudo jps
[sudo] password for aruna: 
6722 HRegionServer
6525 HQuorumPeer
6975 Jps
  1) When installing i added below to the site.xml file  <property>
<name>hbase.rootdir</name>
<value>/home/aruna/hbase-1.2.4/hbasestorage</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/aruna/hbase-1.2.4/hbasestorage/zookeeper</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>  2) I added below to the  file   export JAVA_HOME=/usr/lib/jvm/java-7-oracle
 export HBASE_MANAGES_ZK=true  I starts the Hbase using below command.  ./bin/start-hbase.sh 
  I don't know why its not running HMaster deamon? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache HBase
			
    
	
		
		
		02-23-2017
	
		
		06:21 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Sindhu below are the results  1) hive> describe formatted telecom.recharge;  hive> describe formatted telecom.recharge;
OK
# col_name                data_type               comment             
          
cell_no                 int                                         
city                    string                                      
name                    string                                      
          
# Detailed Table Information          
Database:               telecom                  
Owner:                  aruna                    
CreateTime:             Wed Feb 22 12:40:21 SGT 2017     
LastAccessTime:         UNKNOWN                  
Protect Mode:           None                     
Retention:              0                        
Location:               hdfs://localhost:9000/user/hive/warehouse/telecom.db/recharge     
Table Type:             MANAGED_TABLE            
Table Parameters:          
    COLUMN_STATS_ACCURATE    true                
    numFiles                2                   
    totalSize               93                  
    transient_lastDdlTime    1487773869          
          
# Storage Information          
SerDe Library:          org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe     
InputFormat:            org.apache.hadoop.mapred.TextInputFormat     
OutputFormat:           org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat     
Compressed:             No                       
Num Buckets:            -1                       
Bucket Columns:         []                       
Sort Columns:           []                       
Storage Desc Params:          
    field.delim             ,                   
    serialization.format    ,                   
Time taken: 0.141 seconds, Fetched: 32 row(s)
  2) hadoop fs -ls /user/hive/warehouse  aruna@aruna:~/hadoop-2.7.3/sbin$ hadoop fs -ls /user/hive/warehouse
Found 4 items
drwxrwxr-x   - aruna supergroup          0 2017-02-23 12:00 /user/hive/warehouse/myreatail.db
drwxrwxr-x   - aruna supergroup          0 2017-02-22 19:45 /user/hive/warehouse/telecom.db
drwxrwxr-x   - aruna supergroup          0 2017-02-22 12:23 /user/hive/warehouse/telecome_bkp.db
drwxrwxr-x   - aruna supergroup          0 2017-02-22 12:26 /user/hive/warehouse/telecome_bkp_extend.db
  3) hadoop fs -ls /user/hive/warehouse/telecom.db  aruna@aruna:~/hadoop-2.7.3/sbin$ hadoop fs -ls /user/hive/warehouse/telecom.db
Found 3 items
drwxrwxr-x   - aruna supergroup          0 2017-02-22 22:31 /user/hive/warehouse/telecom.db/recharge
drwxrwxr-x   - aruna supergroup          0 2017-02-22 17:48 /user/hive/warehouse/telecom.db/recharged
drwxrwxr-x   - aruna supergroup          0 2017-02-22 19:45 /user/hive/warehouse/telecom.db/recharged2
 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-23-2017
	
		
		06:16 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I created table from  hive and add some data using txt file.  when i check the below command it shows me last 4 lines are null.  hive> select * from txnrecords;  84    05-26-2011    4006771    41.0    39    Exercise & Fitness    Machine Weight Accessories    Singapore    Changi
85    05-26-2011    4006771    41.0    39    Exercise & Fitness    Machine Weight Accessories    Singapore    Changi
NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL
NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL
NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL
NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL    NULL
  1) How can i remove these null values? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hadoop
- 
						
							
		
			Apache Hive
			
    
	
		
		
		02-22-2017
	
		
		06:46 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Ok. even  there is a database(telicome) and table(recharge) already created  , but i can't fine the hive/warehouse folder.Since i didn't add them in hive-site.xml that folder may created in a default location ?So where can be that folder ?  when i am installing hive i created folders using below command and add permission.  hadoop fs -makedir -p /user/hive/warehouse //create folder 
hadoop fs -chmod  g+w /user/hive/warehouse   //add permission
hadoop fs -chmod  g+w /tmp   //add permission 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-22-2017
	
		
		06:34 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Jay SenSharma , below are the results for queries.  1) hive shell ;  hive> hive shell;
NoViableAltException(26@[])
    at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1071)
    at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:202)
    at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:396)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:0 cannot recognize input near 'hive' 'shell' '<EOF>'
  2)  use telicom;  hive> use telecom;
OK
Time taken: 0.02 seconds
  3) select * from recharge  hive> select * from recharge;
OK
9999999    ind    abc
6399    usa    mnb
9087    uk    sasadda
Time taken: 0.351 seconds, Fetched: 3 row(s) 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-22-2017
	
		
		06:05 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I don't have hive-site.xml file in conf folder. There is a hive-default.xml.template file. It have hive.metastore.warehouse.dir value as below 
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property> 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-22-2017
	
		
		05:54 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 when i am installing hive i set below in .bashrc file  export HIVE_HOME=/home/aruna/apache-hive-1.2.1-bin
export PATH=$PATH:/home/aruna/apache-hive-1.2.1-bin/bin  Create folders  hadoop fs -makedir -p /user/hive/warehouse   //create folder
hadoop fs -chmod  g+w /user/hive/warehouse   //add permission
hadoop fs -chmod  g+w /tmp   //add permission  and added hadoop path to  hive-config.sh file.
  export HADOOP_HOME=/home/aruna/hadoop-2.7.3  
  these are the things i have done.I don't  remember  that i set the "hive.metastore.warehouse.dir"  value?  
 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-22-2017
	
		
		05:34 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I installed Hive and create table.When i check the table properties using "describe extended recharge", then   i found the table was created in below location  location:hdfs://localhost:9000/user/hive/warehouse/telecom.db/recharge  I try to find the location inside user folder, but i can't find the hive/warehouse folder ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hadoop
 
        







