About fnushelly

fnushelly · ‎03-02-2016

Hi Harsh, Thanks for respond, I have no problem in creating the table when backed table file is already in avro type. But once I have created the table and loaded the data, it is done. After that if I have another text file, which I want to load in the same table, how will I do that? I was trying to explicitly convert that text file into avro file (using avro tools) the files contents in my previous question is that one. so here it was not converting the text file into proper avro file which could be loaded by the hive.

fnushelly · ‎02-25-2016

Hi All, i have to convert the text file into avro file following are the contents of my files emp_avro.avsc { "type" : "record", "name" : "emp_avro", "namespace" : "default", "fields" : [ { "name" : "id", "type" : [ "null", "int" ], "doc" : "\u0000\u0000\u0000", "default" : null }, { "name" : "name", "type" : [ "null", "string" ], "default" : null }, { "name" : "sex", "type" : [ "null", "string" ], "default" : null }, { "name" : "state", "type" : [ "null", "string" ], "default" : null } ] } emp_info.txt 1,amit,male,TX 2,manisg,male,CA 3,hsan,male,AZ 4,shnk,male,FL 5,manisha,female,TX 6,shana,female,AZ 7,sahana,female,CA 8,seema,female,FL i am using the following command to convert the text file into avro file java -jar avro-tools.jar fromtext /home/cloudera/Downloads/emp_avro.avsc /home/cloudera/Downloads/emp_info.txt > /home/cloudera/Downloads/emp_avro.avro and i got this file emp_avro.avro Writing line = 1 Writing line = 20 Writing line = 22 Writing line = 26 Writing line = 16 Writing line = 18 Writing line = 31 Writing line = 33 Writing line = 20 Writing line = 6 Writing line = 20 Writing line = 34 Writing line = 20 Writing line = 6 Writing line = 19 Writing line = 34 Writing line = 20 Writing line = 6 Writing line = 21 Writing line = 34 Writing line = 20 Writing line = 5 Writing line = 1 so this file can't be accesible through hive tables; any assistance will be helpful

fnushelly · ‎01-26-2016

to connect to hive metastore you need to copy the hive-site.xml file into spark/conf directory. After that spark will be able to connect to hive metastore. so run the following ommand after log in as root user cp /usr/lib/hive/conf/hive-site.xml /usr/lib/spark/conf/

fnushelly · ‎01-25-2016

Hi, I am trying to access the already existing table in hive by using spark shell But when I run the instructions, error comes "table not found". e.g. in hive table is existing name as "department" in default database. i start the spark-shell and execute the following set of instructions. import org.apache.spark.sql.hive.HiveContext val sqlContext = new HiveContext(sc) val depts = sqlContext.sql("select * from departments") depts.collecat().foreach(println) but it coudn't find the table. Now My questions are: 1. As I know ny using HiveContext spark can access the hive metastore. But it is not doing here, so is there any configuration setup required? I am using Cloudera quickstart VM 5..5 2. As an alternative I created the table on spark-shell , load a data file and then performed some queries and then exit the spark shell. 3. even if I create the table using spark-shell, it is not anywhere existing when I am trying to access it using hive editor. 4. when i again start the spark-shell , then earlier table i created, was no longer existing, so exactly where this table and metadata is stored and all.... I am very much confused, because accroding to theortical concepts, it should go under the hive metastore. Thanks & Regards

fnushelly · ‎01-16-2016

Hi, CCA 175 is the replacement of CCDH 410. because CCDH410 no more available. And in this exam now there will be only handson . No more objective questions

fnushelly · ‎01-16-2016

Hi, I am preparinf for CCA 175 exam. currently I am using CDH 5.5 (cloudera Qickstart VM). Will in the exam I will get same enviornment?? Is it ok that I keep practicing on this enviornment. Currently I am not using the cloudera manager or express sevices etc.. 2. in CDH 5.5, is there any IDE availabe for python or scala, or I have to use the normal linux edior for coding, like gedit etc. 3. Is there any way to update the versions e.g. on CDH 5.5, the python is 2.6 but I want to use the Python 2.7 or latest version. Thanks Shelly

fnushelly · ‎01-08-2016

Thanks Judd, I was just seeking for some sample questions, And beside this I want to confirm that now there will be mo more objective questions also. So, previous CCD 410 practice test papers will no longer be useful. Thanks for support Shelly

fnushelly · ‎01-08-2016

Hi, 1. Is there any sample questions available , which can give an idea about exam patter and its depth. 2. Does cloudera provides the paid test papers for CCA 175 also?? Thanks Shelly

Online	Offline
Last Visited	‎03-19-2019 02:50 PM

Member Since	‎01-08-2016 12:26 PM
Last Visited	‎03-19-2019 02:50 PM
Posts	13
Kudos received	3

Cloudera Community

Re: convert text file into avro file..

convert text file into avro file..

Re: how to access the hive tables from spark-shell

how to access the hive tables from spark-shell

Re: CCA 175

Regarding CDH 5.5

Re: CCA 175

CCA 175