About muthaiyaprabhu

muthaiyaprabhu · ‎11-16-2017

What's the command to know primary/composite key for the hive tables? Please advise

muthaiyaprabhu · ‎10-02-2017

@mqureshi Thank you for the prompt response. I am new to this space. Could you please elaborate little bit on setting up spark- env.sh. I understand this is to hand shake with hive, trying to get exact values. My current set up is: HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/usr/hdp/2.6.2.0-205/hadoop/conf} How to add hdfs,hive,core site xmls. In the java code you put above, i don't see the hive connection parameters. Do i need to replace the values on this? .config("spark.some.config.option","some-value") Please advise.

muthaiyaprabhu · ‎09-30-2017

Hi All, I have table 1 in hive say emp1, which has columns empid int, name string, dept string, salary double. In spark, using data frame i would like to read the data from hive emp 1 table, and i need to load them into another table called emp2(assume emp2 is empty and has same DDL as that of emp1). It would be great if i get java reference code. No scala or python code needed. Thanks in advance!

muthaiyaprabhu · ‎08-24-2017

Hive: I have a column name(varchar), table has now both alphabets and pure numeric values. I want to count the column name that has only numeric values

muthaiyaprabhu · ‎04-10-2017

How to map our windows file system with the linux terminal. For example, i was going through some spark videos posted by someone, he downloaded scala, file was in Downloads folder in PC(Windows). But he was accessing downloads folder from terminal. Then he moved the downloaded file to his root directory in terminal. So the question is, how to map the PC window file system in linux terminal?

muthaiyaprabhu · ‎04-07-2017

Thank you Bala!

muthaiyaprabhu · ‎04-06-2017

Thanks Namit Maheshwari, data i am bringing into test is hive data, i need to filter using some criteria, like where condition in hive query. distcp -filters to exclude some files right, not on the data level. I want to filter the hive data using some criteria in production, and then want to bring the filtered data into test region.

muthaiyaprabhu · ‎04-06-2017

I need to bring prod hive table data into test hive table. Since it's a hadoop to hadoop, i can't use sqoop, hence i can use discp to transfer data across the clusters. But i have one more scenario to be handled while bringing data, that is filtering. Say i have 10 million records in prod hive table, i want to filter using some criteria and bring it to test table. is there a way to give filter parameters in distcp command on the fly? Or any other suggestions? Thanks in advance.

Online	Offline
Last Visited	‎03-08-2018 07:15 PM

Date Registered	‎04-01-2016 06:57 PM
Last Visited	‎03-08-2018 07:15 PM
Total Messages Posted	8

Command to get primary/composite key

Re: How to read hive table1 from spark, using data...

How to read hive table1 from spark, using datafram...

numeric check in hive

Accessing windows files from linux terminal

Re: Need to bring prod hive table data into test e...

Re: Need to bring prod hive table data into test e...

Need to bring prod hive table data into test envir...