About rajkumar_singh

rajkumar_singh · ‎06-10-2016

@Varun Kumar Chepuri Can you please elobarate your problem statement bit more?

rajkumar_singh · ‎06-10-2016

Could you please try this http://blog.jaceklaskowski.pl/2015/07/20/real-time-data-processing-using-apache-kafka-and-spark-streaming.html

rajkumar_singh · ‎06-10-2016

if you are running mapreduce over yarn then enable yarn remote log aggregation(http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/), it will centralized all logging for you and you can perform your analysis over on aggregated logs.

rajkumar_singh · ‎06-10-2016

can you try this? ALTER TABLE dtl.ordersummery PARTITION (tran_date='2016-04-06',ordertype='SUPE') COMPACT 'minor';

rajkumar_singh · ‎06-10-2016

I am not sure about whether is any way available to get the desired output from sql server import, but I dont think it matter much if you import sql server smalldate to hive string. you can modify the hive query to get the expected output e.g unix_timestamp you can convert the string to desired timestamp to know more about it you can check following documentation. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF

rajkumar_singh · ‎06-09-2016

with spark-submit option --jar, are you passing spark-kafka-assembly jar along with kafka_2.10-*.jar from /usr/hdp/2.4.0.0-169/kafka/libs/ location.

rajkumar_singh · ‎06-09-2016

what version of hiveserver2 you are running with? it seems that you are trying to connect with old version of thrift client to the newer version of hiveserver2. can you try to check the the libthrift api version of your hive-client vs hive-server. if there is a mismatch then you are not able to connect with hive-server as it is known issue with hive. see this https://issues.apache.org/jira/browse/HIVE-6050

rajkumar_singh · ‎06-09-2016

I think its not problem with hive server thrift api rather it looks a problem with the thrift plugin of Intellij IDEA. could you try this after disabling the thrift plugin in Intellij

rajkumar_singh · ‎06-08-2016

with spark-submit you can try uploading the file to driver using -Dapplication.properties.file=<file path on location>

rajkumar_singh · ‎06-08-2016

is it single file or multiple small files?

Online	Offline
Last Visited	‎08-23-2021 03:30 PM

Member Since	‎04-25-2016 07:57 AM
Last Visited	‎08-23-2021 03:30 PM
Posts	579
Kudos received	568

Cloudera Community

Re: Why Hive Compaction is failing ?

Re: how to setup queue name for squirrel

Re: How to set the logging level of the hiveserver...

Re: Hive Tez Client Memory

Re: Resource Manager API ?

Re: How do I initialize the hive metastore?

Re: can someone point me to a good tutorial on spa...

Re: Approach to collect logs from mappers

Re: How to run manual compaction on the "nested pa...

Re: import with sqoop smalldatetime from sql serve...

Re: getting error while submitting spark job

Re: while connecting to hiveserver2 from my java c...

Re: while running a hive jdbc client from Intellij...

Re: Loading Local File to Apache Spark

Re: Loading Local File to Apache Spark