Reply
Contributor
Posts: 35
Registered: ‎02-12-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi Cloudera community!

 

Happy to join your community!

I'm a sysadmin and who love my job and like to works on new technology. So, I'm on Cloudera now!

 

For some test, we create a cluster with 3 nodes in a labs. 1 node for Cloudera Manager, 1 node for NameNode and DataNode, and the last one as DataNode only. 

It's a labs to discover the new version of Cloudera 5.5. So it's just to made some test on it, not to be in production!

We install these services: hdfs, hive, hue, impala, oozie, zookeeper, Mapreduce2 (Yarn), Sqoop1.

 

One our developers, try to import some data into Hive, but we got an error. 

 

Here the command line use by our developers:

sqoop import --connect jdbc:mysql://our.database.url/database --username user --password passwordtest --table table_product --target-dir /path/to/db --split-by product_id --hive-import --hive-overwrite --hive-table table_product

The command start successfully, we see the mapper do the job to 100 but when the job finish, we have an error:

6/02/12 15:37:57 WARN hive.TableDefWriter: Column last_updated had to be cast to a less precise type in Hive
16/02/12 15:37:57 INFO hive.HiveImport: Loading uploaded data into Hive
16/02/12 15:37:57 ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
16/02/12 15:37:57 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
    at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
    at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)

....

 

I do some search in configuration file about the HIVE_CONF_DIR and doesn't find something weird. 

 

I don't find a solution about and I block on it... So our developer can't continue his test.

I search in Cloudera Manager configuration too.

 

Have you an idea about that ? I do some search on web with no success.

 

Thanks a lot for your help!

 

 

Explorer
Posts: 13
Registered: ‎01-08-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi,
have you already created the table in hive??? If no, then you need to use
the "create-hive-table" parameter also, it will create the hive table.

one more check there is default path for hive exist on hdfs i.e.
/../hive/warehouse

New Contributor
Posts: 3
Registered: ‎02-17-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi,

I am the latest entrant to this community. I am from mainframe world and have been working in the black world for last 12 years. I am currently working for HSBC for a Core Banking project and am learning Hadoop as a part of my passion.

With all the intellectuals like you guys around, I am sure it wil be an easy ride for me whenever I am stuck.

 

Explorer
Posts: 8
Registered: ‎02-21-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi,

Let me introduce myself. I have 14 years of experience. 11 years into IT and initial 3 years into teaching in university and technical instituations. My IT experience mostly speard into java/j2ee technology stack. Since last 2 years I have created an interest in Hadoop and its ecosystem. I had been going through different topics on and off to learn this new technology. But in last 6 to 9 months I have raised a strong passion towards Hadoop ecosystem. The more I am learning I am becoming more passionate. But now the hadoop fever has gone to that extent where I am planning to quit my regular job and become a serious Hadoop developper/consultant.

 

I am leaving my current company in 2 weeks. After that I will go for CCA 175 cloudera certification. Hope that helps in kick starting my full time career in Data Science.

 

Thanks,

Satya

Posts: 521
Kudos: 57
Solutions: 29
Registered: ‎04-06-2015

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Welcome @RanjeetB and @satyanayak, It is great to have you both here. :)

 




Cy Jervis, Community Moderator - I'm not an expert but will supply relevant content from time to time. :)

Learn more about the Cloudera Community:


Terms of Service


Community Guidelines


How to use the forum

New Contributor
Posts: 5
Registered: ‎02-22-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi!

My name is Hugh Jamieson and I am the Hadoop Principle Engineer at OCLC.org, a non-profit org that serves libraries around the world.  We are heavily invested in big data to support the data processes our community needs, processes that previously used to take months to complete!  We manage library information at all levels and support sharing of resources at cloud scale.  We have a considerable investment in HBase and are true believers. 

 

Like many organizations, we are struggling with the appetite our org has for big data and the velocity at which new projects arrive.  We have many clusters arranged around geographical and organizational boundaries, and managing these little beauties is a real chore.  We are looking for ways to improve our velocity getting new features and tools into production.  We have some roll-your-own utilities for cluster management that simply do not scale.  So, we are looking at CM in the hope that it will help us speed up our deployments and make them much less complex.

 

Like other big data fans, I have no life and seldom see the sun.  JK;  I love my job.  I stumbled upon Hadoop 5 years ago and was completely hooked on its design and capabilities.  I have travelled thru many environments, from mainframe to SGI, and I can say I am a total Hadoop fan-boy.  Such a nerd.

 

In my spare time I like to evangelize Scala and Spark, Streaming, performance, reactive programming, and immutability.  My favorite color is blue.  I have my own cluster in my basement.  Yeah, I'm that sad.

 

 

 

 

Posts: 416
Topics: 51
Kudos: 75
Solutions: 48
Registered: ‎06-26-2013

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

@hughatoclc, welcome to our community!  That is quite an impressive set of skills and requirements you have, I hope you find CM suitable for the task.

 

"my favorite color is blue"  <---  LOL!  

New Contributor
Posts: 2
Registered: ‎02-23-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi,

This is Arun, and I have around 12 years of experience in IT primarily as a developer and a bit of DevOps as well.

I have worked on Cloudera, Hortonworks, MapR platforms as a DevOps engineer.

 

I am happy to join this community and leverage your knowledge to work around the technical issues that I come across and as well as answer any technical questions based on my knowledge and experience with Hadoop platforms.

Visitor
Posts: 0
Registered: ‎02-22-2016

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

Hi,

 

Myself a BI Consultant worked extensively in BI reporting, Analytics, ETL of SAP/Oracle and Tableau. I have hands on experience in Cassandra and Hadoop (RDBMS - SQOOP - Hadoop) and would like to get certified in Hadoop sooner, primarily looking for CCP Data Engineer certification.

 

During spare time i like to read books (biographies, philosophies) and also an avid photographer.

 

I am a member of many forums like SAP-SCN, Oracle, Stack Overflow, Business Objects, Cassandra etc.

Like other forums i hope i can learn and contribute freely.

 

Thanks,
Arun

New Contributor
Posts: 2
Registered: ‎11-03-2015

Re: INTRODUCE YOURSELF: Please Reply here to tell us about you!

 Glade to join this community!!!

 

I am workig as Hadoop Admin from last 2 years.I have started my hadoop carrer with apache hadoop and thanks to Cloudera which make my life easy as an Hadoop Admin.Its very great experice with Cloudera I am working for finance domain.Where security is major concern however Cloudera help us to achive the same we have configured Kerberos,SSL,Sentry and Data Encryption on HDFS in our Hadoop Cluster.

 

When I was not working I enjoy with long drive and visiting new tourist  places with my friends and I loved horse riding and swimming.

 

Announcements