Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can I call a web service from hive?

Solved Go to solution

Can I call a web service from hive?

Explorer

The scenario is: I want to write HiveQL to query from HDFS and if certain value is found which will call a webservice to have the exact meaning of that value from a different storage outside Hadoop stack. is it possible? Can a web service call/API call is possible from Hive?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Can I call a web service from hive?

Master Collaborator

I believe the overall answer is YES... check out this link i found:

https://github.com/sendgrid/hive-udfs-1/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClien...

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

View solution in original post

5 REPLIES 5
Highlighted

Re: Can I call a web service from hive?

Master Collaborator

@Marshal Tito what you describe sounds like an easy task to accomplish using NiFi to execute the HQL statement, capture a value, make the webservice call, and proceed.

If this answer is helpful please choose ACCEPT to mark the question as answered.

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Highlighted

Re: Can I call a web service from hive?

Explorer

@Steven Matison Thank you for your response. Yes, using Nifi it is doable. But i was interested to know "is it possible to call a web service through Hive UDF or any other better option in Hive?" Or Hive does not have the capability to make a web service call from Hive to external application? Can hive call a web service from Hive?

Highlighted

Re: Can I call a web service from hive?

Master Collaborator

I believe the overall answer is YES... check out this link i found:

https://github.com/sendgrid/hive-udfs-1/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClien...

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

View solution in original post

Highlighted

Re: Can I call a web service from hive?

Explorer

@Steven Matison Thanks for the URL. Yes, it can be. At the same time i found some blogs where it is saying "Using web service call in Hive UDF is absolutely bad design" whats your opinion regarding this design issue? is there any performance issue which will be a disaster?

Highlighted

Re: Can I call a web service from hive?

Master Collaborator

Being a fan of Nifi and less a fan of programming or code writing (in terms of todays tools), i highly recommend that route over writing any functions. However, it realy is a case of "if it works use it". Test it for your Use Case and determine if the performance is where you need it to be.

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Don't have an account?
Coming from Hortonworks? Activate your account here