Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can I call a web service from hive?

avatar
Contributor

The scenario is: I want to write HiveQL to query from HDFS and if certain value is found which will call a webservice to have the exact meaning of that value from a different storage outside Hadoop stack. is it possible? Can a web service call/API call is possible from Hive?

1 ACCEPTED SOLUTION

avatar
Super Guru
5 REPLIES 5

avatar
Super Guru

@Marshal Tito what you describe sounds like an easy task to accomplish using NiFi to execute the HQL statement, capture a value, make the webservice call, and proceed.

If this answer is helpful please choose ACCEPT to mark the question as answered.

avatar
Contributor

@Steven Matison Thank you for your response. Yes, using Nifi it is doable. But i was interested to know "is it possible to call a web service through Hive UDF or any other better option in Hive?" Or Hive does not have the capability to make a web service call from Hive to external application? Can hive call a web service from Hive?

avatar
Super Guru

I believe the overall answer is YES... check out this link i found:

https://github.com/sendgrid/hive-udfs-1/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClien...

avatar
Contributor

@Steven Matison Thanks for the URL. Yes, it can be. At the same time i found some blogs where it is saying "Using web service call in Hive UDF is absolutely bad design" whats your opinion regarding this design issue? is there any performance issue which will be a disaster?

avatar
Super Guru

Being a fan of Nifi and less a fan of programming or code writing (in terms of todays tools), i highly recommend that route over writing any functions. However, it realy is a case of "if it works use it". Test it for your Use Case and determine if the performance is where you need it to be.