Can I call a web service from hive?

[#1]

The scenario is: I want to write HiveQL to query from HDFS and if certain value is found which will call a webservice to have the exact meaning of that value from a different storage outside Hadoop stack. is it possible? Can a web service call/API call is possible from Hive?

[#2]

@Marshal Tito what you describe sounds like an easy task to accomplish using NiFi to execute the HQL statement, capture a value, make the webservice call, and proceed.

If this answer is helpful please choose ACCEPT to mark the question as answered.

[#3]

@Steven Matison Thank you for your response. Yes, using Nifi it is doable. But i was interested to know "is it possible to call a web service through Hive UDF or any other better option in Hive?" Or Hive does not have the capability to make a web service call from Hive to external application? Can hive call a web service from Hive?

[#4 · Accepted solution]

I believe the overall answer is YES... check out this link i found:

https://github.com/sendgrid/hive-udfs-1/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClien...

[#5]

@Steven Matison Thanks for the URL. Yes, it can be. At the same time i found some blogs where it is saying "Using web service call in Hive UDF is absolutely bad design" whats your opinion regarding this design issue? is there any performance issue which will be a disaster?

[#6]

Being a fan of Nifi and less a fan of programming or code writing (in terms of todays tools), i highly recommend that route over writing any functions. However, it realy is a case of "if it works use it". Test it for your Use Case and determine if the performance is where you need it to be.