question Re: Can I call a web service from hive? in Archives of Support Questions (Read Only)

Can I call a web service from hive?

tito765 — Wed, 29 Aug 2018 01:39:04 GMT

The scenario is: I want to write HiveQL to query from HDFS and if certain value is found which will call a webservice to have the exact meaning of that value from a different storage outside Hadoop stack. is it possible? Can a web service call/API call is possible from Hive?

Re: Can I call a web service from hive?

stevenmatison — Wed, 29 Aug 2018 18:24:43 GMT

@Marshal Tito what you describe sounds like an easy task to accomplish using NiFi to execute the HQL statement, capture a value, make the webservice call, and proceed.

If this answer is helpful please choose ACCEPT to mark the question as answered.

Re: Can I call a web service from hive?

tito765 — Wed, 29 Aug 2018 20:02:50 GMT

@Steven Matison Thank you for your response. Yes, using Nifi it is doable. But i was interested to know "is it possible to call a web service through Hive UDF or any other better option in Hive?" Or Hive does not have the capability to make a web service call from Hive to external application? Can hive call a web service from Hive?

Re: Can I call a web service from hive?

stevenmatison — Wed, 29 Aug 2018 20:37:25 GMT

I believe the overall answer is YES... check out this link i found:

https://github.com/sendgrid/hive-udfs-1/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClient.java

Re: Can I call a web service from hive?

tito765 — Wed, 29 Aug 2018 21:08:59 GMT

@Steven Matison Thanks for the URL. Yes, it can be. At the same time i found some blogs where it is saying "Using web service call in Hive UDF is absolutely bad design" whats your opinion regarding this design issue? is there any performance issue which will be a disaster?

Re: Can I call a web service from hive?

stevenmatison — Wed, 29 Aug 2018 21:34:16 GMT

Being a fan of Nifi and less a fan of programming or code writing (in terms of todays tools), i highly recommend that route over writing any functions. However, it realy is a case of "if it works use it". Test it for your Use Case and determine if the performance is where you need it to be.