question Re: Is Horton Works Community Connection site using Hadoop for processing Data??? in Archives of Support Questions (Read Only)

Is Horton Works Community Connection site using Hadoop for processing Data???

bsuresh — Tue, 15 Dec 2015 16:40:06 GMT

Community connection doing fantastic job by providing solutions to the User's Problems.But My Question is like,Are Developers using Hadoop or related field in developing this??? If yes,what are those???

Re: Is Horton Works Community Connection site using Hadoop for processing Data???

jstraub — Tue, 15 Dec 2015 17:15:22 GMT

At the moment Hortonworks Community is using AnswerHub as a platform. I believe they are using Django as a backend framework, not sure though. As far as I know we have not connected any HDP tool, @Mark Herring has all the details.

We could definitely use some of the tools from our HDP stack to power different services of our community site. Are you working on a similar use case?

Re: Is Horton Works Community Connection site using Hadoop for processing Data???

bsuresh — Tue, 15 Dec 2015 20:42:51 GMT

@Jonas Straub

I am not working on any use case similar to community. I am working on Hadoop only. Just thinking how we can connect our HDP tools to power communities like this.

Re: Is Horton Works Community Connection site using Hadoop for processing Data???

jstraub — Tue, 15 Dec 2015 21:10:47 GMT

Here are some use case for a community like this:

Solr to power the search engine of the community
Hive to analyze access logs and user interaction
Kafka/Storm to prevent spam (kind of like FraudDetection for banks) and process posts (filter bad words, escape posts, etc.)

I also like the use case from Facebook, they are using HBase for their messages, chats, etc. (https://www.facebook.com/notes/facebook-engineering/the-underlying-technology-of-messages/454991608919/)

Re: Is Horton Works Community Connection site using Hadoop for processing Data???

mherring — Wed, 16 Dec 2015 00:45:37 GMT

As @Jonas Straub states we are using AnswerHub for our functionality. The backend database is MySQL, but here is their stack: http://docs.answerhub.com/articles/1073/answerhub-architecture.html

Re: Is Horton Works Community Connection site using Hadoop for processing Data???

jstraub — Wed, 16 Dec 2015 01:38:07 GMT

Wow cool! thanks for sharing the architecture pic 🙂