1) MySQL stores user data such as app id, contact details etc.MySQL also stores ad data, such as the type of ad to be served, what's the location etc? 2) We would like to capture location based information from users' phones.
Q - what's an ideal database to store this information in? Are there any bottlenecks when receiving vast amounts of location data from millions of phones and how to mitigate it? 3) While this location data is streaming in, if an ad is available (ad details are in mysql), then the ad needs to be shown real time as the user enters a geo-fenced area.
Q. what's a real time option to spot the user in a geo-fenced area, know there is an ad associated with that location and show the ad real-time. Can this hadoop technology (probably storm) handle millions of users?
Q. How do I take data from MySQL (for example, who is the user, which ad needs to be shown and the content of the ad) within the Hadoop System when I spot a user in the Geo-fenced area? Can I keep this information synced in Hadoop in real-time as the ad is created? Which technology supports moving MySQL data to Hadoop real-time and which database would be good to store this information? What happens when ad data is changed regularly - how do we sync it in Hadoop?
4) Once an ad is served, how do we store our actions. Which database would you recommend? 5) What technologies can we use for analytics to be performed on the data from MySQL and Hadoops database? Again, how do we reduce bottlenecks using specific technologies so that we don't waste time pulling data from one DB to another local machine and process and redo everything? What's a good way to automate things from the beginning. Thanks for taking the time to advice me. We are a small team of app developers and a java developer. We are just starting to think of ads as a revenue stream and were researching good solutions to go about it. Before we hire the right staff, we wanted to know what are the skill-sets we should look for from such a team we would setup. We prefer Hortonworks platform as it's the only true integrated hadoop solution.
... View more
Hi, I belong to a company that has a couple of android developers, an ios developer, a java programmer who also manages mysql (that's me). We released 5 apps and one of the 5 has really taken off with over 3.5 million users. The other 4 apps reach a combined total of under a million users. This prompted the management to look at generating revenue from the apps via location based advertising and look at a hadoop based solution to deliver these ads. Since the last two months, our management has been in negotiations with companies to secure ad deals. User data like app id, names, contact details etc is stored in MySQL. We want to flip the switch 'on' for tracking user's location to eventually use it for advertising. We understand that 4.5 million users (and growing) sending so much of data per day to mysql would not be right. I have been tasked to research and develop a workflow for hadoop's implementation. So that we can look at the right hadoop developers to be hired. As soon as we start generating some revenues, we would be happy to engage with a full fledged hadoop solutions company such as hortonworks as well. To cut the long story short, most of our data is in mysql right now. Location based data needs to be captured and depending on triggers like a person visiting a particular store, an appropriate ad should be shot out real-time. Ad content would be in MySQL. Also this location data that comes in from all the phones, has to be stored in an appropriate database. Finally we would like to build good analytical reports which would again be a combination of mysql data and location based data. I am just trying to understand an ideal end to end workflow and the various hadoop solutions that would go into it to implement this thing from data ingestion to ads and analytics including bringing mysql data into hadoop real-time so that we can plan our strategy for the right hires better. Thanks for your support in advance.
PS: from my research, i prefer hortonworks as this is the purest open source integrated system that I found. Everybody else - for example mapr - once i get into their proprietary system am locked. I don't wish to be locked for now. So solutions supported by hortonworks ecosystem would be appreciated.
... View more