11-30-2018 11:55 AM
1. Should we just write each log type to it's own Hive table, then do periodic joins between LogA/B/C/D into a new 'master' table partitioned by LogA timestamp?
If we do this, my primary concern is data latency. New data wouldn't be available in the master table until the joins are processed on some interval, right?
2. Are there other options possible? Any recommendations?