I stumbled onto my own answer. Appears that you have to provide a partition spec to the alter command. I figured this out after poking around in mysql to look at my hive metastore to see if that would give me a clue. These queries are what made me think that the serde information is on a partition-by-partition basis: mysql> select * from TBLS where TBL_NAME='rm'\G *************************** 1. row *************************** TBL_ID: 170 CREATE_TIME: 1463833647 DB_ID: 11 LAST_ACCESS_TIME: 0 OWNER: mpetronic RETENTION: 0 SD_ID: 227 TBL_NAME: rm TBL_TYPE: EXTERNAL_TABLE VIEW_EXPANDED_TEXT: NULL VIEW_ORIGINAL_TEXT: NULL LINK_TARGET_ID: NULL 1 row in set (0.00 sec) mysql> select * from SDS where CD_ID=170\G *************************** 1. row *************************** SD_ID: 227 CD_ID: 170 INPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat IS_COMPRESSED: IS_STOREDASSUBDIRECTORIES: LOCATION: hdfs://mpws:8020/jup1_stats/external_tables/rm NUM_BUCKETS: -1 OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat SERDE_ID: 227 *************************** 2. row *************************** SD_ID: 228 CD_ID: 170 INPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat IS_COMPRESSED: IS_STOREDASSUBDIRECTORIES: LOCATION: hdfs://mpws:8020/jup1_stats/external_tables/rm/year=2016/month=5/day=10 NUM_BUCKETS: -1 OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat SERDE_ID: 228 *************************** 3. row *************************** SD_ID: 229 CD_ID: 170 INPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat IS_COMPRESSED: IS_STOREDASSUBDIRECTORIES: LOCATION: hdfs://mpws:8020/jup1_stats/external_tables/rm/year=2016/month=5/day=11 NUM_BUCKETS: -1 OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat SERDE_ID: 229 Once I saw all those 'LOCATION' values in the SDS table, I tried the following command to alter the table and then the query worked. Interesting. Does this imply that you could have a different schema for each partition? Can anyone comment as to why the avro schema is tied to a partition and not simply to the whole table? alter table rm partition (year=2016) set serdeproperties ('avro.schema.url' = 'hdfs://mpws:8020/jup1_stats/avro/rm_1.avsc'); Since all my data is in partitions under the "year=2016" partition, I was able to just specify that one partition and it applied the change to all partitions under that. ... View more

First, spot-on by letting the ZK processes write to their own disks. As for letting the active/passive NNs write to the same physical disks as the JNs, I think you are OK with that approach. I say that as the edits are what are being written to continuously, but the fsimage files are only being read/recreated at key points such as checkpointing and startup. I probably pitched a bit of overkill in a blog I did last year on this topic of filesystems, but feel free to check it out at https://martin.atlassian.net/wiki/x/EoC3Ag if you need some help going to sleep at night. 😉 If you do check it out, you'll notice my very clear advice is that you should still make backups of the fsimage/edits files (even w/HA enabled) to avoid a potential "bunker scene" of your own. Having seen what happens first hand by losing this information (it was a configuration screw-up, not a h/w failure), I know I simply don't want to be there again. ... View more