My Hive script is now failing intermittently with file not found exception pointing to a hdfs partition. The partition gets written a previous insert statement. When I cehck the HDFS file creation time, it shows a later time than the read query kick-off time.
This would have been understood if both queries had not been part of the same script. But in my script, write query appens first followed by the read query.
Not sure what's the reason of this and whether there is any workaround.
Is this somehow related to concurrency?
whats your Hive metastore configuring like Database ?
are you using the default derby or external database for Hive metastore .
Hive in embedded derby mode has a limitation of one active user at a time.