Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

update hive something strange

avatar
Master Collaborator

Hi:

after update a bucket acid table like that, apache drill doesnt detect the changes I mean, the rows that changed the canal, now doesnt appear in my web...but i can see that from Hue, what happend?

update canal_acid set canal='06' where canal='02'and fechaoprcnf LIKE '2016-06-08'
1 ACCEPTED SOLUTION

avatar
Super Collaborator

is drill client configured for Acid? If you see the change in Hive, you must have set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager and hive.support.concurrency=true. Are the same properties set for the session through which Drill connects to Hive?

View solution in original post

6 REPLIES 6

avatar
Super Guru

if I understand you correctly,your ask is you are not able to see the updated value (canal=06) from drill query.

avatar
Master Collaborator

i have updated , and from hue with hive i can see the changes, but not with apache drill query and the dashboard y have que apache drill doesnt show this rows.

avatar
Super Guru

I dont think it is related to meta caching, you are querying ORC which are on HDFS not in metastore. can you try query only specific bucket where the file consisting of this record exists? the second thing you need to verify is that if the delta files are merged in base file in that partition or not. if not then try to run manual compaction and see if it helps.

avatar
Super Guru

avatar
Master Collaborator

its ok because i have this "hive.metastore.cache-expire-after": "write"

{
  "type": "hive",
  "enabled": true,
  "configProps": {
    "hive.metastore.uris": "thrift://host:9083",
    "javax.jdo.option.ConnectionURL": "jdbc:mysql://host/drill",
    "hive.metastore.warehouse.dir": "/tmp/drill_hive_wh",
    "fs.default.name": "hdfs://host:8020",
    "hive.metastore.sasl.enabled": "false",
    "hive.metastore.cache-ttl-seconds": "110800",
    "hive.metastore.cache-expire-after": "write"
  }
}

the problem is the rows that i have uptaded doesnt appear into the drill query, even in the other "canal", so...

any suggestions

avatar
Super Collaborator

is drill client configured for Acid? If you see the change in Hive, you must have set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager and hive.support.concurrency=true. Are the same properties set for the session through which Drill connects to Hive?