Support Questions

Find answers, ask questions, and share your expertise

update hive something strange

Super Collaborator

Hi:

after update a bucket acid table like that, apache drill doesnt detect the changes I mean, the rows that changed the canal, now doesnt appear in my web...but i can see that from Hue, what happend?

update canal_acid set canal='06' where canal='02'and fechaoprcnf LIKE '2016-06-08'
1 ACCEPTED SOLUTION

Expert Contributor

is drill client configured for Acid? If you see the change in Hive, you must have set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager and hive.support.concurrency=true. Are the same properties set for the session through which Drill connects to Hive?

View solution in original post

6 REPLIES 6

if I understand you correctly,your ask is you are not able to see the updated value (canal=06) from drill query.

Super Collaborator

i have updated , and from hue with hive i can see the changes, but not with apache drill query and the dashboard y have que apache drill doesnt show this rows.

I dont think it is related to meta caching, you are querying ORC which are on HDFS not in metastore. can you try query only specific bucket where the file consisting of this record exists? the second thing you need to verify is that if the delta files are merged in base file in that partition or not. if not then try to run manual compaction and see if it helps.

Super Collaborator

its ok because i have this "hive.metastore.cache-expire-after": "write"

{
  "type": "hive",
  "enabled": true,
  "configProps": {
    "hive.metastore.uris": "thrift://host:9083",
    "javax.jdo.option.ConnectionURL": "jdbc:mysql://host/drill",
    "hive.metastore.warehouse.dir": "/tmp/drill_hive_wh",
    "fs.default.name": "hdfs://host:8020",
    "hive.metastore.sasl.enabled": "false",
    "hive.metastore.cache-ttl-seconds": "110800",
    "hive.metastore.cache-expire-after": "write"
  }
}

the problem is the rows that i have uptaded doesnt appear into the drill query, even in the other "canal", so...

any suggestions

Expert Contributor

is drill client configured for Acid? If you see the change in Hive, you must have set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager and hive.support.concurrency=true. Are the same properties set for the session through which Drill connects to Hive?