- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
update hive something strange
- Labels:
-
Apache Hive
Created ‎06-17-2016 05:06 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi:
after update a bucket acid table like that, apache drill doesnt detect the changes I mean, the rows that changed the canal, now doesnt appear in my web...but i can see that from Hue, what happend?
update canal_acid set canal='06' where canal='02'and fechaoprcnf LIKE '2016-06-08'
Created ‎06-17-2016 06:56 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
is drill client configured for Acid? If you see the change in Hive, you must have set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager and hive.support.concurrency=true. Are the same properties set for the session through which Drill connects to Hive?
Created ‎06-17-2016 05:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
if I understand you correctly,your ask is you are not able to see the updated value (canal=06) from drill query.
Created ‎06-17-2016 05:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i have updated , and from hue with hive i can see the changes, but not with apache drill query and the dashboard y have que apache drill doesnt show this rows.
Created ‎06-17-2016 06:41 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I dont think it is related to meta caching, you are querying ORC which are on HDFS not in metastore. can you try query only specific bucket where the file consisting of this record exists? the second thing you need to verify is that if the delta files are merged in base file in that partition or not. if not then try to run manual compaction and see if it helps.
Created ‎06-17-2016 05:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is the issue cache related?
Created ‎06-17-2016 05:23 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
its ok because i have this "hive.metastore.cache-expire-after": "write"
{ "type": "hive", "enabled": true, "configProps": { "hive.metastore.uris": "thrift://host:9083", "javax.jdo.option.ConnectionURL": "jdbc:mysql://host/drill", "hive.metastore.warehouse.dir": "/tmp/drill_hive_wh", "fs.default.name": "hdfs://host:8020", "hive.metastore.sasl.enabled": "false", "hive.metastore.cache-ttl-seconds": "110800", "hive.metastore.cache-expire-after": "write" } }
the problem is the rows that i have uptaded doesnt appear into the drill query, even in the other "canal", so...
any suggestions
Created ‎06-17-2016 06:56 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
is drill client configured for Acid? If you see the change in Hive, you must have set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager and hive.support.concurrency=true. Are the same properties set for the session through which Drill connects to Hive?
