Support Questions

Find answers, ask questions, and share your expertise

Hive/Druid all Queries yield null columns

HI all, been trying to get Hive/Druid/Kafka working (noob to both hive/druid)..'getting close I think' I have gotten to the point of being able load data via kafka->druid I have been able to create 'external druid' tables in hive, and even 'query', such that I can get accurate row counts. but when ever I actually query any columns..I get null back for all columns (except the __time).

This entire stack was setup via: HDP 3, onto 3 boxes.

CREATE EXTERNAL TABLE teststuff5 STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' TBLPROPERTIES ("druid.datasource" = "teststuff5"); 

Queries are coming back like (we see we have a non null time value..and for some reason one other column is non null):

|     teststuff5.__time      | teststuff5.accepteddate  | teststuff5.accountid  | teststuff5.accountname  | teststuff5.apiversion  | teststuff5.carrierid  | teststuff5.carriername  | teststuff5.companyid  | teststuff5.contentencoding  | teststuff5.countrycode  | teststuff5.countryname  | teststuff5.delivereddate  | teststuff5.destinationaddress  | teststuff5.internalmessageid  | teststuff5.liveoperatorlookup  | teststuff5.messageid  | teststuff5.messageoriginator  | teststuff5.messageoriginatorton  | teststuff5.messagestatus  | teststuff5.messagetype  | teststuff5.mtscore  | teststuff5.parentmessageid  | teststuff5.phonenumber  | teststuff5.productid  | teststuff5.productiddescription  | teststuff5.productname  | teststuff5.productsubtype  | teststuff5.programid  | teststuff5.remoteipaddress  | teststuff5.remoteresponsecode  | teststuff5.responsecode  | teststuff5.responsecodedescription  | teststuff5.sourceaddress  |        teststuff5.subaccount        | teststuff5.updateddate  | teststuff5.useragent  | teststuff5.userdataheader  | teststuff5.userdefined1  | teststuff5.userdefined2  |
+----------------------------+--------------------------+-----------------------+-------------------------+------------------------+-----------------------+-------------------------+-----------------------+-----------------------------+-------------------------+-------------------------+---------------------------+--------------------------------+-------------------------------+--------------------------------+-----------------------+-------------------------------+----------------------------------+---------------------------+-------------------------+---------------------+-----------------------------+-------------------------+-----------------------+----------------------------------+-------------------------+----------------------------+-----------------------+-----------------------------+--------------------------------+--------------------------+-------------------------------------+---------------------------+-------------------------------------+-------------------------+-----------------------+----------------------------+--------------------------+--------------------------+
| 2019-02-05 18:53:00.0 UTC  | NULL                     | NULL                  | NULL                    | NULL                   | NULL                  | NULL                    | NULL                  | NULL                        | NULL                    | NULL                    | NULL                      | NULL                           | NULL                          | NULL                           | NULL                  | NULL                          | NULL                             | NULL                      | NULL                    | NULL                | NULL                        | NULL                    | NULL                  | NULL                             | NULL                    | NULL                       | NULL                  | NULL                        | NULL                           | NULL                     | NULL                                | NULL                      | ESA_SMS_DEVICE_NTERNAL  | NULL                    | NULL                  | NULL                       | NULL                     | NULL                     |
+----------------------------+--------------------------+-----------------------+-------------------------+------------------------+-----------------------+-------------------------+-----------------------+-----------------------------+-------------------------+-------------------------+---------------------------+--------------------------------+-------------------------------+--------------------------------+-----------------------+-------------------------------+----------------------------------+---------------------------+-------------------------+---------------------+-----------------------------+-------------------------+-----------------------+----------------------------------+-------------------------+----------------------------+-----------------------+-----------------------------+--------------------------------+--------------------------+-------------------------------------+---------------------------+-------------------------------------+-------------------------+-----------------------+----------------------------+--------------------------+--------------------------+

note: The same query on druid..shows all values..

Calling: : jdbc:hive2://wdc-tst-bdrd-001.openmarket.c> describe formatted teststuff5;

Gives:

INFO  : Compiling command(queryId=hive_20190207234008_c536632e-547f-4843-96bc-e92b8577799d): describe formatted teststuff5
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col_name, type:string, comment:from deserializer), FieldSchema(name:data_type, type:string, comment:from deserializer), FieldSchema(name:comment, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=hive_20190207234008_c536632e-547f-4843-96bc-e92b8577799d); Time taken: 0.062 seconds
INFO  : Executing command(queryId=hive_20190207234008_c536632e-547f-4843-96bc-e92b8577799d): describe formatted teststuff5
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20190207234008_c536632e-547f-4843-96bc-e92b8577799d); Time taken: 0.142 seconds
INFO  : OK
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
|           col_name            |                     data_type                      |                      comment                       |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| # col_name                    | data_type                                          | comment                                            |
| __time                        | timestamp with local time zone                     | from deserializer                                  |
| accepteddate                  | string                                             | from deserializer                                  |
| accountid                     | string                                             | from deserializer                                  |
| accountname                   | string                                             | from deserializer                                  |
| apiversion                    | string                                             | from deserializer                                  |
| carrierid                     | string                                             | from deserializer                                  |
| carriername                   | string                                             | from deserializer                                  |
| companyid                     | string                                             | from deserializer                                  |
| contentencoding               | string                                             | from deserializer                                  |
| countrycode                   | string                                             | from deserializer                                  |
| countryname                   | string                                             | from deserializer                                  |
| delivereddate                 | string                                             | from deserializer                                  |
| destinationaddress            | string                                             | from deserializer                                  |
| internalmessageid             | string                                             | from deserializer                                  |
| liveoperatorlookup            | string                                             | from deserializer                                  |
| messageid                     | string                                             | from deserializer                                  |
| messageoriginator             | string                                             | from deserializer                                  |
| messageoriginatorton          | string                                             | from deserializer                                  |
| messagestatus                 | string                                             | from deserializer                                  |
| messagetype                   | string                                             | from deserializer                                  |
| mtscore                       | string                                             | from deserializer                                  |
| parentmessageid               | string                                             | from deserializer                                  |
| phonenumber                   | string                                             | from deserializer                                  |
| productid                     | string                                             | from deserializer                                  |
| productiddescription          | string                                             | from deserializer                                  |
| productname                   | string                                             | from deserializer                                  |
| productsubtype                | string                                             | from deserializer                                  |
| programid                     | string                                             | from deserializer                                  |
| remoteipaddress               | string                                             | from deserializer                                  |
| remoteresponsecode            | string                                             | from deserializer                                  |
| responsecode                  | string                                             | from deserializer                                  |
| responsecodedescription       | string                                             | from deserializer                                  |
| sourceaddress                 | string                                             | from deserializer                                  |
| subaccount                    | string                                             | from deserializer                                  |
| updateddate                   | string                                             | from deserializer                                  |
| useragent                     | string                                             | from deserializer                                  |
| userdataheader                | string                                             | from deserializer                                  |
| userdefined1                  | string                                             | from deserializer                                  |
| userdefined2                  | string                                             | from deserializer                                  |
|                               | NULL                                               | NULL                                               |
| # Detailed Table Information  | NULL                                               | NULL                                               |
| Database:                     | default                                            | NULL                                               |
| OwnerType:                    | USER                                               | NULL                                               |
| Owner:                        | admin                                              | NULL                                               |
| CreateTime:                   | Thu Feb 07 23:06:21 UTC 2019                       | NULL                                               |
| LastAccessTime:               | UNKNOWN                                            | NULL                                               |
| Retention:                    | 0                                                  | NULL                                               |
| Location:                     | hdfs://wdc-tst-bdrd-001.openmarket.com:8020/warehouse/tablespace/external/hive/teststuff5 | NULL                                               |
| Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
| Table Parameters:             | NULL                                               | NULL                                               |
|                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"__time\":\"true\",\"accepteddate\":\"true\",\"accountid\":\"true\",\"accountname\":\"true\",\"apiversion\":\"true\",\"carrierid\":\"true\",\"carriername\":\"true\",\"companyid\":\"true\",\"contentencoding\":\"true\",\"countrycode\":\"true\",\"countryname\":\"true\",\"delivereddate\":\"true\",\"destinationaddress\":\"true\",\"internalmessageid\":\"true\",\"liveoperatorlookup\":\"true\",\"messageid\":\"true\",\"messageoriginator\":\"true\",\"messageoriginatorton\":\"true\",\"messagestatus\":\"true\",\"messagetype\":\"true\",\"mtscore\":\"true\",\"parentmessageid\":\"true\",\"phonenumber\":\"true\",\"productid\":\"true\",\"productiddescription\":\"true\",\"productname\":\"true\",\"productsubtype\":\"true\",\"programid\":\"true\",\"remoteipaddress\":\"true\",\"remoteresponsecode\":\"true\",\"responsecode\":\"true\",\"responsecodedescription\":\"true\",\"sourceaddress\":\"true\",\"subaccount\":\"true\",\"updateddate\":\"true\",\"useragent\":\"true\",\"userdataheader\":\"true\",\"userdefined1\":\"true\",\"userdefined2\":\"true\"}} |
|                               | EXTERNAL                                           | TRUE                                               |
|                               | bucketing_version                                  | 2                                                  |
|                               | druid.datasource                                   | teststuff5                                         |
|                               | numFiles                                           | 0                                                  |
|                               | numRows                                            | 0                                                  |
|                               | rawDataSize                                        | 0                                                  |
|                               | storage_handler                                    | org.apache.hadoop.hive.druid.DruidStorageHandler   |
|                               | totalSize                                          | 0                                                  |
|                               | transient_lastDdlTime                              | 1549580781                                         |
|                               | NULL                                               | NULL                                               |
| # Storage Information         | NULL                                               | NULL                                               |
| SerDe Library:                | org.apache.hadoop.hive.druid.serde.DruidSerDe      | NULL                                               |
| InputFormat:                  | null                                               | NULL                                               |
| OutputFormat:                 | null                                               | NULL                                               |
| Compressed:                   | No                                                 | NULL                                               |
| Num Buckets:                  | -1                                                 | NULL                                               |
| Bucket Columns:               | []                                                 | NULL                                               |
| Sort Columns:                 | []                                                 | NULL                                               |
| Storage Desc Params:          | NULL                                               | NULL                                               |
|                               | serialization.format                               | 1                                                  |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+

hiveserver2 logging sample

2019-02-08T18:45:21,640 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:21,640 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:21,642 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: operation.OperationManager (:()) - Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=3b771b6b-7ff7-49dc-b212-dfc6b77211dc]<br>2019-02-08T18:45:21,646 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ql.Driver (:()) - Compiling command(queryId=hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d): select * from teststuff5 limit 1<br>2019-02-08T18:45:21,706 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: lockmgr.DbTxnManager (:()) - Opened txnid:185<br>2019-02-08T18:45:21,707 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Starting Semantic Analysis<br>2019-02-08T18:45:21,708 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Completed phase 1 of Semantic Analysis<br>2019-02-08T18:45:21,708 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Get metadata for source tables<br>2019-02-08T18:45:21,733 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Get metadata for subqueries<br>2019-02-08T18:45:21,733 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Get metadata for destination tables<br>2019-02-08T18:45:21,745 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ql.Context (:()) - New scratch dir is hdfs://wdc-tst-bdrd-001:8020/tmp/hive/admin/8dea5cfb-f504-4237-a3b4-fb19d739b792/hive_2019-02-08_18-45-21_696_7674740746834465900-2<br>2019-02-08T18:45:21,745 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Completed getting MetaData in Semantic Analysis<br>2019-02-08T18:45:21,896 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Get metadata for source tables<br>2019-02-08T18:45:21,907 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Get metadata for subqueries<br>2019-02-08T18:45:21,907 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Get metadata for destination tables<br>2019-02-08T18:45:21,908 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ql.Context (:()) - New scratch dir is hdfs://wdc-tst-bdrd-001:8020/tmp/hive/admin/8dea5cfb-f504-4237-a3b4-fb19d739b792/hive_2019-02-08_18-45-21_696_7674740746834465900-2<br>2019-02-08T18:45:21,912 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: common.FileUtils (FileUtils.java:mkdir(580)) - Creating directory if it doesn't exist: hdfs://wdc-tst-bdrd-001:8020/tmp/hive/admin/8dea5cfb-f504-4237-a3b4-fb19d739b792/hive_2019-02-08_18-45-21_696_7674740746834465900-2/-mr-10001/.hive-staging_hive_2019-02-08_18-45-21_696_7674740746834465900-2<br>2019-02-08T18:45:21,917 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - CBO Succeeded; optimized logical plan.<br>2019-02-08T18:45:21,918 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ppd.OpProcFactory (:()) - Processing for FS(2)<br>2019-02-08T18:45:21,918 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ppd.OpProcFactory (:()) - Processing for SEL(1)<br>2019-02-08T18:45:21,918 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ppd.OpProcFactory (:()) - Processing for TS(0)<br>2019-02-08T18:45:21,921 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: parse.CalcitePlanner (:()) - Completed plan generation<br>2019-02-08T18:45:21,921 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ql.Driver (:()) - Semantic Analysis Completed (retrial = false)<br>2019-02-08T18:45:21,921 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ql.Driver (:()) - Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:teststuff5.__time, type:timestamp with local time zone, comment:null), FieldSchema(name:teststuff5.accepteddate, type:string, comment:null), FieldSchema(name:teststuff5.accountid, type:string, comment:null), FieldSchema(name:teststuff5.accountname, type:string, comment:null), FieldSchema(name:teststuff5.apiversion, type:string, comment:null), FieldSchema(name:teststuff5.carrierid, type:string, comment:null), FieldSchema(name:teststuff5.carriername, type:string, comment:null), FieldSchema(name:teststuff5.companyid, type:string, comment:null), FieldSchema(name:teststuff5.contentencoding, type:string, comment:null), FieldSchema(name:teststuff5.countrycode, type:string, comment:null), FieldSchema(name:teststuff5.countryname, type:string, comment:null), FieldSchema(name:teststuff5.delivereddate, type:string, comment:null), FieldSchema(name:teststuff5.destinationaddress, type:string, comment:null), FieldSchema(name:teststuff5.internalmessageid, type:string, comment:null), FieldSchema(name:teststuff5.liveoperatorlookup, type:string, comment:null), FieldSchema(name:teststuff5.messageid, type:string, comment:null), FieldSchema(name:teststuff5.messageoriginator, type:string, comment:null), FieldSchema(name:teststuff5.messageoriginatorton, type:string, comment:null), FieldSchema(name:teststuff5.messagestatus, type:string, comment:null), FieldSchema(name:teststuff5.messagetype, type:string, comment:null), FieldSchema(name:teststuff5.mtscore, type:string, comment:null), FieldSchema(name:teststuff5.parentmessageid, type:string, comment:null), FieldSchema(name:teststuff5.phonenumber, type:string, comment:null), FieldSchema(name:teststuff5.productid, type:string, comment:null), FieldSchema(name:teststuff5.productiddescription, type:string, comment:null), FieldSchema(name:teststuff5.productname, type:string, comment:null), FieldSchema(name:teststuff5.productsubtype, type:string, comment:null), FieldSchema(name:teststuff5.programid, type:string, comment:null), FieldSchema(name:teststuff5.remoteipaddress, type:string, comment:null), FieldSchema(name:teststuff5.remoteresponsecode, type:string, comment:null), FieldSchema(name:teststuff5.responsecode, type:string, comment:null), FieldSchema(name:teststuff5.responsecodedescription, type:string, comment:null), FieldSchema(name:teststuff5.sourceaddress, type:string, comment:null), FieldSchema(name:teststuff5.subaccount, type:string, comment:null), FieldSchema(name:teststuff5.updateddate, type:string, comment:null), FieldSchema(name:teststuff5.useragent, type:string, comment:null), FieldSchema(name:teststuff5.userdataheader, type:string, comment:null), FieldSchema(name:teststuff5.userdefined1, type:string, comment:null), FieldSchema(name:teststuff5.userdefined2, type:string, comment:null)], properties:null)<br>2019-02-08T18:45:21,923 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.TableScanOperator (:()) - Initializing operator TS[0]<br>2019-02-08T18:45:21,924 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.SelectOperator (:()) - Initializing operator SEL[1]<br>2019-02-08T18:45:21,924 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.SelectOperator (:()) - SELECT struct<__time:timestamp with local time zone,accepteddate:string,accountid:string,accountname:string,apiversion:string,carrierid:string,carriername:string,companyid:string,contentencoding:string,countrycode:string,countryname:string,delivereddate:string,destinationaddress:string,internalmessageid:string,liveoperatorlookup:string,messageid:string,messageoriginator:string,messageoriginatorton:string,messagestatus:string,messagetype:string,mtscore:string,parentmessageid:string,phonenumber:string,productid:string,productiddescription:string,productname:string,productsubtype:string,programid:string,remoteipaddress:string,remoteresponsecode:string,responsecode:string,responsecodedescription:string,sourceaddress:string,subaccount:string,updateddate:string,useragent:string,userdataheader:string,userdefined1:string,userdefined2:string><br>2019-02-08T18:45:21,924 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.ListSinkOperator (:()) - Initializing operator LIST_SINK[3]<br>2019-02-08T18:45:21,925 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: ql.Driver (:()) - Completed compiling command(queryId=hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d); Time taken: 0.279 seconds<br>2019-02-08T18:45:21,925 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:21,927 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:21,931 INFO  [HiveServer2-Background-Pool: Thread-4925]: reexec.ReExecDriver (:()) - Execution #1 of query<br>2019-02-08T18:45:21,931 INFO  [HiveServer2-Background-Pool: Thread-4925]: lockmgr.DbTxnManager (:()) - Setting lock request transaction to txnid:185 for queryId=hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d<br>2019-02-08T18:45:22,009 INFO  [HiveServer2-Background-Pool: Thread-4925]: ql.Driver (:()) - Executing command(queryId=hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d): select * from teststuff5 limit 1<br>2019-02-08T18:45:22,010 INFO  [HiveServer2-Background-Pool: Thread-4925]: hooks.HiveProtoLoggingHook (:()) - Received pre-hook notification for: hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d<br>2019-02-08T18:45:22,016 INFO  [HiveServer2-Background-Pool: Thread-4925]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,023 INFO  [HiveServer2-Background-Pool: Thread-4925]: hooks.HiveProtoLoggingHook (:()) - Received post-hook notification for: hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d<br>2019-02-08T18:45:22,023 INFO  [HiveServer2-Background-Pool: Thread-4925]: ql.Driver (:()) - Completed executing command(queryId=hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d); Time taken: 0.014 seconds<br>2019-02-08T18:45:22,023 INFO  [HiveServer2-Background-Pool: Thread-4925]: ql.Driver (:()) - OK<br>2019-02-08T18:45:22,023 INFO  [HiveServer2-Background-Pool: Thread-4925]: lockmgr.DbTxnManager (:()) - Stopped heartbeat for query: hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d<br>2019-02-08T18:45:22,064 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,064 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,064 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,065 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,152 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,152 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,152 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,152 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,177 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,177 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,177 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,177 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,182 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,182 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,186 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: pool.ChannelResourceFactory (:()) - Generating: http://wdc-tst-bdrd-001:8082<br>2019-02-08T18:45:22,211 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,210 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.TableScanOperator (:()) - RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_TS_0:1, <br>2019-02-08T18:45:22,210 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.SelectOperator (:()) - RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_SEL_1:1, <br>2019-02-08T18:45:22,210 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: exec.ListSinkOperator (:()) - RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_LIST_SINK_3:1, <br>2019-02-08T18:45:22,211 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,235 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,235 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,235 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,235 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,238 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,238 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,238 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,239 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,240 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,241 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,241 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,241 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,245 INFO  [HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,245 INFO  [HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Updating thread name to 8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127<br>2019-02-08T18:45:22,245 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: operation.OperationManager (:()) - Closing operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=3b771b6b-7ff7-49dc-b212-dfc6b77211dc]<br>2019-02-08T18:45:22,245 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: operation.OperationManager (:()) - Removed queryId: hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=3b771b6b-7ff7-49dc-b212-dfc6b77211dc] with tag: null<br>2019-02-08T18:45:22,262 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: operation.Operation (:()) - Closing operation log /tmp/hive/operation_logs/8dea5cfb-f504-4237-a3b4-fb19d739b792/hive_20190208184521_4a573dee-6487-4e4e-a72f-83b343aed10d without delay<br>2019-02-08T18:45:22,262 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: conf.HiveConf (HiveConf.java:getLogIdVar(5130)) - Using the default value passed in for log id: 8dea5cfb-f504-4237-a3b4-fb19d739b792<br>2019-02-08T18:45:22,262 INFO  [8dea5cfb-f504-4237-a3b4-fb19d739b792 HiveServer2-Handler-Pool: Thread-127]: session.SessionState (:()) - Resetting thread name to  HiveServer2-Handler-Pool: Thread-127
1 REPLY 1

Expert Contributor

I bet your column names have upper cases in Druid tables, thus Hive will query the wrong columns since it is always down-casing the names