Member since
11-24-2015
56
Posts
58
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3107 | 05-21-2016 02:32 PM | |
5249 | 04-26-2016 05:22 AM | |
12835 | 01-15-2016 06:23 PM | |
13675 | 12-24-2015 04:52 PM |
08-27-2021
01:53 PM
https://community.cloudera.com/t5/Support-Questions/Hortonworks-Hive-ODBC-SSL-certificate-verification-failed/td-p/235787 This helped me! I had to add the entire host certificate chain to /opt/cloudera/hiveodbc/lib/64/cacert.pem file
... View more
08-11-2020
01:40 AM
I did this in root user, found the file and changed it there. But, how to change it for each node?
... View more
12-01-2018
03:07 PM
1 Kudo
Thank you for your answer. I wish I could buy you a beer. 🙂 My solution was slightly different but starring me right in the face. Your answer provided the perfect clue. In my case, there was no "universal" directory. However, I noticed there was a CA bundle file. It DID NOT come preconfigured with the GeoTrust CA we have for the SSL certificate we purchased for our F5. It did have some other GeoTrust CA and many others, as well. I simply opened up the cacerts.pem file and added that GeoTrust CA to the end, saved the file, and ran my test query. It then worked! [mpetronic@vmwhnsqsrclnt01 ~]$ echo "show tables" | isql -d, -b -v f5 mpetronic $(cat ~/.pw.dat)
cro_capacity_extract_tmp
cro_capacity_ranked_tmp
cro_capacity_report_final
cro_efficiency_extract_tmp
cro_efficiency_hourly_tmp
cro_efficiency_report_final
j1_total_user_counts
san_data_2
test
Here is what my directory structure looks like for the ODBC driver version I am using: [root@vmwhnsqsrclnt01 lib]# tree /usr/lib/hive/
hive
└── lib
└── native
├── hiveodbc
│ ├── ErrorMessages
│ │ └── en-US
│ │ ├── DSMessages.xml
│ │ ├── HiveODBCMessages.xml
│ │ ├── ODBCMessages.xml
│ │ ├── SQLEngineMessages.xml
│ │ └── ThriftExtensionMessages.xml
│ ├── EULA.txt
│ ├── Hortonworks\ Hive\ ODBC\ Driver\ User\ Guide.pdf
│ ├── Release\ Notes.txt
│ └── Setup
│ ├── odbc.ini
│ └── odbcinst.ini
└── Linux-amd64-64
├── api.prod.quasar.nadops.net.pem
├── cacerts.pem <<< Added GeoTrust CA to end of this file
├── cacerts.pem.orig
├── HiveODBC.did
├── hortonworks.hiveodbc.ini
└── libhortonworkshiveodbc64.so
... View more
09-09-2016
10:18 PM
Thanks @Constantin Stanca. Parsing that log will work just fine for my use case. Appreciate your help!!!
... View more
05-21-2016
02:32 PM
1 Kudo
I stumbled onto my own answer. Appears that you have to provide a partition spec to the alter command. I figured this out after poking around in mysql to look at my hive metastore to see if that would give me a clue. These queries are what made me think that the serde information is on a partition-by-partition basis: mysql> select * from TBLS where TBL_NAME='rm'\G
*************************** 1. row ***************************
TBL_ID: 170
CREATE_TIME: 1463833647
DB_ID: 11
LAST_ACCESS_TIME: 0
OWNER: mpetronic
RETENTION: 0
SD_ID: 227
TBL_NAME: rm
TBL_TYPE: EXTERNAL_TABLE
VIEW_EXPANDED_TEXT: NULL
VIEW_ORIGINAL_TEXT: NULL
LINK_TARGET_ID: NULL
1 row in set (0.00 sec)
mysql> select * from SDS where CD_ID=170\G
*************************** 1. row ***************************
SD_ID: 227
CD_ID: 170
INPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat
IS_COMPRESSED:
IS_STOREDASSUBDIRECTORIES:
LOCATION: hdfs://mpws:8020/jup1_stats/external_tables/rm
NUM_BUCKETS: -1
OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat
SERDE_ID: 227
*************************** 2. row ***************************
SD_ID: 228
CD_ID: 170
INPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat
IS_COMPRESSED:
IS_STOREDASSUBDIRECTORIES:
LOCATION: hdfs://mpws:8020/jup1_stats/external_tables/rm/year=2016/month=5/day=10
NUM_BUCKETS: -1
OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat
SERDE_ID: 228
*************************** 3. row ***************************
SD_ID: 229
CD_ID: 170
INPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat
IS_COMPRESSED:
IS_STOREDASSUBDIRECTORIES:
LOCATION: hdfs://mpws:8020/jup1_stats/external_tables/rm/year=2016/month=5/day=11
NUM_BUCKETS: -1
OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat
SERDE_ID: 229
Once I saw all those 'LOCATION' values in the SDS table, I tried the following command to alter the table and then the query worked. Interesting. Does this imply that you could have a different schema for each partition? Can anyone comment as to why the avro schema is tied to a partition and not simply to the whole table? alter table rm partition (year=2016) set serdeproperties ('avro.schema.url' = 'hdfs://mpws:8020/jup1_stats/avro/rm_1.avsc'); Since all my data is in partitions under the "year=2016" partition, I was able to just specify that one partition and it applied the change to all partitions under that.
... View more
05-26-2017
01:04 PM
And that said, I actually restarted Ambari as well - so I can't say for certain that the agent restart was sufficient; it may well have been the agents plus Ambari which did the trick.
... View more
05-02-2016
09:00 PM
How big is this specific ORC file and can this be shared with us ? Can you also check if this is hanging in one of the mapper (that is reading this ORC file) or before you get into application/mapper in YARN.
... View more
03-08-2016
10:55 PM
3 Kudos
First, spot-on by letting the ZK processes write to their own disks. As for letting the active/passive NNs write to the same physical disks as the JNs, I think you are OK with that approach. I say that as the edits are what are being written to continuously, but the fsimage files are only being read/recreated at key points such as checkpointing and startup. I probably pitched a bit of overkill in a blog I did last year on this topic of filesystems, but feel free to check it out at https://martin.atlassian.net/wiki/x/EoC3Ag if you need some help going to sleep at night. 😉 If you do check it out, you'll notice my very clear advice is that you should still make backups of the fsimage/edits files (even w/HA enabled) to avoid a potential "bunker scene" of your own. Having seen what happens first hand by losing this information (it was a configuration screw-up, not a h/w failure), I know I simply don't want to be there again.
... View more
03-08-2016
10:01 AM
@Mark Petronic Thanks for the feedback. I would check this and get it fixed as appropriate.
... View more
09-24-2018
02:53 PM
Request for @Ancil McBarnett (or anyone else who knows): Please flesh out a little on ... "You do not want Derby in your cluster."
... View more