Member since
01-25-2019
75
Posts
10
Kudos Received
13
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2358 | 02-25-2021 02:10 AM | |
1138 | 02-23-2021 11:31 PM | |
2137 | 02-18-2021 10:18 PM | |
3067 | 02-11-2021 10:08 PM | |
15472 | 02-01-2021 01:47 PM |
02-18-2021
10:34 AM
Hello @bb9900m The use case in your scenario is make use of Load balancer. You can make user of external Load Balancer and balance the impala coordinators behind the Load balancer. This way, you will have the Load balancer Ip exposed to the client(Virtual IP) which further will balance the load based upon the load balancing mechanism set. Attaching the link and sample configuration for your reference. https://docs.cloudera.com/runtime/7.2.6/impala-manage/topics/impala-load-balancer-configure.html When it comes to SSL, make sure you have the LB hostname added as SAN names on all the impala certificates. Let me know if you have issues in any of the points above.
... View more
02-18-2021
10:26 AM
Hello @saamurai Could you please share the link where you read so. Meanwhile you can use the latest driver links to connect to Hive and Impala respectively for CDP. //For Hive https://docs.cloudera.com/documentation/other/connectors/hive-jdbc/2-6-13/Cloudera-JDBC-Driver-for-Apache-Hive-Install-Guide.pdf //For Impala https://docs.cloudera.com/documentation/other/connectors/impala-jdbc/latest/Cloudera-JDBC-Driver-for-Impala-Install-Guide.pdf You have separate drivers for Hive and Impala respectively. Let me know if the above helps.
... View more
02-11-2021
10:08 PM
1 Kudo
Hello @ryu The purpose of Ranger is to give the necessary user authentication to access the tables/database. If you allow a certain user access to a particular table/database, the user will be able to perform those actions on the table/database and the unwanted user automatically will not be able to remove the table. Let say there are two users test1 and test2. If I allow test1 user to have access to table t1 and test2 user to have access to table t2, test1 will not be able to see table t2 and test2 will not be able to see test1. You can further also add granularity as to what user can perform what actions on the table. This authorization is checked via Ranger hook which is present in the Hiveserver2. Let me know if the above answers your queries.
... View more
02-10-2021
09:45 PM
Hello @ryu Well you can make that user similar to hive user. hive user is mapped to hadoop group and you can make alterations to simulate a normal user to hive user but again as I mentioned earlier, you'll have to spend time managing it and eventually land up in spending more time in tshooting if things break. Remember hadoop is a complex setup with multiple components talking to each other. 🙂
... View more
02-10-2021
09:41 PM
Hello @dwill @Srivatsan CDH 5.7.3 is a very old version and there has been lot of fixes post that. Again coming to that error. Generally when you see the query in created state, we need to check where exactly the query is waiting. For example, query can be in created state if it is not able to fetch metadata from catalog server which is needed for submitting the query. It can also be in created state if the resources are less and query is queued.
... View more
02-09-2021
07:46 AM
Hello @ryu If you run the job with the end user, you will eventually end up managing internal permissions, job submission permissions your self. Also you will find difficulty integrating things as per my experience. But if you submit the job and let the hive user take care of the file creation, managing part in the backend, admin's job life become easier. You also will be able to hook/integrate things more properly. the above was just a jist, recommendations are to authenticate using the end user but then keep the impersonation off and let the hive take care of things in the backend.
... View more
02-01-2021
09:06 PM
@anujseeker and the HMS logs ?
... View more
02-01-2021
01:52 PM
hello @pphot You can migrate HMS and HS2 instances to any other hosts. So you can add another hosts for HS2 and HMS instance and remove the previous ones once the new ones are added and are functioning normally. For backend database, if you migrate then you have to update the configuration in the CM so that HMS has the updated information and access as to which host it needs to communicate in order to access its backend DB. Please note this is mandatory as Hive has all the information stored in its backend DB. Let me know if the above helps. Regards, Tushar
... View more
02-01-2021
01:47 PM
Hello @BhaveshP The error you are seeing is because certificate CN/SAN name mismatch than that of hostname. Let me try to explain you with an example. client [A]--> Balancer[B] -->NIFI Nodes[C,D,E] Here you have a client A who wants to access NIFI nodes C,D and E and he wants to access via B. When you create SSL certificates for C,D,E, you create 3 certificates with 3 different "Common Names" C, D. E respectively. Now when you try connecting to nifi nodes C,D,E respectively directly from client A, you will not observe the issue. But when you try to access C,D,E via balancer or any proxy B, you are likely to get the error. WHY? Client A is trying to talk to Nifi Node C or D or E but to client the NIFI node is B. During SSL handshake, the certificates are passed by Nifi server to A where in client gets confused becuase it wanted to talk to B however the certificate it got is from C. Name don't match. FIX: Make use to SAN Names in certificates. (Subject Alternative Names) Issue certificate of C,D,E such that it has a SAN name as B. What I am trying to say is, the certificate should have both the name that of C and B in nifi node C, D and B in nifi node D and E and B in nifi node E. In this way client will come to know that when it talks to B and when the certificate received to A from C, it will not get confused because the certificate will have two name present in SAN that of B and C. Let me know if the above gives some clarity as to what exactly is happening.
... View more
02-01-2021
01:34 PM
Hello @anujseeker It seems you are using the wrong path of hive. Below command works for me. hive --orcfiledump -d --rowindex 5 /warehouse/tablespace/managed/hive/tkathpal.db/orctable/delta_0000001_0000001_0000/bucket_00000 Now in my case, hive points to the actual parcel. [root@c2511-node1 ~]# which hive /usr/bin/hive [root@c2511-node1 ~]# ls -lt /usr/bin/hive lrwxrwxrwx 1 root root 22 Aug 3 2020 /usr/bin/hive -> /etc/alternatives/hive [root@c2511-node1 ~]# ls -lt /etc/alternatives/hive lrwxrwxrwx 1 root root 62 Aug 3 2020 /etc/alternatives/hive -> /opt/cloudera/parcels/CDH-7.1.1-1.cdh7.1.1.p0.3266817/bin/hive So when I run hive, ultimately the jars is being picked up from the right path. Could you please check the same on your end?
... View more