Member since
02-02-2021
116
Posts
2
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
745 | 08-13-2021 09:44 AM | |
3696 | 04-27-2021 04:23 PM | |
1372 | 04-26-2021 10:47 AM | |
923 | 03-29-2021 06:01 PM | |
2752 | 03-17-2021 04:53 PM |
02-23-2021
05:20 PM
Hi, I just tried to install hive on a server. This server already has an existing hive service on it. My goal is to upgrade the version for our existing hive in our hadoop cluster. When I ran hive after installing it manually by following the instructions: https://cwiki.apache.org/confluence/display/Hive/AdminManual+Installation#AdminManualInstallation-InstallingfromaTarball I ran into the below error: [root@sunnymaster02 apache-hive-2.3.8-bin]# pwd /root/apache-hive-2.3.8-bin [root@sunnymaster02 apache-hive-2.3.8-bin]# bin/hive which: no hbase in (/root/apache-maven-3.6.3/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin) Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/root/apache-hive-2.3.8-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.1.0-129/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/root/apache-hive-2.3.8-bin/lib/hive-common-2.3.8.jar!/hive-log4j2.properties Async: true Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. hive> show databases; FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient hive> Can someone please tell me what the error message means or what am I doing wrong? Thanks,
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
02-22-2021
06:54 PM
Hi, I am trying to install Ranger from source and followed these instructions. https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-BuildRangerAdminfromsource Under the "Install Steps for Ranger Policy Admin on RHEL/CentOS" I followed the steps and after running the command "ranger-admin start" I can see the process running, however, I am having trouble accessing the Ranger URL. When I try to access the Ranger URL, it seems to have trouble loading. [root@sunnymaster02 ranger-admin]# ranger-admin start Apache Ranger Admin has started. [root@sunnymaster02 ranger-admin]# ps -ef | grep ranger root 3798 1 99 20:51 pts/0 00:00:12 java -Dproc_rangeradmin -XX:MaxPermSize=256m -Xmx1024m -Xms1024m -Dlogdir=/usr/local/ranger-0.5.4-SNAPSHOT-admin/ews/logs/ -Dcatalina.base=/usr/local/ranger-0.5.4-SNAPSHOT-admin/ews -cp /usr/local/ranger-0.5.4-SNAPSHOT-admin/ews/webapp/WEB-INF/classes/conf:/usr/local/ranger-0.5.4-SNAPSHOT-admin/ews/lib/*:/usr/local/ranger-0.5.4-SNAPSHOT-admin/ews/ranger_jaas/*:/usr/local/ranger-0.5.4-SNAPSHOT-admin/ews/webapp/WEB-INF/classes/conf/ranger_jaas:/usr/lib/jvm/jre-openjdk/lib/*: org.apache.ranger.server.tomcat.EmbeddedServer root 3877 21271 0 20:52 pts/0 00:00:00 grep --color=auto ranger [root@sunnymaster02 ranger-admin]# netstat -nap | grep 3798 tcp6 0 0 127.0.0.1:6185 :::* LISTEN 3798/java tcp6 0 0 :::6080 :::* LISTEN 3798/java tcp6 0 0 127.0.0.1:52784 127.0.0.1:3306 ESTABLISHED 3798/java tcp6 0 0 127.0.0.1:52791 127.0.0.1:3306 ESTABLISHED 3798/java tcp6 0 0 127.0.0.1:52785 127.0.0.1:3306 ESTABLISHED 3798/java tcp6 0 0 127.0.0.1:52790 127.0.0.1:3306 ESTABLISHED 3798/java tcp6 0 0 127.0.0.1:52788 127.0.0.1:3306 ESTABLISHED 3798/java unix 2 [ ] STREAM CONNECTED 103189513 3798/java unix 2 [ ] STREAM CONNECTED 73379865 20086/java unix 2 [ ] STREAM CONNECTED 103189515 3798/java [root@sunnymaster02 ranger-admin]# curl sunnymaster02:6080 [root@sunnymaster02 ranger-admin]# curl sunnymaster02:6185 curl: (7) Failed connect to sunnymaster02:6185; Connection refused Any help is much appreciated. Thanks,
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Ranger
02-22-2021
11:35 AM
Hi, I recently installed ranger by building it from source. We also have ambari installed and was wondering if anyone can help redirect me to somewhere that explains how to manage Ranger using Ambari. Thanks,
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
Apache Ranger
02-18-2021
03:59 PM
Thanks @tusharkathpal for the response. Can you please let me know what are some of the use cases to storing the external tables at a certain location? Thanks,
... View more
02-16-2021
11:37 AM
Hi, I was just wondering what is the best practice on where to store hive external tables in HDFS? Currently they are being stored in the /tmp directory in HDFS but sometimes someone accidentally deletes the files under /tmp and right now I am trying to add permissions to ranger to prevent this from happening. Thanks,
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
-
Apache Ranger
02-11-2021
05:57 AM
Thanks @tusharkathpal So if all users when running hive jobs are being ran as the "hive" user, is there a way to differentiate between different permissions? For example, If I were to run a hive job and you were to run a hive job, would it be possible to have separate permissions between you and me so I would not be allowed to delete any tables you created and vice versa? So we currently have Ranger installed in our hadoop cluster and my understanding is that the "run as end user" needs to be set to "false". So would Ranger know which user is running the job since all jobs are being run by the "hive" user. Just want to make sure I am setting this up correctly. Thanks,
... View more
02-09-2021
10:54 AM
Thanks @tusharkathpal for the response. But if the user is authenticated as the user but the job is run as the user "hive", does that user also get similar permissions as the hive user?
... View more
02-09-2021
05:53 AM
Hi, I am wondering what are the pros and cons to enabling or disabling "Run as end user instead of Hive user"? Currently we are trying to implement proper permissions for our hadoop cluster to prevent any accidents etc. My understanding is if a user logs into hiveserver2, and if they submit a query, and it runs as the hive user, then technically they should have access to all databases and tables etc, which can cause damage if not careful. Since there is this setting, I would like to better understand the use case scenarios or pros and cons etc. Thanks,
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive
02-04-2021
09:33 AM
Thanks @jagadeesan for your response. Currently we do not have any vendor support and I believe we need to be with Cloudera in order to use Cloudera Manager. But is there any good solution that does not require any paid subscription? Thanks,
... View more
02-03-2021
03:54 PM
Thanks @GangWar for the response. I believe I have checked this site out before. Maybe I missed it in the link you provided, but I am interested in a way to monitor yarn jobs and possibly send out some alert when a job is running too long or possibly send some report of the status of each job, maybe how long each job took to run for the day.
... View more
- « Previous
- Next »