Hello @BSST ,   this page details about memory and storage recommendations per host: https://docs.cloudera.com/cdpdc/7.0/release-guide/topics/cdpdc-hardware-requirements.html   Is it what you were looking for, please?   Thank you: Ferenc ... View more

Hello @matagyula ,   thank you for sharing with us the exceptions you are getting after enabling for  "Kerberos Authentication for HTTP Web-Consoles" for YARN. You will need to configure SPNEGO [1] and enable authentication for HDFS too [2] to overcome the issues described.   Please let us know if the proposed changes resolved your issue!   Thank you: Ferenc   [1] https://docs.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_browser_access_kerberos_protected_url.html   [2] CM -> HDFS service -> search for and enable "Enable Kerberos Authentication for HTTP Web-Consoles", deploy client configuration, restart HDFS and YARN services ... View more

Hello @Sergiete ,   thank you for raising our attention that there was an improvement point in Cloudera CDP documentation. We have added clarification in our CDP documentation about "Installing Hive on Tez" [1] and that the Spark execution engine is not supported, as it's been replaced by Tez [2].   Based on our updated documentation [1] the correct order of installing the Hive service is: Install the Hive service, designated Hive on Tez in CDP. HiveServer is installed automatically during this process. Install HMS, which is designated Hive. Best regards: Ferenc   [1] https://docs.cloudera.com/runtime/7.0.3/hive-introduction/topics/hive_installing_on_tez.html [2] https://docs.cloudera.com/runtime/7.0.3/hive-introduction/topics/hive-unsupported.html ... View more

Hello @sh1vam ,    thank you for reporting that your HiveServer2 is not  coming up and throwing the below exception: exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) ... Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:226) registerAllFunctionsOnce ... Caused by: javax.jdo.JDOUserException: ... org.datanucleus.store.query.QueryNotUniqueException: The query returned more than one instance BUT either unique is set to true or only aggregates are to be returned, so should have returned one result maximum at  ... at org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:4091)     I found that your issue is resolved now.  Therefore I would like to add here the strategy that might be helpful for someone to troubleshoot similar issues: Start reading the stacktrace from bottom-up. Read the last "Caused by:" section first. We can see "QueryNotUniqueException". Now start reading the lines after the Caused by to see if you can find a some meaningful classname that is coming from hive. The first one is: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:4091) Google "org.apache.hadoop.hive.metastore.ObjectStore.getMRole" for the source code. The very first match was this one, which was OK for our purpose. The number in brackets is the line number that we should look up. Since we do not know the exact version you are using, we just search for the term getMRole in the hope we will get the clue what it is. Based on the context it seems to be a Metastore Role that we are trying to fetch. So far we know that a metastore role is not unique. Now we start reading further the caused by sentences and find that this caused that the SessionHiveMetaStoreClient could not be instantiated.     Based on your report you found out that the metastore.ROLES table had two admin roles and this was causing the issue and you were using the below commands for your troubleshooting: mysql> select * from metastore.ROLES; You identified that the role with id 3 is not required, hence deleted the role: mysql> delete from metastore.ROLES where ROLE_ID=3;   In production environment, please do not forget to create a backup or to have a means of recovery before issuing delete command.   Kind regards: Ferenc ... View more

Hello @sinhapiyush86 ,   thank you for raising the question about getting Unsupported JDBC protocol: 'null' exception in PySpark.   Please make sure you have initialised HWC in the session, otherwise you will get the below exception: java.lang.RuntimeException: java.lang.IllegalArgumentException: Unsupported JDBC protocol: 'null'   You can initialise HWC by the below code segment [1]: from pyspark_llap import HiveWarehouseSession hive = HiveWarehouseSession.session(spark).build()   Please let us know if it resolved your issue.   Thank you: Ferenc [1] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/integrating-hive/content/hive_hivewarehousesession_api_operations.html ... View more

Hello @gmalafsky ,   thank you for raising this question about how to configure Impala JDBC driver from a Windows machine.   Although the original question was raised some time ago, I would like to update this thread with the latest information.   For the latest Impala JDBC driver release the installation guide can be found under [1] in a PDF format. Page 8. describes that "Before you use the Cloudera JDBC Driver for Impala, the JDBC application or Java code that you are using to connect to your data must be able to access the driver JAR files. In the application or code, specify all the JAR files that you extracted from the ZIP archive."   For Java7, please follow the guide under [2] to configure the classpath correctly.   For the detailed instructions, please follow [1] section "Installing and Using the Cloudera JDBC Driver for Impala".   For release notes, please navigate to [3].   Please let us know if there is any additional information is required for this thread to be marked as solved.   Kind regards: Ferenc   [1] https://docs.cloudera.com/documentation/other/connectors/impala-jdbc/latest/Cloudera-JDBC-Driver-for-Impala-Install-Guide.pdf   [2] http://docs.oracle.com/javase/7/docs/technotes/tools/windows/classpath.html   [3] https://docs.cloudera.com/documentation/other/connectors/impala-jdbc/latest.html ... View more

Hello @mahfooz , thank you for raising your question about Hive's ACID support on CDH5.16. Based on [1] and, no Hive currently does not support ACID transactions.   Cloudera recommends using the Parquet file format, which works across many tools. Kind regards: Ferenc   [1] https://docs.cloudera.com/documentation/enterprise/5-16-x/topics/hive_ingesting_and_querying_data.html#hive_transaction_support   ... View more

@Rups can you please check if you provided all the required parameters from [1], especially check for the " SSL Context Service" parameter.   [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.InvokeHTTP/ ... View more

Hello @BVNP , thank you for raising the question about how to check if CM is running on a host.   Do I understand correctly that you can ssh into the host where CM is deployed, please? You can use for example the following commands to verify if CM is up and running: - "cat /var/run/ cloudera-scm-server.pid" and then use the ps command to see if the process with the PID is running - CURL the CM Web UI with "curl -k  https://<cm-host>:7183/cmf/login". If TLS/SSL is not enabled for CM,  you can use  "curl -k  http://<cm-host>:7181/cmf/login". This should return the HTML for the CM login page. - tail the CM Server log " tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log"   Please let us know if you need further troubleshooting tips.   Thank you: Ferenc     ... View more

Based on the Cloudera documentation [1] you need parcels for rolling upgrades. Rolling upgrade means less downtime/disruption in your cluster caused by the upgrade process.   You can migrate package installation to parcel based installation following the instructions under [2].   For further information about upgrading your CDH, please follow instructions under [3].   [1] https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_managing_software.html [2] https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_migrating_packages_to_parcels.html [3] https://docs.cloudera.com/documentation/enterprise/upgrade/topics/ug_cdh_upgrade.html ... View more