Member since
09-29-2015
5193
Posts
21
Kudos Received
34
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
329 | 07-13-2022 07:05 AM | |
1501 | 08-11-2021 05:29 AM | |
923 | 07-07-2021 01:04 AM | |
496 | 07-06-2021 03:24 AM | |
1541 | 06-07-2021 11:12 PM |
07-13-2022
07:05 AM
Hello @dk_srini ,
it is a great question! Have you seen this section of the Cloudera Doc about Redaction of Sensitive Information from Diagnostic Bundles, please?
If it covers your enquiry, please mark this answer as a solution. Best regards:
Ferenc
... View more
08-24-2021
12:54 AM
Hello @roshanbi , thank you for the clarification. I am not aware of any automated way of doing the translation. If I was facing with this task, I would dive into the differences between these SQL dialects and if Kudu needs some different design considerations when building up the queries and either to it manually or consider if it takes less effort to build custom automation, then build scripts for the job. Kind regards: Ferenc
... View more
08-11-2021
06:02 AM
1 Kudo
Hi @roshanbi , thank you for raising this question on the Cloudera Community! Do I understand correctly that you are storing data in Kudu and you would like to re-use your Oracle functions and use Zeppelin as your notebook? Please note, Kudu does not have an SQL Engine, hence it does not have JDBC driver available. First, you need to decide what SQL Engine will be used in combination with Kudu. Would it be for instance Impala, Spark, Nifi...? Once you know what SQL Engine will be used with Kudu, your next question would be how easy is it to translate your Oracle functions into that specific SQL dialect. Hope it helps! Kind regards: Ferenc
... View more
08-11-2021
05:29 AM
Hi @Sriparna , thank you for reaching out to Cloudera Community. I understand that you would like to consume from Kafka topic to Hive table using Structured Streaming and you run into some issues. I've found a Community Article that looks related. Have you seen this? Please note, if you consider using Continuous Processing, it is not supported as it is still in an experimental state. Hope these pointers take you closer to a solution! Best regards: Ferenc
... View more
07-09-2021
07:26 AM
Hi @SujaP , this can happen e.g. when you run out of disk space. 1. Please make sure that the disk have enough space to keep unpacked parcels. 2. Hard stop cloudera-scm-agent service (supervisord) 3. Empty the parcel directories : /opt/cloudera/parcels & opt/cloudera/parcel-cache 4. Start the agent service (#systemctl start cloudera-scm-agent) Kind regards: Ferenc
... View more
07-09-2021
06:58 AM
Hello @Eduardohahn , based on the exception, SMM is configured to use Ranger for authorization, but SMM cannot communicate with Ranger (404 exception). Is your cluster kerberized already? Without kerberos, it is expected to fail. The exception shows "auth:SIMPLE", which suggests to me that the cluster is not kerberized. Kind regards: Ferenc
... View more
07-07-2021
01:04 AM
Hello @Tomek , the exception is coming from Podium Data. Please reach out to Qlik Support, as based on the stack trace, the issue occurs in the Podium Data code, hence we do not have access to its sourcecode. Kind regards: Ferenc
... View more
07-06-2021
05:16 AM
Hello @RajatL , please find the information on the currently available API calls in the "CDP API reference". Hope it helps! Kind regards: Ferenc
... View more
07-06-2021
03:24 AM
Hello @someguy , Thank you for raising the question of how to set up a limit and configure alerts on CDP Public Cloud Usage spending. Because of the nature of how billing and metering work, best of my knowledge it is not possible. Please read the "Cloud Usage Pricing and Billing Terms" and the "Cloudera Data Platform (CDP) Public Cloud service rates", so you can calculate an estimate on the spending in advance. Hope it helps! Best regards: Ferenc
... View more
06-14-2021
11:34 PM
Hello @cardozogp , I am just wondering if you've seen the below documentation regarding the Sqoop - Oracle compatibility: https://support.oracle.com/knowledge/Oracle%20Database%20Products/1510470_1.html#aref_section24 Hope it helps you to move forward with your new requirement! Kind regards: Ferenc
... View more
06-07-2021
11:12 PM
Hello @michael_boulter , the issue you hit looks like YARN-10295. It occurs when the CapacityScheduler Asynchronous scheduling is enabled. The workaround for this issue is to set yarn.scheduler.capacity.schedule-asynchronously.enable to false in YARN's advanced scheduler configurations. Hope it helps! Kind regards: Ferenc
... View more
06-07-2021
03:15 AM
Hello @ebeb , please see the documentation on "Configuring SAML authentication on managed clusters". Please let us know if following the steps there resolved your issue! Kind regards: Ferenc
... View more
06-02-2021
04:06 AM
Hi @apedroso , thank you for starting this thread. In this reply, I will focus on how the YARN RM stores data about historical applications, which can be accessed via the RM Web UI. The RM keeps data about the applications in its state store [1]. It can be LeveldbRMStateStore, FileSystemRMStateStore or ZKRMStateStore. We recommend using ZKRMStateStore (this is what we use in YARN HA as well), because it is a more robust implementation. For example, you can migrate in RM HA standby RM while the active RM is still running and keep the state-store intact. Because the RM Web UI is reading the data from the state-store, it is independent of the presence or lack of YARN Application Logs. What are your exact migration steps? Do I understand correctly that you upgrade your cluster to CDP or do you need to move services to a new cluster, please? [1] Please read section for "yarn.resourcemanager.store.class" in https://hadoop.apache.org/docs/r3.1.2/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html
... View more
06-02-2021
02:31 AM
Hello @dirkj , the answer to your question is in the source code of the INSERT OVERWRITE in a form of comments under FinalizeHdfsInsert. Please let us know if you need further clarification on it! Best regards: Ferenc
... View more
03-23-2021
07:50 AM
For future reference adding here the link to our public documentation on how to connect NiFi with Hive on CDP.
... View more
03-23-2021
07:35 AM
1 Kudo
Please note, the instructions are also available in our public documentation under "Installing OpenJDK".
... View more
03-23-2021
07:09 AM
Please find here the Apache Hive tutorial and more information on selecting distinct values under the Aggregations section. For more on the Apache Hive content roadmap, please refer to our documentation.
... View more
03-23-2021
03:50 AM
Please find more about the "pros and cons to having hive jobs run as the hive user or the end user" vs. Ranger in our public Cloudera documentation for CDP: Enabling or disabling impersonation (doas)
... View more
03-23-2021
02:28 AM
To see more on Hive Managed and External tables, please see our public documentation for CDP Hive.
... View more
03-17-2021
07:09 AM
您好,请看一看下面的解决办法[1]: "12. Error If npm is executed under the root account, it will change the uid to the current account, or the value of the uid is obtained from the user configuration file, and the default value of uid is nobody. So when you run npm install under the root account, you need to change unsafe-perm Option plus. Solve (and then re-edit): # find ~/.m2/repository/ -name "*.lastUpdated" -exec rm -rf {} \; # mvn -B clean install rpm:rpm -DnewVersion=2.7.3.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X -rf :ambari-admin" [1] https://www.programmersought.com/article/72684156755/
... View more
03-09-2021
04:18 AM
Hello @VadymDemchuk , thank you for the deep-analysis. If I understand correctly, the process fails at the hostId. Can you please check on your hosts whether your configuration complies with [1]. Please check the documentation from this section onwards: " Cluster hosts must not use aliases, either in /etc/hosts or in configuring DNS. A properly formatted /etc/hosts file should be similar to the following example:" "Null value(s) passed to lookup by non-nullable natural id" is usually thrown because the hosts don't have any resolvable hostnames. All "hostname" values in the list of hosts should also contain the FQDN. Please let me know if it helped you to alleviate the issue. Thank you: Ferenc [1] https://docs.cloudera.com/cdp-private-cloud/latest/release-guide/topics/cdpdc-networking-security-requirements.html
... View more
10-12-2020
01:27 AM
Hello @Tak567 , thank you for letting us know you are hitting "Could not start EmbeddedOozieServer! Error message: Address already in use" exception. It can occur, for example when there are not enough ephemeral ports available on the OS. Kind regards: Ferenc
... View more
10-01-2020
01:06 AM
Hello @5sdba , thank you for reporting the issue you hit. Can you please specify the CDP version you are using and if you are using CM HA (Cloudera Manager High Availability), please? Thank you: Ferenc
... View more
09-28-2020
08:01 AM
1 Kudo
Hello @HV_Training , thank you for reaching out to Community with your question on how to check whether HDFS is started. Can you please clarify what do you exactly consider "HDFS is fully started"? If we are happy with an active NameNode as a "HDFS service started" state, I would use the CM API to find out if I already have an active NameNode by a REST call: https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/json_ApiNameservice.html Please let us know if it covers your inquiry. Thank you: Ferenc
... View more
09-25-2020
02:06 AM
Hello @Devink , thank you for reaching out to the Community. I understand that your issue is: Hive on Tez job that is failed because of OutOfMemory issue is reported as successful and you would like to know the reason for it. Please note if you have a Cloudera Subscription, a Support Case might be a better forum for deep analysis. Could you please give us some additional information, so we can assist you more efficiently? What is the distribution you are using and what is the version number, please? (e.g. HDP, CDH, CDP) Is the successful-state reported via Hue from an Oozie workflow? What Oozie action did you use to execute the Hive job, please? If it is not via Hue and Oozie, please specify where did you see the application reported as successful. Please note, Oozie is at the mercy of the status reported back from the job it is running. There are two ways that Oozie can detect a failure in e.g. a Java action. If an exception is thrown from the main method of the class executed by the Java action or System.exit is called anywhere in the code, Oozie will treat the action as failed. It will then transition to the Error state configured for the action. Thank you: Ferenc
... View more
09-23-2020
06:13 AM
Hello @Lilit , thank you for raising this question. I believe this post will answer your enquiry. Please let us know if you still need any further information! Thank you: Ferenc
... View more
09-22-2020
02:43 AM
Hi @iEason8 , you are welcome! Thank you for raising these questions and accepting the answer as a solution! Solved articles will surface better on search engines, hence other members with similar questions will find the answer easier. Best regards: Ferenc
... View more
09-22-2020
02:22 AM
Hello @iEason8 , Thank you for your observation on the API returns 404 when a request was sent to CM API without authentication. Based on W3C recommendation about authentication: "... If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead." Therefore it is OK to return 404 when there is an unauthenticated request and it is a design decision. Kind regards: Ferenc
... View more
09-22-2020
01:06 AM
Hello @Mondi , thank you for asking for clarification if CDP has a Cloudera Express-like "no license required" distribution. Our licensing strategy changed and you can find the details about it here. For your convenience, I am pasting here a part of the "Subscription Agreement" section that might be relevant for you: "3. Will Cloudera offer Developer or Trial software Subscriptions? Customers and developers will be able to access our products with a subscription agreement with Cloudera. We will have partner development subscription agreements for developers through our Cloudera Connect Partner Program, and free (unsupported) short-term trial subscriptions through Cloudera.com." Based on the documentation Cloudera Express is discontinued from version 6.3.3. Hope it clarifies, let us know, please! Kind regards: Ferenc
... View more