About GangWar

GangWar · ‎10-22-2020

@hitachi_ben Go to CM > Hive > Configuration and then set MapReduce Service to YARN. For example below. Let me know if this helps.

GangWar · ‎10-21-2020

@muslihuddin I would encourage you to use the live doc version: https://docs.cloudera.com/runtime/7.2.1/security-ranger-authorization/topics/security-ranger-tag-service-add.html However that's been fixed in recent releases, but not back-ported to 7.1.1 yet: https://docs.cloudera.com/cdp-private-cloud-base/7.1.4/security-ranger-authorization/security-ranger-authorization.pdf Thanks for the feedback.

GangWar · ‎10-21-2020

@hughatoclc Unfortunately the Maven Artefacts For CDP 7.1.4 is still under construction so will be available after some time (could be 1 or 2 week). Apologies for inconvenience caused.

GangWar · ‎10-20-2020

@saudcibc There is no minor release of doc means there will be 6.2 doc which will valid to 6.2.x versions. https://docs.cloudera.com/documentation/enterprise/6/6.2/topics/administration.html Only minor version release doc will be available the way you are looking: https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cm_6_release_notes.html#cm6_release_notes

GangWar · ‎10-20-2020

@manojdara Not from API but just for the information you can check the information here in this doc: https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_63_packaging.html#cdh_630_packaging CDH 6.3.3 Packaging Component Component Version Apache Avro 1.8.2 Apache Flume 1.9.0 Apache Hadoop 3.0.0 Apache HBase 2.1.4 HBase Indexer 1.5 Apache Hive 2.1.1 Hue 4.4.0 Apache Impala 3.2.0 Apache Kafka 2.2.1 Kite SDK 1.0.0 Apache Kudu 1.10.0 Apache Solr 7.4.0 Apache Oozie 5.1.0 Apache Parquet 1.9.0 Parquet-format 2.4.0 Apache Pig 0.17.0 Apache Sentry 2.1.0 Apache Spark 2.4.0 Apache Sqoop 1.4.7 Apache ZooKeeper 3.4.5

GangWar · ‎10-20-2020

@Yuriy_but For CDH related service like Yarn you can configure NodeManager Maximum Log File Backups in Service Configuration but if these are some custom logs then I would say setup a cron job to delete these periodically.

GangWar · ‎10-20-2020

@emeric Looking at your logs seems to be a port issue and you have to check if port is listening or not.

GangWar · ‎10-16-2020

@cuian01 There is no inbuilt feature. This is only possible using some external proxy application as per my understanding.

GangWar · ‎10-16-2020

@Yuriy_but Please follow below doc for NN heap calculations. https://docs.cloudera.com/documentation/enterprise/latest/topics/admin_nn_memory_config.html Also there is another thread nicely explained the same thing here: https://community.cloudera.com/t5/Support-Questions/How-a-NameNode-Heap-size-is-calculated/td-p/227052

GangWar · ‎10-12-2020

@Muraliselvaraj When a Sqoop program is executed, it will create a java file like QueryResult.java in the current directory. If QueryResult.java is already there as we already executed the Sqoop command before, the message will be shown. The path needs to be new every time, as this is how the sqoop is designed. As far as I know it is by design we are seeing this ERROR message, this is a harmless message. 'xxx.java' is the Java source code file which is generated by Sqoop itself on the fly. This would not cause any job failure. 'xxx.java' is only used to be compiled to the actual .class file which is used to submit as MR job. After the job is submitted, we can safely delete it. Are you seeing any job failure because of this error message?

GangWar · ‎10-08-2020

@alexsunny The error says you are not able to connect with FQDN, Have you tried with IP address? grunt> cd s3://datasets.elasticmapreduce/ngrams/books 2013-08-22 00:43:13,413 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. Wrong FS: s3://datasets.elasticmapreduce/ngrams/books, expected: hdfs://ip-172-31-35-24.us-west-2.compute.internal:8020 Details at logfile: /var/lib/hadoop-hdfs/pig_1377132180339.log

GangWar · ‎10-06-2020

@atl-techdev Cloudera don’t support Oracle Database for Nifi Registry officially. Please check the matrix here: https://docs.cloudera.com/cfm/2.0.4/support-matrix/topics/cfm-supported-databases.html

GangWar · ‎10-06-2020

@Tak567 you can manually install/upgrade the agent packages by something like below commands. # yum install https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm # yum install https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm Make sure to replace your CM version and user/password accordingly.

GangWar · ‎10-06-2020

@Ananya_Misra Namenode restart failed with below exception : 'org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not exist during upgrade. Finalize or rollback first.' This issue occurs because, during an upgrade, NameNode creates a different directory layout under NameNode directory, if this is stopped before Finalize upgrade, NameNode directory will go to an inconsistent state. NameNode will not be able to pick the directory with a different layout. To resolve this issue start Namenode manually through command line with the -rollingUpgrade started option and proceed with the upgrade: # su -l hdfs -c '/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config /usr/hdp/current/hadoop-client/conf start namenode -rollingUpgrade started

GangWar · ‎10-06-2020

@Mondi As you know Atlus is not packaged with Cloudera 6.x version so this is very tedious to install on same. Also if you managed to install the Atlus even this can not guarantee to work with Cloudera 6.x version as there are so many dependencies and Configurations inbuilt which needs to work in CM. Anyway you can take a look at this doc to give a try: http://atlas.apache.org/2.0.0/InstallationSteps.html

GangWar · ‎10-06-2020

@Ananya_Misra This seems your curl command is not working. In this case there might be a possibility Curl installed in the system doesn't support. In this case, curl command in the OS prompt returned the following output: [hdfs@namenode_hostname ~]$ curl --negotiate -u : -s "http://namenode_hostname:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem" curl: option --negotiate: the installed libcurl version doesn't support this curl: try 'curl --help' or 'curl --manual' for more information [hdfs@namenode_hostname data]# which curl [hdfs@namenode_hostname data]# curl --version So I would suggest you to manually run above curl command to see if that works and what is the output. So you might need to uninstall curl package in the nodes and and install the same from the default OS packages. Or change $PATH so that the OS default curl is used but make sure to verify with above command first. Also if you are an entitled Cloudera Customer feel free to open a case with Cloudera so that you can get prompt assistance.

GangWar · ‎09-30-2020

@farouk You have to set those in hdfs-site.xml. For example: HDFS Service Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml HDFS (Service-Wide) View as XML Name dfs.client.datanode-restart.timeout Value 30 Description Description Final Or something like below: <property><name>dfs.client.datanode-restart.timeout</name><value>30</value></property>

GangWar · ‎09-30-2020

@Cluster-CDP You can follow below Cloudera Recommendation about Role distribution as per number of nodes in the cluster. Though in your case this seems a best fit for me. https://docs.cloudera.com/cloudera-manager/7.0.3/installation/topics/cdpdc-runtime-cluster-hosts-role-assignments.html Master Hosts Utility Hosts Gateway Hosts Worker Hosts Master Host 1: NameNode YARN ResourceManager JobHistory Server ZooKeeper Kudu master Spark History Server One host for all Utility and Gateway roles: Secondary NameNode Cloudera Manager Cloudera Manager Management Service Hive Metastore HiveServer2 Impala Catalog Server Impala StateStore Hue Oozie Flume Gateway configuration HBase backup master Ranger Admin, Tagsync, Usersync servers Atlas server Solr server (CDP-INFRA-SOLR instance to support Atlas) Streams Messaging Manager Streams Replication Manager Service ZooKeeper 3 - 10 Worker Hosts: DataNode NodeManager Impalad Kudu tablet server

GangWar · ‎09-29-2020

@dallanic Sure. Keep posted here and don't forget to mark this post as solution so that this will help other members as well.

GangWar · ‎09-29-2020

@dallanic It is indicating that the procedure that was run to remove the entry got hung for xyz reasons. So there might be some issue with hosts/disk/memory/table itself. You have to do a clean perform I would say. Try to kill hung process and then do.

GangWar · ‎09-29-2020

@Pavel_kostyukov You have to install on the need basis, if you need them then you can install by following below document. https://docs.cloudera.com/documentation/enterprise/latest/topics/impala_udf.html#udf_demo_env

GangWar · ‎09-29-2020

@dallanic Is there any error message you are getting? You might want to look at various commands and try again, below doc can be useful. https://learnhbase.net/2013/03/02/hbase-shell-commands/

GangWar · ‎09-29-2020

@b1995assuncao I don't think it's possible. The range is dynamic and by default always will be in latest time range.

GangWar · ‎09-29-2020

@rjpsarath2 Please read carefully the section below on same page: What external websites will I have access to while taking the exam? The exam gateway has a firewall that restricts your access to external resources. You will have whitelisted access to the Apache and Cloudera websites. Other sites we determineto be necessary and appropriate for a given exam may also be made available. Websites like Google, StackOverflow etc. and other search functionality will not be accessible. Downloading external code or packages is not permitted. Where Apache and Cloudera websites means below Websites: Apache: https://hadoop.apache.org/docs/stable/ Cloudera: https://docs.cloudera.com/documentation/index.html Regarding the tools, as far as I know during the exams you have been given a environment where you have to perform the task so you can use any commands available in that OS or related with environment. If you are expecting using a tool out side of exam window then it's not. As the exam guide clearly saying: You may not launch any other application on your computer during the exam You may not take screenshots of the exam environment Hope this help. Let me know if you still have any question.

GangWar · ‎09-28-2020

@rjpsarath2 This FAQ page will resolve your most of the queries. Please check below: https://www.cloudera.com/about/training/certification/faq.html

GangWar · ‎09-28-2020

@b1995assuncao you can do that from the top right corner and select custom range. Hope below screenshot will help you.

GangWar · ‎09-28-2020

@HanzalaShaikh You can see the audit hadoop-cmf-hdfs-NAMENODE-namenode1.us-east1-b.c.coherent-elf-271314.internal.log.out file. Try to clean start the HDFS again and see if that helps.

GangWar · ‎09-28-2020

@muslihuddin Yes, that's right.

GangWar · ‎09-28-2020

@muslihuddin The CM server down situation will not affect any HDFS data. For your specific use case you need: 1. A backup of CM DB (NOTE: This is most important to get your old configuration back) 2. Then you can install CM server on another hosts and restore the old CM DB backup and you should be able to monitor the old cluster again. You old monitoring data will not be available as that is still resides on different node so you will loose that at this point means old info in charts and dashboard will be not available.

GangWar · ‎09-25-2020

@mohammad_shamim You might be hitting a known bug with Java recent versions (OpenJDK 1.8u242 or JDK 11.0.6). TSB-394. To resolve this issue, take the following action on all impacted nodes solved as appropriate for the environment. Edit java.security file located in the active JDK on the clusters. Add or alter sun.security.krb5.disableReferrals parameter, to ensure that the following is set to true: sun.security.krb5.disableReferrals=true

Online	Offline
Last Visited	‎10-23-2020 06:13 AM

Date Registered	‎10-01-2018 03:22 AM
Last Visited	‎10-23-2020 06:13 AM
Total Messages Posted	439
Total Kudos Received	66

Re: Missing service dependency for Hive: YARN

Re: Documentation Screenshot Issue

Re: Maven Artifacts For CDP 7.1.4

Re: Is oracle 12c and 19c fully compatible with Ni...

Re: Namenode not starting For HDP 3.1.5 HDFS

Re: Missing service dependency for Hive: YARN

Re: Documentation Screenshot Issue

Re: Maven Artifacts For CDP 7.1.4

Re: Cloudera Enterprise 6.2.2 documentation?

Re: kafka parcel details in cloudera 6.3x

Re: yarn local logs

Re: Cannot start Ambari services

Re: How to shorten Cloudera URLs?

Re: heap size of namenode/ blocksize

Re: Sqoop import error : orm.CompilationManager: C...

Re: can't connect w/ Amazon S3?.

Re: Is oracle 12c and 19c fully compatible with Ni...

Re: Failure: Upgrade agents in CM v6.3.3

Re: Namenode not starting For HDP 3.1.5 HDFS

Re: How to Manually Install Atlas?

Re: Namenode not starting For HDP 3.1.5 HDFS

Re: can not find dfs.client.datanode-restart.timeo...

Re: Cluster CDP Best Practices

Re: CDH 6.2.0 : Unable to remove peer from hbase s...

Re: CDH 6.2.0 : Unable to remove peer from hbase s...

Re: impala-udf-devel package

Re: CDH 6.2.0 : Unable to remove peer from hbase s...

Re: Filter in the Report

Re: CCP data engineer (DE575) EXAM Tools Used

Re: CCP data engineer (DE575) EXAM Tools Used

Re: Filter in the Report

Re: Error while cluster setup in installation.

Re: Cloudera Manager reinstallation and manage exi...

Re: Cloudera Manager reinstallation and manage exi...

Re: RM is getting down after enabling kerberos