Solved

i use kadmin to generate impala principal keytab(it previous generated by cloudear manager), now impala start failed even though new keytab generated by Cloudera Manager. in cdh admin page error is: unable to login from keytab: Decrypt integrity check faileduse kinit in command line error is: kinit: Password incorrect while getting initial credentials any help will really appreciated.

I am trying to decommission a data node. I started by decommissioning the regionserver, and the nodemanger. I have 7 nodes in the cluster. When I followed the steps to remove the one nodemanager it says it was successful. Even though it says 1 was decommissioned 7 are still showing as active. How do I resolve?  

Hello Team, we are having some performance issue when sending data to Kafka using JSON as format. Performs is very slow. Performance is faster when using DSV parser. How can we fine tune JSON formatter? Regards,Roshan

Hi,   I am getting failure in PutEmail Processor in our organization's nifi server.The error states that "Could not convert socket to TLS:routing to failure: javax.mail.MessagingException: Could not convert socket to TLS:nested exception is:javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake"   I routed the failure relation back to processor and after 30-40(randomly) or more retries it processes successfully.Though trying to figure out why its getting failure in first place. Attaching the log here. 2021-10-25 12:44:36,643 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.PutEmail PutEmail[id=bdc0872f-2860-12ce-c193-24605d5b6823] Failed to send email for StandardFlowFileRecord[uuid=844b4761-2a4c-439a-9f3d-5e6f06739c00,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1635165813775-357600, container=default, section=224], offset=3718, length=82370],offset=0,name=SampleFile.csv,size=82370]: Could not convert socke

Hi Support team,I kindly need your support.I am doing some tests in order to evaluate into Cloudera the Atlas data lineage by importing Hive DB.I executed several tests and also followed entirely this sample: https://community.cloudera.com/t5/Community-Articles/Using-Apache-Atlas-to-view-Data-Lineage/ta-p/246305 In order to migrate all the entities into Atlas, I successfully executed the import-hive.sh script that is located in Atlas hook-bin folder.The result of the migration shows (into Atlas) Hive test DB with all its entities but I cannot see in Atlas any lineage (that should appear as the join between two tables in order to create the third one), simply the message: " No lineage data found ".I also checked that if I create EXTERNAL TABLES rather than TABLE, something appears in Lineage section but is incomplete and not showing exactly the dependencies with source table names.Could you please guide me in the solution of this issue that is hiding the Data Lineage

I have 5 Hadoop/HDFS nodes running RHEL 7.9, Ambari 2.7.5.0, HDP 3.1.5. /hadoop/yarn/local/usercache/hive/filecache/ on all of the Hadoop nodes is growing very large and I want to decrease its size. Is there a YARN setting I can change to lower the amount of filecache specified above?  Do I need to manually delete directories that are X days old to clean this up? Please advise.  Thanks! Mike

Hi All, I have a requirment in my company to use an existing password vault for all the technical accounts used by cloudera CDP. Do you know if there is a way to integrate DCP with any password vault ? in my case HashiCorp ThanksAlexis 

Hi Team,  I have a requirement where I need to schedule an oozie work flow using shell action. Shell action is using a shell script which in turn will be calling a hql file which is available on HDFS location. I am not able to call that hql file from the shell script which I am using in shell action. I know using beeline - f parameters to use to call local files but can you please guide me to call a file which is on HDFS. Any help/suggestions will be highly appreciated. Thanks, Wasim

Timeline Service V2.0 Reader not startingdetails=row 'prod.timelineservice.entity' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=sandbox-hdp.hortonworks.com,17020,1628801419880, seqNum=-12021-09-29 10:23:01,232 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=20, retries=36, started=231328 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server sandbox-hdp.hortonworks.com,17020,1632910734969 is not running yetat org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1487)at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2443)at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)at org.apac

Hi all,I have a cluster of which computers are with different hardware spec.Is there a way to make Region Server to hold different number of regions depending on the hardware spec of the host?Or is there other way for performance tuning under this circumstance?The hardware spec for HBase Region Server: CPU (cores)Memory (GB)NumbersComputer 14321Compurer 28642Computer 3242565There are YARN and HBase on this cluster of 8 Computers.

We are using Cloudera 6.2.1 and enabling row and cell level ACLs on our Hbase tables.  After turning on the Hbase authorization toggle, in some environments, everything works perfectly.  We can add new labels and auths from Hbase shell. In a few environments when trying to call the 'add_labels' command in Hbase shell it complains that 'Table hbase:labels does not exist'.  To fix it i ran the following command. create 'hbase:labels','f' Then I was able to call the add_labels command no problem.  I was just curious what actually creates this table and was trying to understand why in some environments it wasn't created. Richard

Description:Nifi node will be stuck and all processors won't deal with flow file when one processor is processing a large flow file, like PutFile, ExecuteStreamCommand, ConvertRecord. Nifi node will be very very slow and heavy cpu usage. Sometime the cpu usage rate will over 700%. Then the heartbeat may be Disconnected. Heavy cpu usage also can cause other processors stuck and never consume any flow file until the large flow file processing is completed. As I know, the PutFile or ExecuteStreamCommand shouldn't cost too many cpu, it is just executing a cp command in linux. We have tried to increase the total thread count. But it is not any effect. This issue happened after we upgrade the nifi version from 1.8 to 1.12. We can't understand why these processors can cost so many cpu than nifi node will be disconnected from cluster. Anyone can help me? Thanks a lot. Nifi Version: 1.12Machine capacity:cpu 8 core,memory 64G,Xmx 20G,Nifi node amount 5

If I am trying to access Zeppelin interpreter page , I keep getting the error "You don't have permission on this page" . 

Hello,we observe lot of annoying messages in kafka logs like this. 2021-09-01 17:09:47,435 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: unable to return groups for user OU=Dept,O=Company,C=DE,ST=Germany,CN=some_userPartialGroupNameException The user name 'OU=Dept,O=Company,C=DE,ST=Germany,CN=some_user' is not found. id: OU=Dept,O=Company,C=DE,ST=Germany,CN=some_user: no such userid: OU=Dept,O=Company,C=DE,ST=Germany,CN=some_user: no such userat org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:291)at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:215)at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroupsSet(ShellBasedUnixGroupsMapping.java:123)at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupSet(Groups.java:413)at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:351)at org.apache.hadoop.security.Groups$

HelloI was wondering how one would best implement the case where multiple users have their own python environment with their respective packages installed in a Spark2 interpreter. I have not implemented or tested the two approaches below yet but was wondering if I am on a completely wrong path and there would be a very easy and straightforward solution.   Two ways I could think of would be the following: 1.) For each notebook set the variable zeppelin.python If one would instantiate the interpreter per user you could run the following at the start.   %spark2.confzeppelin.python </your/users/python_env>   This would add some complexity for the users because they have to write down their environment paths but would otherwise be very easy to implement (If it really works without hiccups).    2.) Create a separate Interpreter for every user This approach would be to create an interpreter for every user (%spark2 -> %spark2userName). This would be relatively

Hola,i need stop/start the roles and CM services many time for patching the servers. Is there a solution by one script?Thank you!!

Dear team, We are facing the below issue on one of the Solr nodes.  2021-10-17 04:05:57.006 ERROR (qtp1916575798-2477) [c:ranger_audits s:shard1 r:core_node2 x:ranger_audits_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Cannot talk to ZooKeeper - Updates are disabled. at org.apache.solr.update.processor.DistributedZkUpdateProcessor.zkCheck(DistributedZkUpdateProcessor.java:1245) at org.apache.solr.update.processor.DistributedZkUpdateProcessor.setupRequest(DistributedZkUpdateProcessor.java:582) at org.apache.solr.update.processor.DistributedZkUpdateProcessor.processAdd(DistributedZkUpdateProcessor.java:239) at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.AddSchemaFieldsUpdateProc

Dears, Greetings!I have the requirement to get the all HDFS filename which are older than N days.I got the all last level directory which are older But requirement is all filenames.Kindly support with hdfs command or script or code for the same.  also suggest on if we query any Hive table and done some sum on one column or join with other table Is this will change the timestamp of underlining hdfs file of respective Hive table. obviously write or update to table will change the timestamp of respective hdfs file. Appreciate swift response

I am a newbie to NiFi and I am trying to pass a JSON object to NiFi. Having failed at multiple attempts, I am trying a very simple flow as below. I have checked the JSON in online validators and have confirmed that it's a valid JSON file. Yet, I am not getting the output I want i.e. Country and Capital in flow file attributes. Please advise where I am going wrong. Thanks in advance. 

I installed a 2 node hadoop cluster. The master and slave node starts separately but the datanode isn't shown in namenode webUI. The log file for datanode shows the following error : 2016-02-09 23:30:53,920 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2016-02-09 23:30:53,920 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting 2016-02-09 23:30:54,976 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop-master/172.17.25.5:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 2016-02-09 23:30:55,977 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop-master/172.17.25.5:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)2016-02-09 23:21:15,062 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/172.17.25.5:9000 The hosts

I am trying to use a ListSFTP processor to delete files from an SFTP site older than the most recent  "History" file. In order to do this, I need to continually (process would run daily or weekly) query all the files in the SFTP server to see if a new "History" file was added, and if so to remove all files from the site before this newest "History" file. The problem is that ListSFTP only shows the newest files added so I am unable to do this. I can ClearState on the processor to make it work, but I would like to avoid this being a manual process. So how can I have ListSFTP show all the files every single time I run it? Note: I've seen some posts about automatically clearing the state using a curl request, but this only works on a stopped processor which defeats the purpose of this being an automated process (https://community.cloudera.com/t5/Support-Questions/Nifi-clear-state/td-p/242347)

Hi Everyone, Wanted to check your opinion on one backup and DR situation. So for one of our Dev Platform, we are still on CDH 5.4 version (with EOD already passed not under technical support scope), all the servers for this platform are going to be physically migrated from one Data Center to another Data Center. These 34 servers are placed in three separate racks, so considering in-built replication and Rack awareness, so technically I believe as long as we loose few servers and even one single Rack, we should still not be facing any data loss. Now considering the worst case scenario that multiple Racks or all servers are damaged during migration, I need to have a backup and DR solution in place. To consider few limitations, we don’t have architectural capacity to spin up separate cluster to copy existing hdfs data there for recovery purpose. Currently we are thinking to backup all servers using Avamar backup tool, all OS data and data disks we can backup. I am going to take

I would like to undo the delete operation performed using Impala on the underlying Kudu table. Is it something supported in Impala/Kudu? Could someone help me with it? 

Hi all, I recently installed Ranger on Cloudera Private Cloud Base 7.1.7. Everything is installed perfectly but the ranger admin UI is not loading. The cluster is kerberized. I am able to curl the admin URL with the --negotiate flag. Is the UI not loading because of spnego authentication? If yes, how do I disable it?We have option to disable that for HDFS and YARN, but not sure about Ranger.Thank you!

Hi,I found this new CVE for Hadoop (CVE-2020-9492).The solution is to upgrade to Hadoop 2.10.1 but HDP 2.6.5 (with Apache Hadoop version 2.6.7) is the latest version of Apache Hadoop 2.Could be possible to fix any CVE into HDP 2.6.5.Thanks in advance.