Solved

when I upgrade to the HDP3.1 and hbase also upgrade to 2.0(ambari server show 2.0.0.3.1,and hbase version shows HBase 2.0.2.3.1.0.0-78),and how can i use hbck2 if I need compile myself,and how I set the hbase version apache,or HDP ,where can I get the HDP pom.xml ,now hbck can't fix my hbase2.0

Hello everyone i am now having a problem connecting tableau to druid on HDP.... Anyone know how to connect tableau to druid ?

Anyone know..How to connect R Studio to Hive in Hortonworks Data Platform 3 ?I am using Hive 3.1.0

Hi.I have a problem with outOfMemory error on impala catalog server.Data in tables about 30TB and about 175k partitiions.I set Java Heap Size of Catalog Server in Bytes=30gb, but still have out_of_memory error in start.How can i calculate how much memory needs for catalog server? And is it have any limit to set ?

Hi, I'm using NIFI EvaluateXPath module.I am trying to extract the value from the first Channel parameter (Microsoft-Windows-Sysmon/Operational) and add it as an attribute - The channel1 attribute is empty when setting the value to  "/Event/System/Channel"    I'm trying to understand what should be the Value in the Channel1 property  XML  <?xml version="1.0" encoding="UTF-8"?><Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name="Microsoft-Windows-Sysmon" Guid="{5770385f-c22a-43e0-bf4c-06f5698ffbd9}"/> <EventID>10</EventID> <Version>3</Version> <Level>4</Level> <Task>10</Task> <Opcode>0</Opcode> <Keywords>0x8000000000000000</Keywords> <TimeCreated SystemTime="2020-12-03T14:23:35.660463800Z"/> <EventRecordID>94211</EventRecordID

I am trying to create Ranger hive row-level filter policies using Rest API. I am truying to use this bellow Json format but it is creating just regular hive policy instead of row-level filter policy. Any clue on the best Jason format ? curl -iv -u USERID:PASSWD -H "Content-Type: application/json" -X POST http://HOSTNAME001:6080/service/public/api/policy/ -d '{"policyName": "RESTAPI_TEST","databases": "Test_databse","tables": "Test_table","columns": "*","udfs": "","description": "Hive Row Level Filter Policy","isEnabled": true,"isAuditEnabled": true,"rowFilterPolicyItems" : [ {"rowFilterInfo" : {"filterExpr" : "ROW_DVI = 02"},"delegateAdmin" : false,"accesses" : [ {"type" : "Select","isAllowed" : true} ],"users" : [ "USER_ID123" ]}]}'

I am running into a problem with the NiFi fetchgcsobject processor.. It fails when trying to retrieve files with spaces in the file name... Below is a sanpshot of the error. Interestingly the listgcsbucket works fine and lists the file. fetchgcsobject is where it fails. It also works fine with files that have no spaces in the file name. This works on version 1.10. Is this a bug on 1.12.1? I tried to encode spaces with %20 .. that doesnt help either.. Failed to fetch GCS object due to Blob BlobID(bucket=test-bbbb, name=outbound/test file.pdf) not found    

kindly anyone answer this question i have been asked in an interview i am preparing for interviews

After upgrading from Ambari 2.5.0.3 to 2.6.2.2, ambari-server.log file suddenly start recording following entries: 28 Nov 2020 16:48:52,372 WARN [ambari-action-scheduler] ActionScheduler:316 - Exception receivedjava.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat7 could not be found at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253) at org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:704) at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:417) at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat7 could not be found at org.apache.ambari.server.state.st

Hello everyone, I am trying to authenticate with  InvokeHttp NiFi Processor to Salesforce by using Bulk API v2 and with OAuth 2.0 : ck_sadchaouche_6-1606816870985.png The InvokeHttp Processor is configured as follow : ck_sadchaouche_7-1606817038511.png The Remote URL is made by using this syntax:   https://< mydomain >.salesforce.com/services/oauth2/token?grant_type=password&client_id=<client_id>&client_secret=<client_secret>&username=<username>&password=<password + key> As we are using the HTTPS protocol, we add a “SSL Context Service” by setting a new “SSL Context Controller Service”: ck_sadchaouche_8-1606817459585.png And the Post request is already tested and worked with Postman tool, by responding with an authentication token. The blocking point for this use case is that, with InvokeHTTP NIFI Processor we have not been able to get a response from the Bulk API ( the first image show that we never receive any response from Sale

I have 20 CSV files and I need to convert these files into parquet format before loading them to S3. I prepared schemas for 20 files(different schema for each file). For this to work I need to route all these files using route on attribute and process. This works but it create flow with 20 convert record processors. I want to know is there a way that I can store all these 20 schemas and I can use them according to filename with single convert record processor. Thanks.

https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/ch04s03s05.htmlI read above documentaion and I don't understand how to input username and password.I tried to input my cloudera username and password but it resulted 403 forbiddenhttps://username:password@archive.cloudera.com/p/ambari/ubuntu18/2.x/updates/2.7.5.0/ambari.list Please give me username and password.If cannot, please tell me how can I contact cloudera using e-mail.

Hi, I was trying to install Cloudera Manager 6 with CDH 6.3.0 for a 3 node cluster. I have downloaded and installed cloudera-manager-installer.bin in CentOS6 and installed cloudera server and agent. Then I have logged into CM using admin:admin to configure cluster and I stuck here at Add Cluster Configuration step as Upload Oozie ShareLib step failed with no log4j2 file found. PFB logs. Upload Oozie ShareLib Also, I have checked in /var/log/cloudera-scm-server/cloudera-scm-server.log and found the same error message there as well. Is this log4j2 something that I need to install manually before configuring cluster ? If so, please suggest me on how to do that. stdout and stderr for reference. PFB!! stdout:Mon Nov 30 08:10:40 UTC 2020JAVA_HOME=/usr/java/jdk1.8.0_181-clouderausing 6 as CDH_VERSIONCONF_DIR=/var/run/cloudera-scm-agent/process/61-oozie-OOZIE-SERVER-upload-sharelibCMF_CONF_DIR=Found Hadoop that supports Erasure Coding. Trying to disable

Hi everyone, Ambari :- 2.7.4HDP :- 3.1.4.0 I had done the ambari installation using local repositories with no Internet access for Ambari and HDP as mentioned in the official docs by creating directory structure /var/www/html and, untarred all the HDP and Ambari files into /var/www/html/hdp and /var/www/html respectively, as mentioned in the docs. :- [ Ref :- https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/getting_started_setting_up_a_local_repository.html https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/setting_up_a_local_repository_with_no_internet_access.html ] Then, I used python's Simple HTTP Server to host my local repository by running the command `python -m SimpleHTTPServer 2000` from inside /var/www/ instead of using Apache HTTP server as mentioned in the docs, to host my repositories in html directory on port 2000. Installation was done and services are running fin

Hi,    I have one question about the impact of change exist rack id.   we have a CDH 5 cluster with 6 data nodes(replication factor is 3), they are belong one rack rack0. and we add 1 data node to this cluster, but it is added rack rack1 by mistake.    now we met issue "Under-Replicated Blocks" due to only 1 node in rack1, it is need replicate 1 of 3 to node in rack1. but the disk capacity is not enough to storage 1 replica of all files.   Now, we have decomissioned the node in rack1, and want to change the node back to rack0. my question is what is the steps to change the node back to rack0 safely? any suggestions?   Thanks!  

reasons behind name node failure?

Hello, I would like to know if it's possible to revert or reinstall an older version of Cloudera Manager. I have the following case: the current CDH and Cloudera Manager are both running on version 5.16.2. I have a request to upgrade only Cloudera Manager to 6.3.3. Now, I would like to know if it's possible to revert/reinstall to CM 5.16.2 in case the CM 6.3.3 version will not work as expected on CDH 5.16.2. Thank you,  

Hi All,I'm using Cloudera Express 6.3.0. I'm trying to configure alert emails to be received by a list of users. While configuring properties, it is asking for mail server user name & password. We have SMTP enabled and we are triggering emails with sendmail command on the nodes where Cloudera is installed. But, here why we need to provide the mail server user name & password for receiving the alert emails on our mail id. I have found a solution for this in Cloudera community (https://community.cloudera.com/t5/Support-Questions/Why-does-Alert-Publisher-require-mail-server-login-amp/m-p/41003#M36254), but it is not working for my version of Cloudera. Does any know a turnaround for this.Please refer to the link above for the solution provided & please also see the picture attached for better understanding. When i click on Send Test Alert, i see this pop up & the email is not triggering Mail Server UN & PWD

Hi,   When i try to 'Generate Missing Credentials' in Cloudera Manager:  CM >Administration>Secuity>Kerberos Credentials , I get the following error 2020-11-20 00:13:31,793 INFO GenerateCredentials-0:com.cloudera.cmf.security.GenerateCredentialsCommand: Generating kerberos credentials using custom script /etc/hadoop/conf/keytabgen.sh2020-11-20 00:13:31,794 INFO GenerateCredentials-0:com.cloudera.cmf.security.GenerateCredentialsCommand: Creating credentials for roleId 1322020-11-20 00:13:31,794 INFO GenerateCredentials-0:com.cloudera.cmf.security.GenerateCredentialsCommand: Generating new keytab for spark/clouderadp02.ibm.cloud@IBM.CLOUD2020-11-20 00:13:31,805 ERROR GenerateCredentials-0:com.cloudera.cmf.security.GenerateCredentialsCommand: unable to create credential for role 132 due to:/etc/hadoop/conf/keytabgen.sh failed with exit code 1 and output of <<cp: cannot stat ‘/etc/hadoop/conf/keytabs/spark_clouderadp02.ibm.cloud@IBM.CLOUD.keytab’: No such file or d

Hi,I would like to run the oozie job only on the weekends.Can someone help me on how to configure this in the coordinator.xml?Thanks in advance

Hi,I am trying to register an environment using AWS but I get this error:  Data Lake creation failed: Operation timed out. Template install timed out with this command id: 1546333784  When I log into AWS I can see that three instances were provisioned1. cdp-env7-freeipa-10778-master02. datalake7-79244-master3. datalake7-79244-idbroker Where can I find more logs? What am I doing wrong? Thanks, Marcin

I have Nifi flow created (ListSFTP-->FetchSFTP-->PutHDFS) and have parameters like sourcedir,hostname,HDFS path ,etc..I would like to pass the values dynamically instead of maintaining the parameter values in parameter context so that if there is any value changes , i can do in properties file itself. I have tried to use lookup attribute processor but we do not have CouchDB in the cluster . Please suggest option to pass dynamic values to this parameters. Like in Java program , we pass properties files to maintain default values.@Nifi @NifiDev 

I have to retrieve information from all columns of a row based on a given value.the data isfname - Johnlname - Andrewssnumber - 008degree - MIT I have this query that returns only the degree column:scan 'CWRK', {COLUMNS => 'STUDENT', FILTER => "ValueFilter ( =, 'binaryprefix:MIT')"} this gives the result:student:008             column=STUDENT:degree, timestamp = ......... , value=MITI want to run a query that returns values for all 4 attributes (fname, lname, snumber, degree) for the row-number student:008 as that is the row that has the MIT value for degree key.Can this be done?

Hello, I am a newbie to Cloudera, so give me some slackI  assigned 4 cores and 8GB to my Cloudera VM in VMware players. It seemed to be working fine. After several suspends, restarts, and queries, the system becomes really slow and I cannot pinpoint why. I am not doing anything special, simple queries to small 100k tables. I even reinstalled the VM, restoring its speed, but with later degradation.  I was never able to get Cloudera Manager working, it just stays there.How can I diagnose what is making my system so slow? I am running this in an i7 8th gen with 16GB RAMNote: I usually get a message of an unresponsive script like , "Script: http://quickstart.cloudera:888…ery.nicescroll.7958d4c833ae.js:1990"  Thank you for your inputs.Cheers.

Dear community,We have a NiFi cluster installation of 3 nodes with 30GB of RAM and 15 CPUs each. The NiFi version that we are using is the 1.7.1 and we are running 43.000 NiFi processors in the cluster.The question is if there is a known limitation of the amount of processors that we can run in the same cluster considering that we can scale horizontally and also add more CPU and memory.Also if somebody is running more processors in the same cluster or have experience in this topic that would be a great piece of information.Thanks a lot for any advice on thisBest Regards!Juan Pablo