Solved

Hi, I would like to know where can I download a CDH open-source for tests : for Ubuntu (or Centos)?   Many thanks!   BR, jludac  

Hello,I need to move one big table (16 GB)  from one oracle database to another. When I try to move table with more then 1 GB data, my nifi GUI crashes. Can you suggest me how to do it? Thanks' in advance Lejla 

Hello,When trying to access Hbase through Hue i have this error in the log WARN org.eclipse.jetty.server.HttpChannel: handleException / org.apache.hadoop.security.authorize.AuthorizationException: User: hbase is not allowed to impersonate hueknowing that i had set a Thrift Server instance on Hbase and it is running okay. 

 Now I used SSL with Kafka nodes that made from Self cert but it cannot pass audit by infra team. I need to adapt company cert without using self cert to use with syntax look likes as follow : openssl req -new -sha256 -nodes -newkey rsa:2048 -config <(cat <<-EOF[req]default_bits = 2048prompt = nodefault_md = sha256req_extensions = req_extdistinguished_name = dnSamplePassword123[ dn ]C=THST=BangkokL=ChaxxxO=xtac Public Company LimitedOU=Enterprise Service SupportemailAddress=john.doe@xtac.co.thCN = rs-xxx-hmb-201.xtac.dev [ req_ext ]subjectAltName = @alt_names [ alt_names ]DNS.1 = rs-xxx-hmb-201DNS.2 = rs-xxx-hmb-201.xtac.dev EOF) It cannot use all of these syntax, please help me how should I do or what I did it wrong??

Hi! I am trying to connect to a MySQL RDS instance in AWS with Apache NiFi on Linux Ubuntu. The MySQL database is behind a SSH tunnel which is working in a dedicated terminal. The MySQL database is accessible through another Linux terminal. $ ssh -i /home/bernardo/.ssh/sshbernardo3.pem -N -L 33061:<db_name>.<id>.eu-west-1.rds.amazonaws.com:3306 <user>@<ip_address> -p <port> I have uploaded and unzipped the JDBC driver from https://downloads.mysql.com/archives/c-j/ The file java.sql.Driver is located in the directory /usr/share/java/META-INF/services in Linux I have installed the jdbc connector in Linux and set up the user to use JDBC$ sudo apt-get install libmysql-java$ cd /home/bernardo$ gedit .bashrc Added the following lines at the end of the file: # Setting up the user to use JDBCCLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java-8.0.26.jarexport CLASSPATH $ source .bashrc In NiFi I have used a D

Hello community,  I would like to configure a StandardSSLContextService for kafka to read the password for the keystore from an environment variable. Sadly it is not possible to use the "Expression Language" for this sensitive property. Background: We are using an operator to provision kafka users. The operator generate a kubernetes secret with the keystore and keystore password. The kubernetes secret is mounted into the nifi container and can be referenced in the StandardSSLContextService. Currently we have to add the password for this keystore manually. But when the kafka user certificate expires and is rotated, the password of the keystore changes too. I am looking for a solution to read the password automatically, e.g. by setting it as environment variable in nifi. Any idea how I could achive this? Is there a mechanism I am missing?Thank you a lot for your help,Constantin 

I am getting a 502 Bad Gateway error when I try to load the Apache Ambari UI at localhost:8080.

My Nifi instance of 5 node is not starting with the below error , please help on resolving this , I have tried deleting the repos and setting nifi.flowcontroller.autoResumeState=false in nifi.properties , start nifi service then also getting the same error , please help on this I have also increased the heap size that also did not resolve this 2021-11-24 14:20:43,413 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError: nullat java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:123)at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:117)at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)at org.apache.nifi.util.file.FileUtils.copy(FileUtils.java:304)at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:110)at org.apache.nifi.controll

My environment contains two separate and independent setups. My development environment runs all software on a VM with everything being local. Either using localhost or my internal IP address. My production environment runs with everything being in a pod. Each having a hostname. Currently when I create a template from my development environment and move it to my production, I have to go through all the processes and change the endpoints so that it will continue to run and process. Is there a method in NiFi, that I can define the endpoints on the system that contains the NiFi service, that the processes can rely on to get the proper value for the endpoints. I can move the templates between my development and production and visa-versa without having to reconfigure each process? Thank you.

Guys, I just wondering does the Data Labeling and Data Classification works whenever the data travels around? My main goals is to protect the data is being misused. For example I can block any data that already labelled and classified as Confidential being printed or copied to USB drives. Appreciate your kind insight for this.

Hi, I noticed that in NiFi rest api documentation for version 1.9.0 the parameter context endpoints have been removed, even though they existed in version 1.15.2Furthermore, I am using NiFi version 1.13.2 and I can see that in my browser's debugging tools, in the network tab, whenever I create a parameter context, a post request is sent to http://localhost:8080/nifi-api/parameter-contexts: and the following payload is being sent :{ "component": { "name": "aaaaaaaaaaa", "description": "aaaaaaaaaaaaaaaa", "parameters": [] }, "revision": { "clientId": "3d3697c2-017e-1000-0610-31b28073cf9b", "version": 0 }} The parameter context is successfully created. However when I attempt to send a post request from an external API or even hit the previously mentioned endpoint through the browser using the same payload, I get:HTTP 405 Method Not Allowed My questions are:1. Is anyone aware whether these endpoints have been removed/r

Hi, I want to create a hive table which will store data with orc format and snappy compression. Will power bi be able to read from that table? Also do you suggest any other format/compression for my table?

I have an ETL flow which transfers data from a hive table to another through pyspark. The tables are partitioned. Although I see that in the partition's path in HDFS there are small parquet files. I want to ask:1)How can I merge these files?2)Is there any max size or recommended size for hive partitions?

Hello everybody, Recently my DBA team called me informing that Sqoop was sending queries to the database without parameter binding. Looking in the Sqoop documentation I got confirmation of the use of Prepared Statement by Sqoop only in the export command, with --batch parameter. With import command there is not this option to use. Could you help me identify how I could (if possible) use Sqoop's parameters to make it use binding variables in import querys? Any thoughts will be appreciated.Thanks! PS: Above is the description of the --batch parameter.--batch Use batch mode for underlying statement execution.

Accessing the URL on port 18088 returns a ERR_SSL_PROTOCOL_ERROR error in Chrome. history server logs have the following: 11:25:29.130 AM WARN HttpParser Illegal character 0x16 in state=START for buffer HeapByteBuffer@1af6bd16[p=1,l=517,c=8192,r=516]={\x16<<< ...}11:25:29.130 AM WARN HttpParser Illegal character 0x16 in state=START for buffer HeapByteBuffer@21158554[p=1,l=517,c=8192,r=516]={\x16<<< ...}11:25:29.130 AM WARN HttpParser bad HTTP parsed: 400 Illegal character 0x16 for HttpChannelOverHttp@2390a367{r=0,c=false,a=IDLE,uri=null}11:25:29.131 AM WARN HttpParser bad HTTP parsed: 400 Illegal character 0x16 for HttpChannelOverHttp@7843db5f{r=0,c=false,a=IDLE,uri=null} Note: some lines truncated to avoid exposing data. There was a similar post regarding Schema Registry UI in Ambari but the solution doesn't appear to be applicable here (though I suspect the issue is similar). Any thoughts?

I use version CDP Private Cloud Base7.1.4(Private Cloud) now, but I don't find any information about high availability in the document; But in Enterprise 6.3.4 (https://docs.cloudera.com/documentation/enterprise/latest/topics/admin_cm_ha_overview.html) and Cloudera Manager7.5.1(Private Cloud • PCE 1.3.1) Version (https://docs.cloudera.com/cloudera-manager/7.5.1/managing-clusters/topics/cm-high-availability.html) can see that there is highly available information. Which scheme should I follow?

When CDH cluster adds sentry service, an error appears in the above picture, indicating that HDFS or core configuration is required (but this service also needs to be configured with core.defaultFs). I want to know whether HDFS is generally necessary for sentry service, but I don't see relevant  instructions (Prerequisites) on the official website (Before You Install Sentry | 6.3.x | Cloudera Documentation). Can anyone help provide relevant documents?  because I need to convince customers that HDFS is necessary.  Thankssss!!!

We have a lab cluster. We moved the MySQL DB to Oracle Cloud. The on-prem DBs remained but are now several months old. I am trying to revert back to the on-prem without copying the data back from OCI. The loss of changes in the several months does not matter as this is a lab cluster. The issue is the license was updated in between these DB changes and now SCM will not start. FactoryBean threw exception on object creation; nested exception is java.lang.RuntimeException: Upgrade not allowed in Express or Trial mode What table has the license info. I can grab just that table from OCI?

I am new with Cloudera and we are currently running CDP Datalake and CDP Datahub 7.2.6. I am trying to follow the instruction to do the mitigation for log4j (TSB 2021-545) but I could not follow the instruction to navigate to Cloudera Manager. I can go as far as the Cloudera management Console but could not see the Cloudera Manager. Can you please help me or send me the URL. 

Hi,I need some assistance with Kudu, We have 2 tables that are consuming more space on our Kudu Cluster however, we could not find these tables under the database. Would need some guidance/assistance on a way to check where these tables are located under which Db. I can see the table in Kudu Web UI, however, when I try to query it via Impala I get the error: [My_server:21000] kwid> show create table ALARMS_COUNTERS_2021_11_01;Query: show create table ALARMS_COUNTERS_2021_11_01ERROR: AnalysisException: Table does not exist: kwid.ALARMS_COUNTERS_2021_11_01 If I run “show tables” against the DB I get a list of all the tables, but this table is not in the list. Thanks Wert

Hello,I've CDP 7.3.1 private cloud cluster (3 master, 1 manager, 1 edge & 5 slaves) . I'm wondering how can I secure a backup/restore point for it. Is it technically right and "supported by Cloudera" if I take a VM snapshots on machines level for master nodes ? Should it be cold (offline) backup (stop or pause the cluster operations) to ensure of consistencies between distributed services on master nodes? Thanks,Mahmoud

Hello,We are using Cloudera V5.1.1 with log4j v1.2.17, without option to upgrade Cloudera to later version at this moment. Does anyone has suggestion or solution how can we upgrade log4j component to latest version(2.16.0) on CDH V5.1.1?  Thanks 

Dear Community,We are planning to install latest version (2.2.0) of the Apache Atlas, where can I find related versions of the core components of Atlas, like -HBase, Solr, Graph Engine.This is mainly to evaluate Log4j vulnerability 

Hi folks, We've just install CDP Public Cloud over Azure and we are working with data hubs cluster. We have a brand new environment and data hubs. We are unable to access to Nifi UI with any user except the account owner. The users have privileges in the environment(EnvironmentUser) and also in the data hub (DHUser) What should I do next? is this related with ranger? Thanks in advance

Hello.I want to disable logging for some of the policies in Ranger (Ranger 1.2.0, HDP-3.1.4.0-315) I edit the policies to switch off the Audit Logging button and when I save this configuration in Ranger Audit I see that nothing changes (all the logs from the policies that I disabled logging are visible).Why can this be so?