Q&A

Connecting to impala using one of the BI tools with jdbc string on port 21050The connection is successful but drops intermittently for some time with below errorjava.net.SocketException: Connection or outbound has closedThe firewall is open, there is no network issue between two servers 

Hi，I have now deployed NiFi on my Kubernetes cluster, and for ease of learning, I modified the contents of the start.sh script in the scripts folder to enable HTTP access. I set all certificate-related content to be empty.Now, I am encountering an issue: when I try to change the environment variable "NIFI_CLUSTER_IS_NODE" to "true", my container fails to start. Strangely, there are no errors in the logs.When such content appears in the logs, the container stops and attempts to restart, repeatedly going through this restart loop.It looks all right, doesn't it? But after this the container just stops.I am wondering if this is because I enabled HTTP access?By the way, can I start a NiFi cluster using the embedded ZooKeeper? I didn’t see an option to configure "nifi.state.management.embedded.zookeeper.start" in the environment variables of the NiFi docker image.I hope to get your help, thank you very much！

2024-11-19 04:52:41,378 WARN [Monitor Processor Lifecycle Thread-1] o.a.n.controller.StandardProcessorNode Timed out while waiting for OnScheduled of ExecuteScript[id=acb441ba-c36b-1fdd-53f2-3a4821d43833] to finish. An attempt is made to cancel the task via Thread.interrupt(). However it does not guarantee that the task will be canceled since the code inside current OnScheduled operation may have been written to ignore interrupts which may result in a runaway thread. This could lead to more issues, eventually requiring NiFi to be restarted. This is usually a bug in the target Processor 'ExecuteScript[id=acb441ba-c36b-1fdd-53f2-3a4821d43833]' that needs to be documented, reported and eventually fixed.

Hi I have a new line character in the data, which I want to process using apache Nifi …since the new line character is in data itself, the data is getting spilled over in new line . Please suggest the solution of the problem

Logged SELECT statement targets table with 10736979 rows. What can the reason for this error? Is it related to query timeout? What is the default driver query timeout? prepareStatement has been running for around 3 hours before driver thrown the error.  Caused by: java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:Invalid session id: 854422e228b99fe7:8a73e8f716a58fbf), Query: SELECT `type_enr`, `num_enr`, `num_carte`, `seq_carte`, `type_carte`, `cd_pdt_pilotage`, `code_tarif`, `num_cpt`, `letr_cle`, `numero_personne_host`, `nom_porteur`, `typ_porteur`, `autoris_comerc`, `statu_crt`, `dt_statut`, `dt_saisie`, `dt_vente`, `dt_eche`, `dt_der_fab`, `dt_der_fact`, `code_renouv`, `code_suivi`, `date_suivi`, `nbr_recalcul`, `dt_recalcul`, `nbr_relance`, `dt_relance`, `opt_releve`, `num_dern_rlv`, `matr_vend`, `cd_motif`, `usr_crea`, `trm_crea`, `usr

Hello!The source csv file is:          123456TextValue1          654321TextValue2where 123456 and TextValue1 are different values, separated by binary delimiter (\u0001)Similar, 654321 and TextValue2   have non-visible at Web-UI binary delimiter (\u0001)I use ConvertRecord for updating the delimiter from "\u0001" to ";"RecordReader is CSVReader with the following properties:- Schema Access Strategy: Use 'Schema Text' Property- Schema Text: #{test_schema}- Value Separator: \u0001- Treat First Line as Header: false- Ignore CSV Header Column Names: trueRecordWriter is CSVRecordSetWriter:- Schema Access Strategy: Use 'Schema Text' Property- Schema Text: #{test_schema}- Value Separator:   ;- Include Header Line: truetest_schema is{"type": "record","name": "test_schema","fields": [{"name": "FIELD_1","type": ["int","null"],"description": "FIELD_1"},{"name": "FIELD_2","type": ["string","null"],"description": "FIELD_2"}]

The document said that "To change attributes based on some condition, use the Advanced User Interface (UI) in the processor by clicking the Advanced button in the lower right corner.". But I cant find anything in Properties of UpdateAttribute 

Iam trying to connect hive database using hive-driver npm package in nodejs. But the session can't be established. If anyone please help me on this

Hi,I wanted to know if it is possible to disable kerberos once it has enabled in cloudera. I couldn't find any option to for kerberos in he configuration in cloudera.

we have cluster with 12 Kafka machines and 3 zookeeper servers on Linux servers- Kafka version is 2.7 version , ( broker and controller are co-hosted on the same PID )as known Kafka have 2 important logs and they are **server.log** and **controller.log**about **controller.log** , when we look on this log we can see the following words - "`Shutdown completed`" in the log[2024-08-20 21:42:01,582] INFO [ControllerEventThread controllerId=1001] Shutdown completed (kafka.controller.ControllerEventManager$ControllerEventThread)the first thinking when we see the messages about "`Shutdown completed`" - is like this message is "bad" and why controller stopped ...but when we look on all machines most of the machines have this message as - `Shutdown completed` (`kafka.controller.ControllerEventManager$ControllerEventThread`)**but on other hand** only one controller should be active from all brokers and maybe the messages as "`Shutdown completed`" are only indicate that controllers that are not ac

Hi Team,We are configuring the job in CML to run every 4 hours as shown below. Currently, the job is getting triggered automatically at 12 PM, 4 PM, 8 PM, and 12 AM. But the jobs are not getting triggered at 4 AM and 8 AM. Also, it's not getting triggered over the weekend. Please help us with your expertise if there is any additional setting or any admin configuration that we need to validate here. Thanks in advance! 

Hello,Yesterday, I purchased the CDP-2001 CDP Administrator - Private Cloud Base Exam, but I haven’t yet received the follow-up email to schedule my exam. I only received the payment confirmation. All of my colleagues received their scheduling email right away. Could you please look into this for me?Thank you!

HiI'm new to NiFi and exploring my options.We are planning to use NiFi as a File Movement platform, where we will be transferring thousands of files everyday of sizes ranging upto 50GB without any extraction, transformation or loading. We just need to pull the file from one location and push to another location using SFTP and/or S3.Is NiFi a right tool to use to do this activity ? Thanks  

Hi,         I am trying to run simple query on Hive with execution engine as Tez,Here is the query     Insert into table test values('a', 'b');there are only two columns in table and both are string columns.If i insert into int column  then it works fine.also if i insert through load data command, this also works fine with string command. Below is the error that i am getting in yarn logs: ERROR : Status: FailedERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1731065811939_0001_1_00, diagnostics=[Vertex vertex_1731065811939_0001_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed, vertex=vertex_1731065811939_0001_1_00 [Map 1], java.lang.RuntimeException: Failed to load plan: hdfs://localhost:9000/tmp/hive/hadoopuser/a86dedab-ce0f-4f06-8056-c37aef3cca55/hive_2024-11-11_12-26-32_369_2945486912443049901-2/hadoopuser/_tez_scratch_dir/8d991848-0c76-4ffa-a82e

Hi,I want to find the space required for storing Kudu Table Backup using spark-submit command.How it is related to Kudu Table On disk size and how to calculate the space required to store the Backup of Kudu table?I am using the following command as reference to run Kudu backup for a table - spark-submit --class org.apache.kudu.backup.KuduBackup [***FULL PATH TO kudu-backup2_2.11-1.12.0.jar***] --kuduMasterAddresses [***KUDU MASTER HOSTNAME 1***]:7051,[***KUDU MASTER HOSTNAME 2***]:7051 --rootPath file:/// [***DIRECTORY TO USE FOR BACKUP***] impala::[***DATABASE NAME***].foohttps://docs.cloudera.com/cdp-private-cloud-base/7.1.7/kudu-recovery/topics/kudu-backing-up-tables.htmlTo check the Kudu table On disk size I used the statistics kudu command -kudu table statistics <master_addresses> <table_name>https://kudu.apache.org/docs/command_line_tools_reference.html#table-statisticsI have a table with following statistics -TABLE sampleTable1on disk size: 31667556474live

I have a total of 3 tablet servers.Nodes with 3 tablet servers installed all have 100% CPU usage.The cause seems to be the operation of inserting data with java's kudu api.I'm curious as to how to fix it and what the exact cause is.Help Me please!

Hello guys.I'm trying to use kudu as my data store.But my kudu leader master keep dead by OOM.Follower masters and all tablet uses very few memory. Leader master's memory usage Follower master's memory usageTablet Server's memory usage  Kudu Master spec is:3 masters16GB physical memory per master32GB swap memory per masterKudu Tablet spec is:8 tablet servers16GB physical memory per tablet serverMetrics says generic_heap_size is 35335258112(35335 MB, 35.34 GB).I tried to debug memory usage, but pprof don't catch all memories.(pprof shows only 2063 MB)  Does anyone know reason? 

We faced the problem on our production kudu cluster. The hard disk with wal catalog was failed on the tablet server. We install new disk and clear data directory according to Kudu documentation https://kudu.apache.org/docs/administration.html#rebuilding_kudu . After starting the failing tablet server we have seen that kudu ksck displayed two instance tablet server for one server with different UUID. One of this server had status "WRONG SERVER_UUID". Why may the error occure? Are there any ways to avoid it? Is there way to solve the problem without restarting master server? Also found the command "kudu tserver unregister" for removing tablet server with wrong UUID but we hadn't found this step in documentation. Steps for reproduce the similar problem:1.Install Apache Kudu Quickstart.Instructions - https://kudu.apache.org/docs/quickstart.html#_bring_up_the_clusterClone the Apache Kudu repository using Git and change to the kudu directory:$ git clone https://github.com

Hello,In a CDP 7.1.6 + Cloudera Manager 7.3.1 cluster.3Masters +3Workers.I'm getting the error all the time: Corruption: master consensus error: there are master consensus conflicts This is the cluster ksck:Master SummaryUUID | Address | Status----------------------------------+--------------------------+---------5620e4a103894151b7bdee5e436f37d8 | master-2.local | HEALTHY9cea3b56cc9b4be4846a02c0d89be753 | master-1.local | HEALTHYa98a1f26d0254293b6e17e9daf8f6ef8 | master-3.local | HEALTHYAll reported replicas are:A = 9cea3b56cc9b4be4846a02c0d89be753B = 5620e4a103894151b7bdee5e436f37d8C = a98a1f26d0254293b6e17e9daf8f6ef8The consensus matrix is:Config source | Replicas | Current term | Config index | Committed?---------------+--------------+--------------+--------------+------------A | A B C | 10120 | -1 | YesB | A B* C | 10120 | -1 | YesC | A B* C | 10120 | -1 | Yes It seems the A node is not voting, this is the log output:W1111 11:12:00.526211 18688 leader_election.cc:334

Hello, I'm running a Kudu 1.17 cluster with 1 master and 1 tserver configured with the following flags: --block_cache_capacity_mb=256--use_hybrid_clock=false--fs_data_dirs=/mnt/data0/data--fs_metadata_dir=/mnt/data0/metadata--fs_wal_dir=/mnt/data0/tserver--log_container_metadata_runtime_compact=true--rpc_dump_all_traces=true--rpc_service_queue_length=200--rpc_authentication=disabled--rpc_encryption=disabled--webserver_doc_root=/opt/kudu/www--tablet_history_max_age_sec=300--tserver_master_addrs=kudu-masters-0.kudu-masters-db-services-headless:7051--heap_profile_path=/tmp/kudu.1--unlock_experimental_flags=true--unlock_unsafe_flags=true--memory_limit_hard_bytes=2073741824--stderrthreshold=0 I'm currently running 878 tablets on that single tserver.For my workload, I'm executing around 10 concurrent scans via python kudu client on different tables, all having a partition by hash on the primary key. On the background, some write operations are performed from time to time, inse

Hi all,Am new to Hadoop, so while working i found this alert where the users getting this error intermittently. As i have checked logs it says due to memory issue job not able to run but when i checked memory its not full.What is the solution for this? We are facing impala connectivity issue.[ERROR] 2024-11-14 05:32:42,548 com.tcl.move.dao.JDBCConnection - Exception Occurred while creating a connection java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500605)  Error occurred while opening a session with the server. No additional detail from the server regarding this error is available. Please ensure that the driver configuration is compatible with the server configuration. This type of error can also occur when the server is too busy to handle the request. Please try again later.

hi guys. want to ask it is support if CDH 5.15.x integrate with Hitachi Object Storage (HCP)?and if CDP it is support integrate with Hitachi Object Storage (HCP) ?i read this post below. but didnt know if is supported or not.https://community.cloudera.com/t5/Support-Questions/Integration-with-Hitachi-Object-Storage-HCP/td-p/63398 Thanks

Hi, My hadoop cluster use HDP3.1.5-152.0, I start httpfs in cli command, and my webhdfs is work fine. But when i send request to the httpfs service , i got 500 code error like this:15:50:39,064 WARN ServletHandler:632 - /webhdfs/v1/userjava.lang.IllegalArgumentException: Empty key at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96) at org.apache.hadoop.security.authentication.util.Signer.computeSignature(Signer.java:93) at org.apache.hadoop.security.authentication.util.Signer.sign(Signer.java:59) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:587) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751) at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1619) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.hadoop.http.NoCacheFilte

  I only have access to RHEL 9.4 as my operating system and cannot use any other version. However, according to the Cloudera support matrix, there is currently no version of CDP (Cloudera Data Platform) that is certified for RHEL 9.4. I am trying to find out if there is any workaround or way possible to install any version of CDP on this specific OS version.

I have a folder in S3 bucket let us say Test/Now the Test folder has multiple sub folders and each of these subfolders have multiple files.Now I want to list only the sub folder names within the Test folder and I don't want the names of contents within these sub folders.I configured the processor like this: prefix: Test/With this configuration, I get all the subfolders along with their contents.When I give the delimiter as /, then I don't get any result.