Solved

Hello,Following https://docs.cloudera.com/data-engineering/cloud/api-access/topics/cde-api-get-access-token.html I am unable to get a response from the curl command, just hanging. Maybe it would finish with a timeout.Seems this endpoint is not accessible. Any help to troubleshoot is welcomed.

Hello,I'm trying to figure out whether Azul Zulu versions 8.58.0.13, 11.52.13 or 17.30.15 are supported by Cloudera and compatible with CDP Private Cloud 7.1.7, CM 7.4.4 and CDSW 1.9.0 ? Thanks and regards,Olek

Hello, I have completed registration for CDP Public Cloud(60 days trial) But still I am waiting for access details from Cloudera end. How much time it will take to get the required details? Can anyone please help me with this. Thanks 

I want to copy a table from HBase to HBase across clusters by using copytable command, by default it is set to 1 mapper and scans all rows which cause a timeout. Are there any options available for the HBase copytable command in a way to optimize performance? without specifying any parameter to HBase-site.xml. hbase org.apache.hadoop.hbase.mapreduce.CopyTable --peer.adr=myserver:/hbase --new.name=<<tablename>>  <<tablename>> 

Hi all, I am passing a FlowFile with several custom attributes to my ExecuteScript processor (running python). I want the ExecuteScript to create two new FlowFiles, each having their own set of attributes (the values are to be retrieved from the input FlowFile). Problem is, I am unable to add any attributes to the two new flowfiles and it throws the error "org.apache.nifi.controller.repository.StandardFlow" object has no attribute "putAttribute".  flowFile = session.get()if flowFile != None: ... destFlowFile = session.create(flowFile) # doesn't work destFlowFile.putAttribute(destFlowFile, "logMsg", "Testing Msg") destFlowFile.putAllAttributes(destFlowFile, backupAttributes)  I've followed the examples listed in this guide https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-1/ta-p/248922, hence I wasn't expecting this to not work. Is it just not possible to add attributes to created FlowFiles and if so, are there any a

Hi,Wanted clarification/guidance related to finalize process for HDFS. Could we keep using the cluster while the finalize upgrade process is ongoing e.g., data is being written to hdfs while finalize process continues in the background OR would we need downtime for the same? ThanksWert

I am trying to get the data from elasticsearch from processor group via port. I want to apply some python code on the output data. How can I read the input and execute the code?

Hello,after some operation in NiFi GUI (parameter value change in a parameter context), the GUI got sort of stuck.A controller service show status disabled in GUI, however, I cannot enable or delete a controller service, as the NiFi claims it is running on a node.Do you have some Idea how to get rid of this situation?Best regardsJaro

Hello experts. I have a text file reading into Nifi flows. Part of my flow is splittext > extracttext. I have split the text as line by line using SplitText Processor. Daily file is generated with the process output reads as below: 20211129-04:00:26 RG1287.kla EOF mark not found!20211129-04:00:55 RG9625.kla EOF mark found!....and so on How can i configure extracttext processor to result values as attributes for the each flow file.The values would be:date:20211129 Group 1:RG1287.klaGroup 2:EOF mark not found! Appreciate the help. Thanks! 

Hey, we are running a Nifi cluster that connects to a Hive database to pull some data. After a period of time we get the below error.       : java.net.SocketException: Broken pipe (Write failed).: java.sql.SQLException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe (Write failed)       Our current solution is to restart the Hive Controller services and this refresh's the connection and the process works again, this is obviously not ideal for a production implementation. We have also set the Kerberos user's timeout to be a large number of days in the hive-site.xml but the issue still persists. We are running Nifi version 1.12.1. Any help is very much appreciated.

Hey,  I am using nifi to load data from hdfs to impala table. I am using ExecuteSQL with the following command: LOAD DATA INPATH '/user/nifi/20211115_nifi_test' INTO TABLE arcsight.`20211130_nifi_test` PARTITION (year=${year:toString()}, month=${month:toString()}, day=${day:toString()}) And I am getting the following error:AnalysisException: Value of partition spec (column=year) has incompatible type: 'SMALLINT'. Expected type: 'STRING' Anyone can help?  

I see some query submitted to Impala with "LIMIT 0' at the end which obviously do not return any rows. It seems like these queries are coming from IBM Datastage. I am from Production support team and I monitor such queries. i know we can use limit 10 or so to limit the number of rows returned but why on the earth someone wants to limit to 0? Is it just to check the syntax of query or connection or something? Before I question the team who submits such query, wanted to double check if there is any reason one would like to say limit 0 at the end.  official documentation at official impala document  does not say any thing about it. 

 Is there any way to cast string to map in Impala?e.g: select cast("A:1,B:1,C:1") as map<string, int> 

Hi every one, I encountered the following error when I ran a cde cli command launching spark or airflow job.  ERROR Start TGT gen failed for user: my_user, error: rpc error: code = Internal desc = java.lang.RuntimeException: com.cloudera.thunderhead.service.tgtgenerator.services.KeytabException: unable to get keytab fileName  Have you ever seen this before? To resolve this issue, I just ran my command again and it passed after some tries. Does someone have a solution to pass this command at the first time?My command seems:  cde job run --name my_job_name --variable var1=$(date +'%Y%m%d%H%M%S') --variable var2=$(date +'%Y%m%d%H%M%S') --variable var3=100 --variable var4=yyyyMMddHHmmss  Thank you

keyidtest161289462valid50578283validUGM49O3validI am using below query but still getting wrong results.Select keyid,CASEWHEN keyid like '^[A-Za-z]*%' THEN 'invalid'else 'valid'end as testfrom tableI am expecting belowkeyidtest161289462valid50578283validUGM49O3InvalidAppreciate your help in advance.Thanks

Hi, I'm trying to develop a Jolt Spec for the following transform:JSON Input:  [ { "randomkeyname1": "randomvalue1" }, { "randomkeyname11": "randomvalue11", "randomkeyname22": "randomvalue22" }, { "randomkeyname111": "randomvalue111", "randomkeyname222": "randomvalue222", "randomkeyname333": "randomvalue333" }]  JSON Output:  [ { "data": [ { "key": "randomkeyname1", "value": "randomkeyvalue1" } ] }, { "data": [ { "key": "randomkeyname11", "value": "randomkeyvalue11" }, { "key": "randomkeyname22", "value": "randomkeyvalue22" } ] }, { "data": [ { "key": "randomkeyname111", "value": "randomkeyvalue111" }, { "key": "random

Hello.I'm trying to add users to Ranger via RESTAPI, but I could only add one singular user at a time.This is the command I'm using with a json file curl -u admin:$PASSWORD -i -X POST -H "Accept: application/json" -H "Content-Type: application/json" https://$RANGER_URL:6182/service/xusers/secure/users -d @users_RESTAPI.json -vvv And the json file as the following{ "name":"user_1",  "firstName":"",  "lastName": "",  "loginId": "user_1",  "emailAddress" : "",  "description" : "",  "password" : "pass123",  "groupIdList":[3],  "status":1,  "isVisible":1,  "userRoleList": [ "ROLE_USER" ],  "userSource": 0},{ "name":"user_2",  "firstName":"",  "lastName": "",  "loginId": "user_1",  "emailAddress" : "",  "description" : "",  "password" : "pass123",  "groupIdList":[3],  "status":1,  "isVisible":1,  "userRoleList": [ "ROLE_USER" ],  "userSource": 0} Only the first us

I get following health issue.   The host's NTP service did not respond to a request for the clock offset.   How do I resolve it?   Thanks, Kewal

I have a hive insert statement which by default will use all available resources in YARN as it is reading a large volume of data.   I am happy for the query to take longer and use less resources so that other users can also have access to compute resources.   I don't want to set up YARN queues as this is an unusual query and so don't want to permanently restrict the cluster.   If I was using Spark can do this quite easily with setting a number of executors. Is there a hive config that allows me to do this at a query level.   I have looked at various other posts such as those below, but nothing seems to allow this.  https://community.cloudera.com/t5/Support-Questions/How-to-control-number-of-containers-in-a-hive-query/td-p/297734  https://community.cloudera.com/t5/Support-Questions/Is-there-a-way-to-set-minimum-maximum-number-of-containers/m-p/190660#M152749  https://community.cloudera.com/t5/Support-Questions/Can-I-limit-the-number-of-containers-alloc

안녕하세요 Streams Replication Manager 를 통해서 아래와 같이 미러링을 하고 있습니다.- Topic 갯수 : 95- AVG Throghput : 319B/s- AVG Replication Latency : 19.6sec- Status : WARNING 위와 같이 계속 WARNING을 표시가 되는데 원인과 조치방법을 문의 드립니다. 

안녕하세요. 아래와 같이 Stream Replication Mananger에서 Error 로그를 발생하고 있습니다.max.request.size 를 어떻게 설정해야 하는지 답변 요청드립니다. ERROR WorkerSourceTaskWorkerSourceTask{id=MirrorSourceConnector-3} failed to send record to m16.ftl.adapter.ic.m16.edp.edes.hub.cmndcol:org.apache.kafka.common.errors.RecordTooLargeException: The message is 1270478 bytes when serialized which is larger than 1048576, which is the value of the max.request.size configuration.ERROR WorkerTaskWorkerSourceTask{id=MirrorSourceConnector-3} Task threw an uncaught and unrecoverable exceptionorg.apache.kafka.connect.errors.ConnectException: Unrecoverable exception from producer send callbackat org.apache.kafka.connect.runtime.WorkerSourceTask.maybeThrowProducerSendException(WorkerSourceTask.java:263)at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:232)at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184)at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)at ja

Hello everyone! I need a help in Jolt transform.I have array, that consist from many json objects. (https://jolt-demo.appspot.com/ ) [  {     "table_name": "table_vd",     "scratch_name": "l1_sample_1_1",     "scratch_order": 1  },  {     "table_name": "table_vd",     "scratch_name": "l1_sample_1_1",     "scratch_order": 34  },  {     "table_name": "table_vd",     "scratch_name": "l1_sample_2",     "scratch_order": 3  }]  What transformation of Jolt needs to be done to get the following result? Now i have next Jolt construction. [  {     "operation": "shift",     "spec": {        "*": {           "table_name": "table_name",           "*": { "@0": "data[0].&1" }    &nb

Hi list, I’m facing a weird problem I can’t resolve or even understand with my secured nifi cluster. Below is the situation.We have setup a secured nifi cluster with 3 nodes, say node1,node2 and node3.For each of theses nodes, we’ve manually created a SSL certificate signing request (CSR) (using a password protected private key) to be signed by our internal CA.Once we’ve get the certificates signed, I’ve installed each node certificates following this procedure: Add the full certificate chain (root + intermediate certificates) into the signed certificate.cat nifi-nodeX.pem cert_chain.pem  > full-nifi-nodeX.pemCreate a PKCS12 certificate using private key (.key) and full signed certificate (.pem)openssl pkcs12 -export  -in full-nifi-nodeX.pem  -inkey nifi-nodeX.key  -out nifi-nodeX.p12 \                    -name nifi-nodeX -passin pass:"XXXXXX" -passout pass:YY

Hello everyone!I have many json files like this:{"table_name" : "train_vd","data" : [ {"battery_power" : 1954,"clock_speed" : 0.5} ]} {"table_name" : "train_vd","data" : [ {"battery_power" : 842,"clock_speed" : 2.2} ]}... I used the MergeContent and MergeRecord processors and used the table_name field as the Correlation Attribute Name (i have ${table_name} attribute). However, this does not work and the result is as follows:[{"table_name" : "train_vd","data" : [ {"battery_power" : 509,"clock_speed" : 0.6} ]}{"table_name" : "train_vd","data" : [ {"battery_power" : 842,"clock_speed" : 2.2} ]}]... However, I want to get the following result:[{"table_name" : "train_vd","data" : [ {"battery_power" : 509,"clock_speed" : 0.6},{"battery_power" : 842,"clock_speed" : 2.2}]}]May you tell me how to solve this problem? Need i use a complex Jolt transformation or to configure the incoming Avro schema in the MergeRecord processor, so that then everything is combined using a s

Hi Team, I've a MergeContent processor  to process the two files (file1, file2) which are coming from two FlowFiles. But here the File2 comes sometimes not regularly , whereas File1 comes on daily basis.so this process two Files successfully. But How we can process the File1 whenever File2 is not present? is it possible through MergeContent Processor or is there any different approach (routing FileFlow) we need to use?2 FlowFiles Routingmerging two FlowFiles   I've tried processing File1 alone but the flow got stuck in queue in between UpdateRecord & MergeContent.