Solved

Hi Team, I bought CDP Public platform. And I have user and my purpose is to create Data-Hub Cluster and with my custom definition and template I have created a data hub cluster. And I see list of URL for access like Cloudera Manager, Hue and some. If I click cloudera manager URL it directly login to Cloudera Console page without any authentication. But I want to create multiple user with Cloudera access, Hue UI access. I have created machine user and I have created workload password also but how to access the UI with this User as there is only one UI access which I created initially but for the rest I'm not able to create or access console page for any services.Please help me with this. Regards,V

未处理的表格：{    "日期":"2022-09-03 19:23:53",    “设备名称”：“032-013”，    "device_id":"1144",    “传感器”：[        {            "sensor_id":"5836",            “价值”：“1”，            “传感器名称”：“设备状态”        },        {            "sensor_id":"5842",            “价值”：””，            “传感器名称”：“12345”        }…         ],    “状态”：“1”}最后结果： {    “日期” ：“2022-09-03 19:23:53” ，    “设备名称” ：“032-0

Hi I am very new to nifi and try to install and configure the system. I started nifi.sh start and it shows  Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/HomeNiFi home: /Users/daniel/Downloads/nifi-1.15.3 Bootstrap Config File: /Users/daniel/Downloads/nifi-1.15.3/conf/bootstrap.conf And when i enter nifi.sh  status, terminal shows: 2022-02-26 09:31:33,238 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is currently running, listening to Bootstrap on port 52929, PID=74313 I changed nifi.web.http.port in nifi.properties to 8080. But still I do not see any on localhost:8080. In the log file there is no message that server has started on 8080 port.  I have no idea what problem caused the issue. Please advise me. Thanks

Hello Guys,How to Extract recent past Saturday based on today date and is there any expression language to get by update attribute 

I am a newbie to Nifi Jolt transform.  I've been looking through the document, but still cannot figure out how to do a simple Jolt Transform.Input:{"id": 123456,"manager_id": 98765}desired output:{"reportTo": [{"userId": 98765}]} I've got the part to change the manager_id to userId by the following Spec.  But I was stuck how to make it another layer.  Any insights/help to solve this? [{"operation": "shift","spec": {"manager_id": "userId"}}}]

Hi , We're having three node mangers , out of which two not working due to the below error:resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh su yarn -l -s /bin/bash -c 'ulimit -c unlimited; /usr/hdp/3.1.5.0-152/hadoop-yarn/bin/yarn --config /usr/hdp/3.1.5.0-152/hadoop/conf --daemon start nodemanager'' returned 1. ERROR: Cannot set priority of nodemanager process 30232 Please suggest some solutions.

https://github.com/GitJason1027/cm_api_programs/blob/main/bdr_build_plan.pyHere is my program to create schedule in BDR function.（CDH professional version）But some options I cannot find in the cm_api interface,s such as "Permanently Delete", "Copy HDFS File " options. Is someone can tell me how to deal with it ? https://github.com/cloudera/cm_api/blob/master/python/src/cm_api/endpoints/types.pyline 686 class ApiHdfsReplicationArguments

Hello Cloudera Community, We have a use case where we cannot afford the restart of catalogd, as the reload of metadata takes many hours and queries are impacted during this long reload. Is it possible to have multiple catalogd instances with an active / passive setup? In this case the active catalogd would act as the main catalogd and in case of failure, one of the passive catalogds would take the lead becoming active. This would eliminate the huge metadata reload time in case of failure in the catalogd node.  It would be really interesting to have the feedback of the community about such setup, and see if someone has already tested it. Thanks a lot for your replies 

Cluster manager looks fineUbuntu 20.04, bare metal host. No SElinux, no firewallThe agent log saysBEGIN sudo dpkg -l openjdk8 | grep -E '^ii[[:space:]]*openjdk8[[:space:]]*'dpkg-query: no packages found matching openjdk8END (1)BEGIN sudo apt-cache show openjdk8E: No packages found___________If you skip JDK installation in the wizard, it fails withBEGIN sudo dpkg -l cloudera-manager-agent | grep -E '^ii[[:space:]]*cloudera-manager-agent[[:space:]]*'dpkg-query: no packages found matching cloudera-manager-agent Any clues?

Hi,I need to call a soap service that is secured with WS-Security username token authentication and not basic auth. I also need to rely on the Parameter Context value for the id and password. I am using replace text to crate the soap request with the http header. When I try to reference the password it says I cannot due to the fact its sensitive and Sensitive Parameters may only be referenced by Sensitive Properties. I understand basic auth and WS-Security are different types of security. I do not see how to build/populate the security header using my value from the parameter context. Any help would be appreciated. Thank you!

Hi,  I need to install an OS package (yum install ...) -- in order to install packages needed for a (CML) R session. How can I do it on CML ? apparently I cannot do it from the session terminal (on /home/cdsw)  thanks

Hello all,I would like to know what would be the best approach, to change a schema of flowfile. I have got the following scenario : I ingest a csv file, and I would like then to rename column name inferred from the header and put them in a parquet file. Additionally, I want to cast some columns (String) to Integer or other types.I think it may be possible to do this job using the convertRecord component, or may be with the QueryRecord component but I am not sure if it is the best approach.Do you have any idea? If you have got few example it would be nice too.Thanks

Hello Team,   We are experiencing an issue with tables for Hive on HBase where we get the following issue, Note: The pure Hive tables and query on HBase are working fine.       Failed after attempts=11, exceptions:2022-02-28T12:12:54.719Z, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60115: Call to rs.host.500/rs.host.ip.500:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=5,methodName=Scan], waitTime=60002, rpcTimeout=59991 row '' on table 'a-hbase-table' at region=a-hbase-table,,1598840250675.94c031e70c63dbb0f4726251987eb4ec., hostname=rs.host.500,16020,1645349772604, seqNum=550418org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=11, exceptions:2022-02-28T12:12:54.719Z, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60115: Call to rs.host.500/rs.host.ip.500:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=5,methodN

Hi Team, We are planning for the log4j vulnerability in Ambari 2.7.5.0 ofHDP 3.1.5.0. Since we are in EOF. Kindly suggest then patch details 

Hi Team, I have configured my CDP and Keycloak. First I took my Keycloak XML file from "SAML 2.0 Identity Provider Metadata" and with this I have created one Identity provider. And I got one JSON from cloudera community and with this JSON created one client. Json file pasted atlast. With the client I created I pasted my CDP Identity provider ID in "Assertion Consumer Service POST Binding URL" in Keycloak client. After doing these steps with the provided "Target IDP initiated SSO URL" I tried to login all I get is "404 Misconfigured account. IdentityProviderConnector for ID '{39XXX08}' not found" . But if I copy my cloudera Manager URL and paste that in incognito this will redirect me to my keycloak server with this error "We are sorry... Invalid Request". JSON FILE:{"clientId": "cdp-client","surrogateAuthRequired": false,"enabled": true,"clientAuthenticatorType": "client-secret","redirectUris": ["https://consoleauth.altus.cloudera.com/saml"],"webOrigins": ["https://conso

Hadoop components are using default timestamp as UTC. Is there any way to change the default time zone from UTC.Now only seeing option to do the shifting using  

INSERT into `edl_dev_sathyam_raw_mpi_fnxnegotiat`.`negotiation` VALUES (1,'sat', 0)when running above query in impala it was executing successfully but not executing in hivethouing error like below could guide on this how to fix """Permission denied: user=hive, access=WRITE, inode="/user/sathyam.kunda/dev/data/raw/multiplan/mpi_fnxnegotiator.db/negotiationteam":impala:edlpocdev:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:553)"""

I am Getting the following warning on my cluster:This role's Heap Dump Directory is on a filesystem with less than 10.0 GiB of its space free. /tmp the following is df -h command on the partitionFilesystem Size Used Avail Use% Mounted on/dev/mapper/rhel-root 40G 32G 5.8G 85% / how to resolve this issue as i cant find any log or temp files on this filesystem.also there most of it is being consumed by this folder21G /var/lib/solr-infra/ranger_audits_shard1_replica_n1/data Any suggestions.   

Hi,When I setup a MiNiFi flow with a RPG I can set the various batch settings for the input ports (count, duration & size) and see them in the exported xml:<inputPorts><batchSettings><count>5</count><duration>15s</duration><size>1 MB</size></batchSettings> However, when I use the toolkit to convert it to yml these properties no longer appear.Are these settings supported on MiNiFi and, if not, is there any plan to support them? Thanks for your help. 

I want to transform this JSON:{  "_id" : "6218e53465793fa20ea11524",  "patientorderitems" : [ {    "poi_id" : "6218e53465793fa20ea1152a",    "patientorderlogs" : [ {      "pol_id" : "6218e53465793fa20ea1152e",      "useruid" : "61ee4995f16eebb6b7e1c644",      "modifiedat" : "2022-02-25T17:18:28Z"    } ]  }, {    "poi_id" : "6218e53465793fa20ea11525",    "patientorderlogs" : [ {      "pol_id" : "6218e53465793fa20ea11529",      "useruid" : "61ee4995f16eebb6b7e1c644",      "modifiedat" : "2022-02-25T17:18:28Z"    } ]  } ]}To this JSON: {{ "_id" : "6218e53465793fa20ea11524", "poi_id" : "6218e53465793fa20ea1152a", "pol_id" : "6218e53465793fa20ea1152e", "useruid" : "61ee4995f16eebb6b7e1c644", "modifiedat" : "2022-02-25T17:18:28Z"},{ "poi_id" : "6218e53465793fa20ea11524", "poi_id" : "62

Hello,         We are trying to migrate the data from Cloudera solr 4.10.3  to plain apache Solr cloud.         We didn't find any proper document to migrate seamlessly.         Anyone can provide clear documentation on the migration or suggest us the process to follow for the migration.  Thanks & Regards,J. Ganesh Kumar. 

I have a nested json with few first level keys. I want to compare two keys before and after in that json to see if the states are matching or not. If the states match then route to update api else insert api.What I have tried so far : getfile-->evaluate json path(to get the two keys before and after)-->route on attributesNow my problem is that route on attributes always evaluates to unmatched even if the values in eval json path for before and after are the same.Expression : Eval json path 1 : a_before = $.['a']['before'][*]['ip'] Eval json path 2 : a_after = $.['a']['after'][*]['ip']Above results in the same ip - ["1.0.0.0"]RouteOnAttribute : ${a_after:equals(${a_before})}This always results in an unmatched even if the records match.Any help will be appreciated.

Hi,how can I create, as Account Administrator, a CDP user using the default cloudera-sso IdP? I've already set an external IdP with SAML, but I would set a new local account and use it to sign-in with Cloudera SSO.Thank you, best regardsAndrea 

Hello all,I am running into an issue with authentication when trying to connect my Flink (version 1.14.3) server with NiFi. NiFi 1.15.3 is running in secure mode in cluster with 3 nodes. When my Flink flow try to connect to NiFi, it throws: Flink Log: org.apache.flink.util.SerializedThrowable: Tried all cluster URLs but none of those was accessible. Last Exception was org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 401:Unauthorized with explanation: null at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:372) at org.apache.nifi.remote.client.SiteInfoProvider.refreshRemoteInfo(SiteInfoProvider.java:69) at org.apache.nifi.remote.client.SiteInfoProvider.getPortIdentifier(SiteInfoProvider.java:220) at org.apache.nifi.remote.client.SiteInfoProvider.getOutputPortIdentifier(SiteInfoProvider.java:204) at org.apache.nifi.remote.client.socket.SocketClient.getPortIdentifier(SocketClient.java:79) at

I'm using self signed certificates for mTLS communication between minifi agents and a stand-alone EFM server. Do I need to put the public keys of each into the others' truststores or is it ok to simply have the root/intermediate CA that was used to sign the public certs in the truststores? So far I haven't been able to establish communication over mTLS for either case, but would like to know so that I can go on and plan on developing infrastructure to automate importing into the EFM truststore all the minifi agent certificates if that is the case. I'm using Cloudera EFM 1.3.1 and Apache minif 1.3.0 Thanks,minh