Solved

Bonjour, j'ai installé NIFI en suivant la documentation, j'ai bien evidement vérifié que j'avais le bon java, mais au moment d'accéder au portail localhost, je n'y arrive pas, ca me met ça: Bien evidemment je l'ai la,cé avant, voici ce que dit la cmd ou je le lance, aussi je ne trouve pas mes identiifiants comme convenu dans le fichier logs, quelqu'un pourrait m'aider svp 

Hello, I am following the Apache NiFi Walkthroughs, specifically the NiFi Standalone Using External CA.  I am trying to install a 5 node cluster.  The process I am following is:I created, using TinyCert the following:1.  A signed NiFi server certifcate for each NiFi server in my cluster i.e., nifi1, nifi2, nifi3, nifi4, and nifi5.2.  The matching private keys in PEM format.3.  A signed client certificate that I created on nifi1 (which I then planned to reuse for the other nodes i.e., nifi2, nifi3, nifi4, and nifi5).4.  The matching private key in PEM format.5.  The CA certificate in PEM format. Then, on each server i.e., nifi1, nifi2, nifi3, nifi4, and nifi5, i followed the steps indicated in the walkthrough i.e., 1.  concatenate the server certificate and CA certificate to form the certificate chain.2.  Form the PKCS12 keystore from the certificate chain and private key.3.  Convert the PKCS12 keystore for the NiFi instance in

I have a feed coming in with time spent value set in seconds. Example 900 or 3600Is there way in nifi to convert attribute to 00:15 and 01:00 (HH:mm format)?

There is an error during cdsw model deployment where the package of code is not imported. - CDSW version1.9.2 - Folder Configuration/home/cdswl- ex_package_onel    l- ex_package_twol    l-ex_pkg_code.pyll- predictl    l- predict.py predict.py is the code used in the model deployment.- Package import code within predict.py....from ex_package_one.ex_package_two import ex_pkg_code...- Error logModuleNotFoundError: No module named 'ex_package_one'- Attempt to resolve the errorI printed it out as os.getcwd() from predict.py and it came out as /cdsw/home.sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname("__file__"))))The path was added using the above code, but the same error occurred.No package errors will occur if the code is transferred to /home/cdsw/predict.py.Is there any other way to solve this problem?Thank you for any help.

Hello , can anyone help with a spec to transform this JSON input to the desired output . JSON INPUT=========={"saleStrategy": {"strategy": "strategy1","code": "code1"},"saleStrategyCollection": [{"id": "2","saleStrategy": {"strategy": "strategy2","code": "code2"}},{"id": "3","saleStrategy": {"strategy": "strategy3","code": "code3"}}]}== desired output json =={"saleStrategies" : [{"strategy" : "strategy1","code" : "code1"},{"strategy" : "strategy2","code" : "code2"},{"strategy" : "strategy3","code" : "code3"}]}== current spec === // this currently only transforms the initial single saleStrategy object to an array called salesStrategies. Desired output above shows that If there is a saleStrategy array also present in the input JSON , then the spec needs to add the saleStrategy objects within that array to the salesStrategies output array as well . NOTE the transform must not include "id" in the output.[{"operation": "shift","spec": {"saleStrategy": {"strategy": "saleStrategies[0].st

Hi Community, I'd like to know if it's possible to display a table or column description from Power BI that is connected to Hive using ODBC driver? Issue is following: I set a comment in a Hive table and on its columns and I would expect that these are loaded and visible as "Description" fields from external tools (like Power BI). For example:create table mytable1(mycol1 string comment 'column description') stored as ORC TBLPROPERTIES('comment' = 'table description'); I would expect that when Power BI accesses this table, it would retrieve these comments as descriptions of the table/column, but the "Description" fields are all 'null'. Note that these table/column descriptions are visible from HUE. Thanks in advance for your anwsers.

impala执行sql语句时速度不稳定，同样类型的sql存在十几秒完成，也存在几分钟才完成，从sql执行持续时间上来看，[客户端获取等待时间]占整体时间占比很高，请问是什么因素会影响[客户端获取等待时间]   负载均衡中间件：Haproxy 负载超时时间设置：30m     查询慢的脚本 查询慢的脚本1查询慢的脚本2查询慢的脚本3  

hi  all, hbase version: cdh 6.2.0 balancer config <property><name>hbase.master.loadbalancer.class</name><value>org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer</value><final>false</final><source>hbase-default.xml</source></property> <property><name>hbase.balancer.period</name><value>300000</value><final>false</final><source>hbase-default.xml</source></property> but the balancer not scheduler  interval  5min and no error logs  and use  balancer command in shell  will return  1  hbase(main):007:0> balancertrueTook 0.0746 seconds=> 1 and there any help to fix it ?

Hi,we would like to install Cloudera CDP 7.1.7 SP1 using a Powerscale Storage with OneFS 8.2 or 9.2.Is there any limitations about the Cloudera CDP Runtime features? (Es. Kudu, HDFS ACL, etc.)I can't find any compatibility matrix for this purpose.Thanks in advance. 

Hi, I have inherited some Hadoop 3.1.1 clusters. There is a configured connection between SAP BO and Hadoop using Knox 1.0.0 as the Proxy for authentication. The problem that I have is that I don´t count with user to authenticate to Knox, and I need to edit the gateway. Is there a way to recover the admin password? Thanks for your support.

Following the Apache walkthrough titled "NiFi Cluster Using NiFi CA", i created a 3 node cluster using 3 separate aws instances.  However, when I access the GUI at https://nifi1:9337 (or nifi2, or nifi3), for nifi1 and nifi2, the GUI only reflects 2 nodes (nifi1 and nifi2 but not nifi3).  https://nifi3:9337 only reflects 1 node i.e., nifi3. I am using an external zookeeper and these are the properties I changed in nifi.properties:nifi.cluster.is.node = truenifi.cluster.load.balance.host = nifi[1,2,or 3]...depending on which host the nifi.properties file exists innifi.zookeeper.connect.string = zookeeper1 private IP address:2181, zookeeper2 private IP address:2181, etc... Additionally, per the directions, I updated the state-management.xml file and added the list of zookeeper instances in the Connect String property i.e., zoo1private IP address:2181, etc... I also updated the authorizers.xml file, by adding the following:  in userGroupProvide

I've checked the configuration file and other information, and I don't see any problem. 2023-02-08 15:49:16,113 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.RecoveredContainerLaunch: Unable to recover container container_e05_1612422638528_1073494_02_000023java.io.IOException: Problem signalling container 20735 with NULL; output: and exitCode: 24 at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.signalContainer(LinuxContainerExecutor.java:515) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.isContainerProcessAlive(LinuxContainerExecutor.java:570) at org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor.reacquireContainer(ContainerExecutor.java:179) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.reacquireContainer(LinuxContainerExecutor.java:473) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.RecoveredContainerLaunch.call(RecoveredContainerLaunch.java:83) at org.apa

Hi, Gor message out of the blue - speaking to agents. Any one know if its part of Cloudera.com or is it a scam? Thanks 

Is there a way to get or see the log message in case of validation failure of a flow-content versus the provided json schema ? 

I am setting up a Pre prod cluster from Path B. I ran the command "/usr/share/cmf/schema/scm_prepare_database.sh mysql -h mysqldatabase scm temp password" to check the database status. I am getting the error of log4j however the SCM database is configured correctly. Please find the below error and suggest any workaround for this error.[root@clouderamanage cloudera-scm-server]# /usr/share/cmf/schema/scm_prepare_database.sh mysql -h webserver scmmysqldatabase passwordJAVA_HOME=/usr/java/jdk1.7.0_67-clouderaVerifying that we can write to /etc/cloudera-scm-serverCreating SCM configuration file in /etc/cloudera-scm-serverExecuting: /usr/java/jdk1.7.0_67-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.log4j: ERROR Could not find value for key log4j.

I have created a new Nifi registry image using our own base image. I deployed this image in one of my cluster, on creating a processor group in Nifi I can do import from Nifi registry and create a new bucket with flow.But on deploying this same image in the another cluster import from Nifi registry works fine. Creation of new buckets with new flow is taking some time. In some case I have to delete the pod of Nifi registry to get this done.Both nifi and Nifi registry pods are in the same namespace . No DB and PV for Nifi registry.Does anyone here have any idea what might be causing this issue.

I want to access the reporting task id of a particular reporting task. For doing the same , I thought about fetching all reporting task id and then searching for a particular one. Is there any nifi-api that can give me all the reporting task ids or any other approach to solve this problem.

Hi All, I'm new to HBase. I have learnt it by some books and Udemy courses. Can someone help me to understand, how to automate it? For example, if one wants to create tables or wanted to perform some administrative tasks, has to create script and run it periodically. in this case, we can wrte script and schedule it in scheduling tool. Which scripting is being used here? Shell Script or HBase has internal scripting mechanism? Any books, tutorial suggestion are welcome.

We have a large json file which is more than 100GB and we want to split this json file into multiple files. We used Split Text processor to split this json file into mutliple files by specifying Line Split Count. Is there any way we can pass attribute/variable in Line Split Count and then split the records based on the attribute/variable as currently Line Split Count does not support attributes/variables.Kindly suggest if there is another approach to split these json files based on attribute/variables Sample Json File {"name": "John","lastName": "Wick","phoneNumber": "123123123"} {"name": "Paul","lastName": "Jackson","phoneNumber": "123123123"}{"name": "Paul","lastName": "Jackson","phoneNumber": "123123123"} 

On Cloudera Public Cloud the storage unit is GCS. Therefore, hive tables and any inserted data are stored on GCS rather than local disks ( like as on-prem ). But, this adds an overhead because when running a job, the node manager needs to get the data from gcs and cache it locally till the job is done. This eventually hits the performance on the env. Is there any way to move the data location from GCS to Local disks on Public Cloud clusters? As written on the docs, datahub hdfs/disks spaces are temporary places but I would take this risk in favor of performance. @steven-matison would really appreciate your help on this question. Thanks in advance.

Hello team, We are facing issue and expecting your help here regarding Problem Statement :We create groups using in Cloudera user management using APIs.We can see those groups in Cloudera user management but not  could not find them as Ranger Groups. It seems groups are not being synced in Ranger.What are the possible solutions ? Any API is exist to sync users and groups?

Hi team, I have the error Faled running performance when i run the "Inspect network Performance" all the nodes of clouster are good except one. I see this error: [24/Feb/2023 09:19:19 -0600] 1547408 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/perf/host_perf_diag.py input.json logs/result.json[24/Feb/2023 09:19:19 -0600] 1547408 MainThread redactor ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/1502-host-perf-inspector[24/Feb/2023 09:19:19 -0600] 1547408 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 1547410Traceback (most recent call last):File "/opt/cloudera/cm-agent/bin/cm", line 8, in <module>sys.exit(main())File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/cmf/main.py", line 399, in mainroot(obj=argparse.Namespace())File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/click/core.py", line 1137, in __call

Hello, I have a Nifi Flow working to ingest logs messages and send it up to ElasticSearch. Everything worked fine for one month but suddenly some issues appeared without changes about size and volumetry of incoming logs messages: - I have a lookupRecord processor to enrich data and the queue become full for it so i guess this processor is the problem but it never had this behavior before- Nifi tell me it's unable to write flowfile content to content repo due to archive file size constraint- The input processor of the flow is a listenUDP and warn me that the internal queue is full I tried to reboot everything, obsioulsy, and also to increase JVM memory settings to Xmx12g to be sure it's not a lack of memory.I saw that the archive cleanup issue could be a bug from Nifi but I'm actually on 1.17 and even tried to move to 1.20 and everything is same. Any idea of what could cause those problems ?Thanks

We have recentely upgraded our CDP environments from runtime 7.2.12 to 7.2.16 and we are getting a wierd vertex error that we are unable to find the root cause for. anyone have a suggestion where to start looking? ERROR:Vertex vertex_1677501096465_0141_1_61 [Reducer 21] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1677588363293, firstTasksToStart=[ task_1677501096465_0141_1_61_000000 ], lastTaskFinishTime=1677591606664, lastTasksToFinish=[ task_1677501096465_0141_1_61_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=2, numCompletedTasks=1, numSucceededTasks=0, numKilledTasks=1, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=TezYarn, taskSchedulerName=TezYarn, taskCommunicatorName=TezYarn, containerLauncherClassName=org.apache.tez.dag.ap

I'm using a tool in which I have to point out the master node (driver node) of the Cloudera Spark Cluster (spark :// <some-spark-master> : 7077). Also as I learned, Spark has "Master Node" (Driver Node) and "Worker Nodes".So I decided to go to the Cloudera Web Manager and checked the Configuration Tab of the Spark service, but all I found are "Gateway instance" and "History Server instance". Where are the "Driver instance" and "Worker instance"? I can't add these two instances in the "Add Role Instances" too My guess is that it's in Yarn service configuration, but I can't find anything related to "Master"/"Driver" or "Worker" either. So what is the link to "Spark Master" that ends with 7077? I can't find it anywhere in the Configuration tab