Find answers, ask questions, and share your expertise
Recently active
Hi Team, I have created an rest endpoint on NIFI as per below Screen.The purpose of this endpoint is to fetch data from https://jsonplaceholder.typicode.com/todos/2 and present as http response of my API. Below is my template code -<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template encoding-version="1.3"><description></description><groupId>be506fdd-0184-1000-83e1-c82d0bfe4cb7</groupId><name>SampleTemplatetoCallHttpsEndpoint</name><snippet><processGroups><id>78122aea-0a0d-3713-0000-000000000000</id><parentGroupId>f175ed29-51e9-3358-0000-000000000000</parentGroupId><position><x>0.0</x><y>0.0</y></position><versionedComponentId>78122aea-0a0d-3713-ba11-a970a91b757e</versionedComponentId><comments></comments><contents><connections><id>00ded3d7-2d46-349b-0000-000000000000</id><parentG
Hi , Currently i am using NIFI to consume and publish data using kafka and i have a requirement to perform retry only for topicAuthorizationException thrown by NIFI processor (publishKafka processor). Like similar to RouteonAttribute, is there any processor or any log attribute through which i can parse or read this topicAuthorizationException error , so that i can perform retry? when kafka topic does not exist, NIFI throws below exception, the same is seen in nifi-app.log, i need to know how can we read this error in NIFI org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [test_topic]
Bonjour, j'ai installé NIFI en suivant la documentation, j'ai bien evidement vérifié que j'avais le bon java, mais au moment d'accéder au portail localhost, je n'y arrive pas, ca me met ça: Bien evidemment je l'ai la,cé avant, voici ce que dit la cmd ou je le lance, aussi je ne trouve pas mes identiifiants comme convenu dans le fichier logs, quelqu'un pourrait m'aider svp
I am have the next error when try to acces resource manager from the knox web ui errror in gateway.log INFO knox.gateway (ConfigurableHADispatch.java:failoverRequest(275)) - Failing over request to a different server: https://awsnode01:7183/favicon.ico?doAs=u3212 INFO knox.gateway (ConfigurableHADispatch.java:executeRequest(162)) - Could not connect to server: https://awsnode01:7183/favicon.ico?doAs=u3212 org.eclipse.jetty.io.EofException: Closed INFO knox.gateway (ConfigurableHADispatch.java:failoverRequest(275)) - Failing over request to a different server: https://awsnode01:7183/favicon.ico?doAs=u3212INFO knox.gateway (ConfigurableHADispatch.java:executeRequest(162)) - Could not connect to server: https://awsnode01:7183/favicon.ico?doAs=u3212 org.eclipse.jetty.io.EofException: Closed INFO knox.gateway (ConfigurableHADispatch.java:failoverRequest(275)) - Failing over request to a different server: https://awsnode01:7183/favicon.ico?doAs=u3212INFO knox.gateway (C
Hello, I am following the Apache NiFi Walkthroughs, specifically the NiFi Standalone Using External CA. I am trying to install a 5 node cluster. The process I am following is:I created, using TinyCert the following:1. A signed NiFi server certifcate for each NiFi server in my cluster i.e., nifi1, nifi2, nifi3, nifi4, and nifi5.2. The matching private keys in PEM format.3. A signed client certificate that I created on nifi1 (which I then planned to reuse for the other nodes i.e., nifi2, nifi3, nifi4, and nifi5).4. The matching private key in PEM format.5. The CA certificate in PEM format. Then, on each server i.e., nifi1, nifi2, nifi3, nifi4, and nifi5, i followed the steps indicated in the walkthrough i.e., 1. concatenate the server certificate and CA certificate to form the certificate chain.2. Form the PKCS12 keystore from the certificate chain and private key.3. Convert the PKCS12 keystore for the NiFi instance in
i have NIFI mongo db connection and need to fetch only new record , i have a long column (transactiondate) (yyyy-MM-dd HH:mm:ss) and need the new transactions in last 1 sec .
I have a feed coming in with time spent value set in seconds. Example 900 or 3600Is there way in nifi to convert attribute to 00:15 and 01:00 (HH:mm format)?
There is an error during cdsw model deployment where the package of code is not imported. - CDSW version1.9.2 - Folder Configuration/home/cdswl- ex_package_onel l- ex_package_twol l-ex_pkg_code.pyll- predictl l- predict.py predict.py is the code used in the model deployment.- Package import code within predict.py....from ex_package_one.ex_package_two import ex_pkg_code...- Error logModuleNotFoundError: No module named 'ex_package_one'- Attempt to resolve the errorI printed it out as os.getcwd() from predict.py and it came out as /cdsw/home.sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname("__file__"))))The path was added using the above code, but the same error occurred.No package errors will occur if the code is transferred to /home/cdsw/predict.py.Is there any other way to solve this problem?Thank you for any help.
Hello , can anyone help with a spec to transform this JSON input to the desired output . JSON INPUT=========={"saleStrategy": {"strategy": "strategy1","code": "code1"},"saleStrategyCollection": [{"id": "2","saleStrategy": {"strategy": "strategy2","code": "code2"}},{"id": "3","saleStrategy": {"strategy": "strategy3","code": "code3"}}]}== desired output json =={"saleStrategies" : [{"strategy" : "strategy1","code" : "code1"},{"strategy" : "strategy2","code" : "code2"},{"strategy" : "strategy3","code" : "code3"}]}== current spec === // this currently only transforms the initial single saleStrategy object to an array called salesStrategies. Desired output above shows that If there is a saleStrategy array also present in the input JSON , then the spec needs to add the saleStrategy objects within that array to the salesStrategies output array as well . NOTE the transform must not include "id" in the output.[{"operation": "shift","spec": {"saleStrategy": {"strategy": "saleStrategies[0].st
Hi Community, I'd like to know if it's possible to display a table or column description from Power BI that is connected to Hive using ODBC driver? Issue is following: I set a comment in a Hive table and on its columns and I would expect that these are loaded and visible as "Description" fields from external tools (like Power BI). For example:create table mytable1(mycol1 string comment 'column description') stored as ORC TBLPROPERTIES('comment' = 'table description'); I would expect that when Power BI accesses this table, it would retrieve these comments as descriptions of the table/column, but the "Description" fields are all 'null'. Note that these table/column descriptions are visible from HUE. Thanks in advance for your anwsers.
impala执行sql语句时速度不稳定,同样类型的sql存在十几秒完成,也存在几分钟才完成,从sql执行持续时间上来看,[客户端获取等待时间]占整体时间占比很高,请问是什么因素会影响[客户端获取等待时间] 负载均衡中间件:Haproxy 负载超时时间设置:30m 查询慢的脚本 查询慢的脚本1查询慢的脚本2查询慢的脚本3
hi all, hbase version: cdh 6.2.0 balancer config <property><name>hbase.master.loadbalancer.class</name><value>org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer</value><final>false</final><source>hbase-default.xml</source></property> <property><name>hbase.balancer.period</name><value>300000</value><final>false</final><source>hbase-default.xml</source></property> but the balancer not scheduler interval 5min and no error logs and use balancer command in shell will return 1 hbase(main):007:0> balancertrueTook 0.0746 seconds=> 1 and there any help to fix it ?
Hi,we would like to install Cloudera CDP 7.1.7 SP1 using a Powerscale Storage with OneFS 8.2 or 9.2.Is there any limitations about the Cloudera CDP Runtime features? (Es. Kudu, HDFS ACL, etc.)I can't find any compatibility matrix for this purpose.Thanks in advance.
Hi, I have inherited some Hadoop 3.1.1 clusters. There is a configured connection between SAP BO and Hadoop using Knox 1.0.0 as the Proxy for authentication. The problem that I have is that I don´t count with user to authenticate to Knox, and I need to edit the gateway. Is there a way to recover the admin password? Thanks for your support.
I am trying update nifi-1.15.3 java version from 1.8 to jdk 11. While trying to start up Nifi getting below error: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'niFiWebApiConfiguration': BeanPostProcessor before instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'metaDataSourceAdvisor': Cannot resolve reference to bean 'methodSecurityMetadataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.security.config.annotation.method.configuration.GlobalMethodSecurityConfiguration': Unsatisfied dependency expressed through method 'setMethodSecurityExpressionHandler' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.
Following the Apache walkthrough titled "NiFi Cluster Using NiFi CA", i created a 3 node cluster using 3 separate aws instances. However, when I access the GUI at https://nifi1:9337 (or nifi2, or nifi3), for nifi1 and nifi2, the GUI only reflects 2 nodes (nifi1 and nifi2 but not nifi3). https://nifi3:9337 only reflects 1 node i.e., nifi3. I am using an external zookeeper and these are the properties I changed in nifi.properties:nifi.cluster.is.node = truenifi.cluster.load.balance.host = nifi[1,2,or 3]...depending on which host the nifi.properties file exists innifi.zookeeper.connect.string = zookeeper1 private IP address:2181, zookeeper2 private IP address:2181, etc... Additionally, per the directions, I updated the state-management.xml file and added the list of zookeeper instances in the Connect String property i.e., zoo1private IP address:2181, etc... I also updated the authorizers.xml file, by adding the following: in userGroupProvide
So I'm working with a batch JSON file with the following value :[ { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "ES", "resourceType": "Party" }, { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "ES", "resourceType": "Party" }, { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "FR", "resourceType": "Party" }, { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "FR", "resourceType": "Party" }]So I'm working with a batch JSON file with the following value :[ { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "ES", "resourceType": "Party" }, { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "ES", "resourceType": "Party" }, { "eventType": "UPDATE", "eventTime": "2021-12-14T12:34:56.789012Z", "country": "FR", "resourc
I've checked the configuration file and other information, and I don't see any problem. 2023-02-08 15:49:16,113 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.RecoveredContainerLaunch: Unable to recover container container_e05_1612422638528_1073494_02_000023java.io.IOException: Problem signalling container 20735 with NULL; output: and exitCode: 24 at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.signalContainer(LinuxContainerExecutor.java:515) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.isContainerProcessAlive(LinuxContainerExecutor.java:570) at org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor.reacquireContainer(ContainerExecutor.java:179) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.reacquireContainer(LinuxContainerExecutor.java:473) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.RecoveredContainerLaunch.call(RecoveredContainerLaunch.java:83) at org.apa
Hi All, We are facing an issue where no matter what we try Impala queries will randomly throw a "Failed to open HDFS file" error. This seemingly started out of nowhere and we are not sure what else to try. Below are some of the things we have tried. 1. Enforce SYNC_DDL2. We used to have 87 impala daemons (both executor and coordinator). We setup dedicated coordinators for Impala (4 coordinator + 83 executors) and load balanced with haproxy.3. Tried adding invalidate metadata, and then removing it. Below is the sequence of queries. 1. Insert Overwrite a table. (approx every 1 hour)2. Refresh 3. Compute stats.4. Select. The select never fails on the same coordinator as insert, but randomly on other coordinators. And it keeps failing until a refresh. As soon as a refresh is run on the other failing coordinator, query succeeds. This leads me to believe it is a metadata sync issue across coordinators. The problem is that multiple applications/dashboar
Hi, Gor message out of the blue - speaking to agents. Any one know if its part of Cloudera.com or is it a scam? Thanks
Is there a way to get or see the log message in case of validation failure of a flow-content versus the provided json schema ?
Please could someone help what can be done here , i'm getting attached error while try to set up the cluster
I am able to create tables and load data in iceberg manually. But how can I do that using nifi? And how can I connect to the remote server using nifi?
I am setting up a Pre prod cluster from Path B. I ran the command "/usr/share/cmf/schema/scm_prepare_database.sh mysql -h mysqldatabase scm temp password" to check the database status. I am getting the error of log4j however the SCM database is configured correctly. Please find the below error and suggest any workaround for this error.[root@clouderamanage cloudera-scm-server]# /usr/share/cmf/schema/scm_prepare_database.sh mysql -h webserver scmmysqldatabase passwordJAVA_HOME=/usr/java/jdk1.7.0_67-clouderaVerifying that we can write to /etc/cloudera-scm-serverCreating SCM configuration file in /etc/cloudera-scm-serverExecuting: /usr/java/jdk1.7.0_67-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.log4j: ERROR Could not find value for key log4j.
I have created a new Nifi registry image using our own base image. I deployed this image in one of my cluster, on creating a processor group in Nifi I can do import from Nifi registry and create a new bucket with flow.But on deploying this same image in the another cluster import from Nifi registry works fine. Creation of new buckets with new flow is taking some time. In some case I have to delete the pod of Nifi registry to get this done.Both nifi and Nifi registry pods are in the same namespace . No DB and PV for Nifi registry.Does anyone here have any idea what might be causing this issue.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.