Find answers, ask questions, and share your expertise
Recently active
Hi all, I want to access Nifi REST API from a different client machine which is in a different domain. The REST API is up and running but gives me a CORS error when I try accessing the API from a client in a different domain. Please see below -Access to XMLHttpRequest at '...' from origin 'null' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. I understand that I need to change Jetty's web.xml to allow cross origin requests. I can't seem to find web.xml in my nifi install. This is a nifi docker container.Any help I can get in this is much appreciated. Please also let me know if there's a better solution than changing web.xml. I tried CORS browser plugins but haven't been able to get them to work. Thanks in advance.
Which versions and distributions of Hadoop are compatible with Spark 3.3?
Hi team I am new to CDP , I have a problem with the ldap changes in the cloudera manager service.It requests a restart of the service after applying the changes , this restart is applied but the changes are not applied.Can you help me , to identify the problem .
On tableau dev server, jobs are failing with below error, whereas its getting successful from tableau desktop. we use MIT kerberos authentication to connect from Tableau to Cloudera. Same config works in our tableau prod machineError:This job failed on Aug 12, 2022, 8:45 AM after running for 0.0 min because of: java.lang.RuntimeException: [Cloudera][ThriftExtension] (9) Error occurred while authenticating via SASL. Error details: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty) Integrated authentication failed
I'm connecting to Hive using Cloudera's ODBC driver version 2.6.9 and I'm getting a weird behavior that ruins the user experience. Any help to solve or debug this problem would be greatly appreciated. I'm using DBeaver to connect to it. All my databases just display a few of it's tables. Here is an example. When I open DBeaver, The dabase just shows the table `localizacao`: This database has 3 tables. If I run the command show tables from raw_aneel, DBeaver interface will display: The error message in english would be: org.jkiss.dbeaver.model.exec.DBCException: SQL Error [S1090]: [Microsoft][ODBC Driver Manager] Invalid string or buffer length Here is the complete stack trace displayed in the Error Log. The error message above is in Brazilian Portuguese: org.jkiss.dbeaver.model.exec.DBCException: SQL Error [S1090]: [Microsoft][ODBC Driver Manager] Comprimento inválido de cadeia de caracteres ou de bufferat org.jkiss.dbeaver.m
Hi All,I want to put the error message or want to attach the log file from local location to the nifi flow. My flow is as below:Getfile -> EvaluateJSON -> ExecuteStreamCommand->RouteAttribute-> PutEmail. Please help me for this. Thank you in advance!!
Hi Team,we have done the patching on the development it has failed document is attached and the UAC team suggested to remove symlink /usr/bin/python, on rhel8 it should not be set. can you please suggest what we can do.Below are the errors in logfile :-fatal: [clrv0000239897.ic.ing.net]: FAILED! => {"changed": false, "cmd": "dnf install -y python2-dnf", "msg": "Could not import the dnf python module using /usr/bin/python (2.7.18 (default, Feb 10 2022, 14:26:12) [GCC 8.5.0 20210514 (Red Hat 8.5.0-10)]). Please install python2-dnf package or ensure you have specified the correct ansible_python_interpreter.", "rc": 1, "results": [], "stderr": "Error: Unable to find a match: python2-dnf\\n", "stderr_lines": ["Error: Unable to find a match: python2-dnf"], "stdout": "Last metadata expiration check: 0:00:26 ago on Tue 02 Aug 2022 07:05:15 PM CEST.\\nNo match for argument: python2-dnf\\n", "stdout_lines": ["Last metadata expiration check: 0:00:26 ago on Tue 02 Aug 2022 07:05:15 PM CEST.", "
Hello,I'm new to nifi so my problem may be dumb. Sorry if it seems really basic.I'm trying to use nifi to :1/ Read a JSON response from a REST GET endpoint.2/ Parse the JSON and split some JSON array into JSON objects3/ insert objects in SQL.I'm stuck at Step 2 because my SplitJSON doesnt return anything. No error as well.Here is a sample of my JSON : { "context":"q%3D%2523all%26q.restriction%3Dclass%253Acr_hashtag%26of%3Djson", "nhits":1000000, "nmatches":1000000, "start":0, "ellql":"#query{nbdocs=26966678, text_relevance.expr=\"@term.score*@proximity+@b\", noproximity, term.score=NO_RANKING}(#and(#true() #category(categories,\"Top/datamodelclass/cr_hashtag\") ))", "executor":null, "estimated":false, "autocorrected":false, "spellCheckSuggestions":{ "origStr":null, "suggestions":[ ] }, "hits":[ { "did":1, "url":"ID=10055&", "buildGroup":"bg0", "source":null, "slice":3, "score":0,
Hi, I wanted to install Apache Nifi 1.16.3 or 1.17.0 on my Ubuntu system. However, I am not able to find the required tar files from the Apache Downloads page. Can someone please drop links where I can see the nifi-1.17.0-bin.tar.gz/nifi-1.16.3-bin.tar.gz to install in ubuntu 18.04?
Hello Experts, We are using HDP 3.1.5 and since short time we have one specific table with several partitions created in it, where the compaction for one of these partitions fails repeatly. I'm not able to find the exact error why it is failing. from yarn I see the following error in the application log:org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to Task failed task_1657267798389_0085_m_000000Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 and in Metastore Log I only find this:ERROR [thread-71]: compactor.Worker (Worker.java:run(198)) - Caught exception while trying to compactid:697908,dbname:standard,tableName:<table-name>,partName:<partitionname>,state:^@,type:MAJOR,properties:null,runAs:null,tooManyAborts:false,highestWriteId:0. Marking failed to avoid repeated failures, java.io.IOException: Major compactor job failed for thread-71-compactor-<dbname>.<tablename>.<partit
Hello, I would like to create Hive tables using Zeppeling and I found the the below document that use HWC:https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/integrating-hive-and-bi/topics/hive-hwc-reading.htmlI'm not sure what means "livy configuration file"? Is "%livy2" a new Interpreter? May I create a new Interpreter with only indicated configurations? Thanks!
Hello, please kindly help. i am trying to convert a datetime(attribute) from one format to the other but it keeps giving me empty value on the attribute:SCENERIO:Flowfile is json----->extract date from Json flowfile using Evaluatejsonpath---->format date using UpdateattributeActual Date: "2022-08-01 00:00:00.0"Desired date: "01/08/2022 00:00:00.000000"I have tried to use this on the updateattribute, but it is not working:${DATE_TIME:toNumber():toDate('yyyy-MM-dd hh:mm:ss.S'):format('dd/MM/yyyy HH:mm:ss.SSSSSS')} Thank you.
Is there any way to update the flow description in the NiFi Registry after the initial creation of the flow that does not require restarting version control? I checked the REST API endpoints as well and didn't see anything. I would like to be able to clean up descriptions (or even just correct a typo) without going through a cumbersome export versions, import flow, import versions, delete and re-import PGs process. Thanks.
Hello Lovely community, I am new to NiFi, I am able to connect to mysql dummy database from Nifi and everything works fine.But I want to understand how can we connect to databases that use SSH tunneling..Because I have not seen any properties related to SSH in the database processors.My processors being used is "capturechangeMysql" This source DB server is secured and I need SSH tunnel to make the connection.I have 100 such sources. Should I add my nifi server ssh keys in all the source servers ? and even then where can i give the SSH tunnel information? and the private key..Because while using any SQL client we can use the tunnel along with main DB username and password as shown in attached screen, but for Nifi I am not able to see a way..Please help me with your suggestions.. /\SSH tunnel
good morning cloudera community, we are using apache ambari in version 2.6.2.2 and in this version we are using HDP 2.6.5, with that, we need to configure the parameter below in HDFS, but unfortunately we are not finding this option in the HDFS configuration, so we need to know how configure this parameter in HDFS? parameter: dfs.client.block.write.replace-datanode-on-failure.policy=ALWAYS PS: would it be in the "custom hdfs-site" option by clicking on "add property" and adding the parameter in the "properties" box?
Hi Team, I am new to Apache Nifi, can you please help me to implement CDC for Postgresql one table to log table in Nifi. I am using Nifi - 1.16.x version.
if I have 5 zookeeper nodes and from that 5 zookeeper nodes one 1 zookeeper nodes goes down which was leader node then how zookeeper will elect the new leader from the remaining 4 zookeeper node?
Hi, I am new to nifi and stuck with two issues for which i need help:- The task is to migrate a set of tables from source to target database. The data will have to be filtered as well based on a field before migrating. The pipeline has to migrate historical data as well as perform the incremental fetch. 1. For incremental fetch, I used the GenerateTableFetch processor and set the max-value column to a date field in the table. I also used the partitioning feature to get the data in chunks by setting the partitioning size and column. It all works well. But considering it is a stateful processor, i have received feedback to go stateless as we can loose the state in case of node crash. How can i achieve the incremental fetch and partitioning feature in a stateless manner? 2. The set of tables to be migrated have a parent child relationship. The incremental fetch is required for all the tables. However child tables don't have any such column that can be used
Hi Team, We have CDH - 5.16.2 in our environment, recently renewed Cloudera -license.But still since today we are not able to create a case on support portal.Please find the attached snapshot.
Hello All, I am using the mysql CDC processor in Nifi, to get incremental changes into DWH, in this processor fo the username field, which permissions should be granted to the user so that it can access cdc files ?should be the one with highest privileges/ root user ?because it is not mentioned anywhere in the help section (https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-cdc-mysql-nar/1.5.0/org.apach... ) And when i provide a user credentials who has GRANT ALL PRIVILEGES ON *.* then it is able to access CDC bin logsBut getting permission denied when trying to access with a user who has only read permissions on *.* Since the processor has nothing to do with write, i thought only read permissions were enough, but looks like its not the case.. Because I need to create new users in the source to access these CDC files.. and I dont want to give them Grant ALL, as it seems riskier.. Please help with your suggestions and th
Hi, We have got this issue post java upgradation. Kindly let us know whether this error occurred because of java upgrade or else please suggest a solution. spark-shellSetting default log level to "WARN".To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).22/08/03 15:34:49 ERROR SparkContext: Error initializing SparkContext.org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:89)at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:63)at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:164)at org.apache.spark.SparkContext.<init>(SparkContext.scala:500)at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2498)at org.apache.spark.sql.SparkSession$
当前Hive版本:3.1.3000.7.1.7.0-551 创建hive外部表: CREATE TABLE IF NOT EXISTS `alter_test` (a string,b string) STORED AS ORC TBLPROPERTIES("transactional"="false");转hive内部表:ALTER TABLE alter_test SET TBLPROPERTIES('external'='false');重命名:ALTER TABLE alter_test RENAME TO alter_test_bak;注:external.table.purge 这个配置是一直为true的此时,表alter_test_bak的location目录名称依然是alter_test。同样的操作步骤,我在Hive版本 3.1.3000.7.1.1.0-565是可以同步更改location目录名的,请问这是怎么回事呢
I have 3 nodes NiFi and 1 node Registry instances running. Registry - SSL enabled and AD-integrated NIfi - SSL enabled and AD-integrated how can I integrate NiFI with Registry, so I can start using Registry. @MattWho please advice for any recommendations
Hi,I am using mirrormaker 2 to copy data of one topic from source cluster to destination cluster. data is getting copied but I see 3 hours of lag. and I am not able to find the exact cause of it (whether the delay is at consumer side or producer ) as it is using kafka connect framework.Below are the config detailstest topic having 2 partition and replication factor as 3number of messages coming into the source topic is 8K per second.
Hi ,I have two kafka cluster where I need to copy only the topic configuration from one cluster to other without copying any data. How can we do that. I know using replicator/mirrormaker it can be done but I am not interested in data here.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.