Find answers, ask questions, and share your expertise
Recently active
Hi, I am very frequently getting the following exception whenever I call the URL seen in the message. 2022-07-06 15:24:56,138 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=33351c02-15ee-1ccc-6c00-dda0e16110ca] Routing to Failure due to exception: javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates): javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates)javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:396) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:337) at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:209) at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226) at okhttp3.internal.connection.ExchangeFinder.findHealth
after completion of schematool successfullyWhen I tried to run or open hive I got this errorHive Session ID = 2398ab4d-e3ad-4750-b299-0543ea26332eException in thread "main" java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:413)at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:389)at org.apache.hadoop.hive.cli.CliSessionState.<init>(CliSessionState.java:60)at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)at java.base/java.lang.reflect.Method.invoke(Method.java:577)at org.apache.hadoop.u
Hi Team , Could you please help me in working with XML tag's in Nifi. I've a scenario where i need to look for the parent xml tag value in entire document, the corresponding child xml tag value should match. if it is not matching we should copy the parent xml tag value to the corresponding specified child tag.here the parent tag is <OriginalQTY name=" 5.000"> and the child tag should match is <OriginalORDEREDPURCHASEQUANTITY> So every time <OriginalORDEREDPURCHASEQUANTITY> should match <OriginalQTY> value in name. <Document><PURCHPURCHASEORDERHEADERV2ENTITY><VENDORACCOUNT>XXXXX</VENDORACCOUNT><CURRENCYCODE>ZAR</CURRENCYCODE><OriginalQTY name=" 1.000"><PURCHPURCHASEORDERLINEV2ENTITY><PURCHASEORDERNUMBER>PO-XXXXXXX</PURCHASEORDERNUMBER><LINENUMBER>0000000100</LINENUMBER><PURCHASEPRICE&g
Hi Everyone, i am trying to create Hive table with data from mongodb. I have installed mongodb in cloudera quickstart. loaded json data. sample: { "_id" : ObjectId("59a47286cfa9a3a73e51e749"), "theaterId" : 1034, "location" : { "address" : { "street1" : "3404 W 13th St", "city" : "Grand Island", "state" : "NE", "zipcode" : "68803" }, "geo" : { "type" : "Point", "coordinates" : [ -98.382164, 40.933624 ] } }} i have create Hive table as below CREATE EXTERNAL TABLE IF NOT EXISTS THEATER (id STRING,theaterId INT,loc STRUCT<address:STRUCT<street1:STRING,city:STRING,city:STRING,city:STRING,street2:STRING>, geo:STRUCT<type:STRING,coordinates:ARRAY<DOUBLE>> >)STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'WITH SERDEPROPERTIES ('mongo.columns.mapping'='{"id":"_id","theaterId":"theaterId","loc":"location"}')TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/movie_flix.theater'
Hi,how can I alter the table using NIFI Processor where source dbname : org, tableName:status, destination dbName:report and tableName: orgStatus.I am using captureChangeMySql Processor to read the binary-bin log file.Below json i am getting in updateAtribute processor:{"type":"ddl","timestamp":1658078637000,"binlog_filename":"mysql-bin.000030","binlog_position":2727,"database":"nrpsubscriptiondb2306","table_name":null,"table_id":null,"query":"ALTER TABLE `nrpuserorgdb2306`.`status` \nADD COLUMN `is_deleted` TINYINT NULL AFTER `code`"}Please find my processor screenshot for this flow.
It seems that when we use DbTxnManager in CDH6.X, if SQL query does not complete successfully or is terminated abruptly (ex. ctrl+c), then the locks it implicitly acquired will not be automatically released, and other new SQL queries which references the same table/partition will be blocked trying to acquire necessary locks, and hence can't be executed successfully. we can't manually clear the locks by issuing unlock statements under DbTxnManager,because it will pop up error: Current transaction manager does not support explicit lock requests.Transaction manager:org.apache.hadoop.hive.ql.lockmgr.DbTxnManager; it seems currently the only way to clear above obsolete locks is to log into the metastore db and use sql statements like below to delete records from table hive_locks: select hl_lock_ext_id from HIVE_LOCKS where HL_TABLE=’prcs_task’;delete from hive_locks where hl_lock_ext_id = 125542; I am wondering, if this is the correct way to ma
HI,I want NIFI to be run with the minimum resource needed. my concern is whether I can run NIFI with JRE or JDK is also required. will I face any issues in future while scaling up? I am using NIFI 1.15.3.
Hello,I know that there are processors such as CosumIMAP and ConsumPOP3 which can receive email by NiFi. These processors need to provide password.Is there another Processor which utilise Office 365 Oath2 authentification?Considering these two links which show using IMAP will be expired .... https://docs.microsoft.com/en-us/lifecycle/announcements/exchange-online-basic-auth-deprecatedhttps://www.bleepingcomputer.com/news/microsoft/microsoft-warns-exchange-online-basic-auth-will-be-disabled/ Thanks for your supports.
Hi All, Getting error in executing below script thru Execute Script processor.Requirement : upper flow will copy files to some folder.. trying to read files from folder and post them to http processor. For each file i am trying to create a new flowfile and move the same to HTTP processor. But below code failing. Can you please suggest what i am missing.--------------------------------import java.nio.charset.StandardCharsetsdef flowFile = session.get()if(!flowFile) returndef flowFiles = [] as List<FlowFile>def inputStream = session.read(flowFile)def list = []def dir = new File("/root/file_data/poc/output/")dir.eachFileRecurse (FileType.FILES) { file -> list << file}list.each{ i -> def newFlowFile = session.create(flowFile) newFlowFile = session.write(newFlowFile, { outputStream -> outputStream.write( i.getBytes(StandardCharsets.UT
Hi,I want to modify the ddl query table name. I am using the replaceText processor to modify the flowfile query table name. Currently I am able to achieve it because I have one source DB and only one table name I have changed in destination DB.Like sourceDBName: nrpsubscriotion and tableName: status -> destinationDBName: nrpresport and tableName: subscription_status. Now I have two source DB(1. nrpsubscriotion, 2. nrpuserorg) and both have same tableName: status. Now Source DB 2 , tableName in destination DB nrpreport has tableName: user_org_status. I have stuck here , I am not getting how can define a condition to replce a tableName in DDL based on schema name and ddl query able name: It should map like below: 1. SourceDBName: nrpsubscriotion and tableName: status -> destinationDBName: nrpresport and tableName: subscription_status.Flow File input: {"type" : "ddl","timestamp" : 1658314685000,"binlog_gtidset" : "e75d07af-eb37-11ec-9d
Hi,I want to manipulate below string in updateAttribute processor. I tried replace function etc but it is not working for me. Can anyone suggest to modify below string to change DBName and TableName as per expected output.query = ${query:replace('nrpuserorgdb2306.status', 'nrpreportdb.user_org_status')} query : "ALTER TABLE `nrpuserorgdb2306`.`status` \nADD COLUMN `is_deleted` TINYINT NULL AFTER `code`"Expected output:query : ALTER TABLE `nrpreportdb`.`user_org_status` ADD COLUMN `is_test` TINYINT NULL AFTER `code`
Hi everyone, I'm trying to add Hive service in cluster but an error occurred.This is the log file:2022-07-19 12:42:25,037 main ERROR Log4j2 ConfigurationScheduler attempted to increment scheduled items after startStarting metastore schema initialization to 3.1.3000.7.1.7.1000-141Initialization script hive-schema-3.1.3000.hive.sql2022-07-19 12:42:25,310 main ERROR Log4j2 ConfigurationScheduler attempted to increment scheduled items after start How can i proceed to solve the problem? Thank you
I have configured the NiFi flow file where I am reading mysql binary-log file. for me insert, update and delete flow is working fine but when I alter a table then I am getting error.sourceDBName: nrpsubscriptiondb2306 and tableName: product_types destinationDBName: nrpreportdb and tableName: product_types Alter table nifi error:Record had no (or null) value for Field Containing SQL: query, FlowFile StandardFlowFileRecord[uuid=8ca029c9-70aa-4e53-aec3-d37c15c8cd07,claim=StandardContentClaim [resourceClaim=StandardResourceClaim
Hi everyone, I would like to fetch all the processor groups in the nifi canvas including sub-processor groups usinf nifi-api. Could you please help me on this
Hi, I'm trying to make a kafka consumer working, but I am having this issue about SSL Handshake failed. Any ideas ? 2022-07-18 14:00:45,216 INFO [NiFi Web Server-203] o.a.n.c.s.StandardProcessScheduler Starting ConsumeKafkaRecord_2_6[id=f5ee162d-1006-1181-c1d1-1d8a7293ffb7]2022-07-18 14:00:45,217 INFO [NiFi Web Server-203] o.a.n.controller.StandardProcessorNode Starting ConsumeKafkaRecord_2_6[id=f5ee162d-1006-1181-c1d1-1d8a7293ffb7]2022-07-18 14:00:45,217 INFO [Timer-Driven Process Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ConsumeKafkaRecord_2_6[id=f5ee162d-1006-1181-c1d1-1d8a7293ffb7] to run with 1 threads2022-07-18 14:00:45,219 INFO [Timer-Driven Process Thread-8] o.a.k.clients.consumer.ConsumerConfig ConsumerConfig values:allow.auto.create.topics = trueauto.commit.interval.ms = 5000auto.offset.reset = latestbootstrap.servers = [bootstrap-url:9092]check.crcs = trueclient.dns.lookup = use_all_dns_ipsclient.id = consumer-integration.cubo-transactions-consumer-
Hi,I am trying to integrate Nifi with Pulsar. But unfortunately i cannot find any of Pulsar nar Processors around. I am running Nifi 1.16.3Thanks for your help.
Hello everyone you want to install the trial version of CDP private cloud on CentOS 7 machines on an external volume. Can I proceed using this guide?https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/installation/topics/cdpdc-trial-installation.html Or should I follow this?https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/installation/topics/cdpdc-manually-install-cloudera-software-packages.htmlThank you
Hi folks,My Datalake backup of CDP public always fails with this reason: Solr: Could not find any valid local directory for s3ablock-0001-. Anyone knows how to fix?
Hi everyone.I am getting stale alerts every hr, if the last point of contact between Data nodes and Namenode is more than 30 s we get these alerts.I am not able to find the root cause of this slowness, I have 32 cores system, but when this alert is generated in htop hdfs usage is more but not all cores are 100% utilized.DataNode Health SummaryDataNode Health: [Live=5, Stale=1, Dead=0]Please suggest changes required to resolve this.
anyone know why I have thousands and thousands of:2022-07-01 16:28:59,992 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,017 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,039 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,061 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,085 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,108 INFO org.ap
I have a flow in which I make a call using InvokeHTTP. The http call returns a JSON response.This JSON response looks like this:{ "token_type": "Bearer", "expires_in": 3599, "ext_expires_in": 3599, "access_token": "eyJ0eXAiOiJKV1QiLCJub25jZSI6InAzVWN1NmRBcTNPRmVGbDM5cTVoZE9EaUpUSFRHT2hVVWdYTUNLMEsyVjQiLCJhbGciOiJSUzI1NiIsIng1dCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSIsImtpZCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLm1pY3Jvc29mdC5jb20iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9iZWJhN2VlNi05ZWRhLTQzZTEtOTY5OS0yMDIxYzVmOTI5NDEvIiwiaWF0IjoxNjU3NTUzNzI3LCJuYmYiOjE2NTc1NTM3MjcsImV4cCI6MTY1NzU1NzYyNywiYWlvIjoiRTJaZ1lMalI4cjV6ZWRSMHRxUkErWFdIV2QrZkJnQT0iLCJhcHBfZGlzcGxheW5hbWUiOiJhcHAtZm9yLW5pZmkiLCJhcHBpZCI6ImRlMjg0MjNmLTgxYmMtNDc5Ni1hYTU3LTIyYTM5NmJkNDliYSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2JlYmE3ZWU2LTllZGEtNDNlMS05Njk5LTIwMjFjNWY5Mjk0MS8iLCJpZHR5cCI6ImFwcCIsIm9pZCI6IjAwN2EwNjkyLTg5ZjYtNDlmYi04ZDFhLTQwZGE3YjcwZjgwNSIsInJoIjoiMC5BVGdB
I have a huge dataset and I want to split it into partitioned table, but I'm wondering, the second column is date in int and I wonder is it possible to split table based on this, because I don't want to create three more columns with year, month and day.
Hello, I have successfully migrated the h2 database of registry to Postgres sql but it did not migrate the users.xml and authorizations.xml to postgres. Could you please help in pointing me in right direction on what setting needs to be changed to get migrate the users and authorizations data aswell to Postgres SQL.@MattWho ThanksMeeran
How to write a dataframe (pandas,spark) with python from CML (Workbench or NoteBook) to a Cloudera Datahubcluster? Is it possible without pyspark? Thanks!
Hello all,I do not know how to configure multi users on 3 nodes Nifi Cluster:Currently, i use the following to generate client & server certificates using TLS toolkit:./tls-toolkit.sh standalone -B password -C 'CN=nifiadmin, OU=NIFI' -n 'node1,node2,node3' --nifiDnPrefix 'CN=' --nifiDnSuffix ', OU=NIFI' -o /shared_folder/vm_shared/certs_100722/ -K password -P password -S password I want to configure 3 users for this 3 nodes cluster.Your help would be greatly appreciated.Thank you.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.