Member since
12-03-2017
157
Posts
26
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2353 | 11-03-2023 12:17 AM | |
| 4407 | 12-12-2022 09:16 PM | |
| 1690 | 07-14-2022 03:25 AM | |
| 2541 | 07-28-2021 04:42 AM | |
| 3642 | 06-23-2020 10:08 PM |
01-31-2023
10:48 PM
Hello Experts, We have a nifi instance with 8 core machine, and recently nifi is looking quite slow with its performance (nifi-api response, UI etc). Recently we have added lot of processors but at any point of time the processor which are processing data will be only 10% of total processors. Will nifi perf goes down with increase in number of processors though 90% total processors will be idle? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
01-31-2023
10:38 PM
Hello Experts, I am using nifi-api to update the variable registry of specific processor groups. It was working as expected, but in recent time its taking too long (some time infinite) time to complete the variable registry update. However if I restart nifi and try again this var registry update api it works quickly as expected. Is it because of increased number/size of variable registry or any resource crunch ? Could you please suggest a solution on this ? Any suggestion is much appreciated in advance. Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
12-12-2022
09:16 PM
1 Kudo
@steven-matison - thank you. Found out the reason for NoClassdef error it was due to the missing nar dependency in my NAR generation project pom. Need to add "nifi-mongodb-nar" dependency to make use of these mongo related classes as they are not part of standard nar. But I am already using "nifi-standard-nar" as dependency in NAR project I cant use one more, so had drop down this idea and write my own Mongo controller service.
... View more
12-12-2022
03:16 AM
Hello Experts, I have custom processor code where I am using 2 controller services as input property. DBCPService (jdbc connection pool) works fine without any issue, but when I try to take input as MongoDBClientService (mongodb connection pool controller service) I get MongoDBClientService NoClassDefFoundError error during project build. public class TrackRequestStatus extends AbstractProcessor { public static final PropertyDescriptor DBDC_SERVICE = new PropertyDescriptor.Builder() .name("JDBC Connection Pool") .description("JDBC Connection Pool") .required(false) .dynamic(true) .identifiesControllerService(DBCPService.class) //.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .build(); static final PropertyDescriptor CLIENT_SERVICE = new PropertyDescriptor.Builder() .name("mongo-client-service") .displayName("MongoDB Client Service") .description("If configured, this property will use the assigned client service for connection pooling.") .required(false) .identifiesControllerService(MongoDBClientService.class) .build(); <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-standard-bundle --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-standard-bundle</artifactId> <version>${nifi16.version}</version> <type>pom</type> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-dbcp-service-nar --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-dbcp-service</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-dbcp-service-api --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-dbcp-service-api</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> <!--<dependency> <groupId>org.mongodb</groupId> <artifactId>mongodb-driver-legacy</artifactId> <version>4.7.2</version> </dependency>--> <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-mongodb-bundle --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-mongodb-bundle</artifactId> <version>${nifi16.version}</version> <type>pom</type> </dependency> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-mongodb-client-service-api</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-mongodb-services</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> [INFO] Generating documentation for NiFi extensions in the NAR... [INFO] Found NAR dependency of org.apache.nifi:nifi-standard-nar:nar:1.16.3:compile [INFO] Found NAR dependency of org.apache.nifi:nifi-standard-services-api-nar:nar:1.16.3:compile [INFO] Found NAR dependency of org.apache.nifi:nifi-jetty-bundle:nar:1.16.3:compile [INFO] Found a dependency on version 1.16.3 of NiFi API [WARNING] Could not generate extensions' documentation java.lang.NoClassDefFoundError: org/apache/nifi/mongodb/MongoDBClientService at com.o9solutions.nifi.processors.o9_custom_processors.TrackRequestStatus.<clinit> (TrackRequestStatus.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance (Constructor.java:423) at java.lang.Class.newInstance (Class.java:442) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:629) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:605) at org.apache.nifi.NarMojo.generateDocumentation (NarMojo.java:577) at org.apache.nifi.NarMojo.execute (NarMojo.java:499) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) Caused by: java.lang.ClassNotFoundException: org.apache.nifi.mongodb.MongoDBClientService at java.net.URLClassLoader.findClass (URLClassLoader.java:387) at java.lang.ClassLoader.loadClass (ClassLoader.java:418) at java.lang.ClassLoader.loadClass (ClassLoader.java:351) at com.o9solutions.nifi.processors.o9_custom_processors.TrackRequestStatus.<clinit> (TrackRequestStatus.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance (Constructor.java:423) at java.lang.Class.newInstance (Class.java:442) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:629) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:605) at org.apache.nifi.NarMojo.generateDocumentation (NarMojo.java:577) at org.apache.nifi.NarMojo.execute (NarMojo.java:499) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) [INFO] Building jar: D:\Project\Platform\Source\IntelliJ-O9Git\Nifi\nifi-dev-repo\atocl\o9_custom_processors\nifi-o9_custom_processors-nar\target\nifi-o9_custom_processors-nar-nifi16-1.0-SNAPSHOT.nar [INFO] [INFO] --- maven-site-plugin:3.11.0:attach-descriptor (attach-descriptor) @ nifi-o9_custom_processors-nar --- [INFO] Skipping because packaging 'nar' is not pom. [INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ nifi-o9_custom_processors-nar --- Can some one please help to resolve this ? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
11-22-2022
09:43 PM
@Green_ Thank you so much ! It worked as expected.
... View more
11-16-2022
08:36 PM
Thanks @Green_ . That did not work for me
... View more
11-15-2022
10:26 AM
Hello experts, I am using below api to create and update variable registry variables.. /nifi-api/process-groups/" + processGroupId + "/variable-registry/update-requests But I am unable to delete any already created variable from specific processor group variable registry using nifi api. Which api I am suppose to use to delete a variable registry variables? Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
11-03-2022
04:32 AM
@MattWho @araujo @smdas Just tagging top solution authors to notify
... View more
11-02-2022
04:46 AM
Hello Experts, I need to execute java class or a java method during Nifi bootup. This is to register/integrate my org health check sdk (which has to be done at nifi bootup time) Is there any option to invoke/execute my java code snippet or a java class during bootup ? Any suggestion would be appreciated. One option I have is to write a custom logback appender class and add that custom appender is logback.xml so nifi will run this during bootup. I am looking for better option. Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
08-09-2022
08:54 PM
Hello Experts, I am new to Flink CEP and trying to test basic things - In below code my expectation is all the input should matched in patter and should print as matched result. But somehow nothing is matching ('matechedStream.print()') any idea about the reason ? Any suggestion/help would be much appreciated. package com.o9.flink;
import com.o9.flink.asyncio.DemandSupply;
import org.apache.flink.cep.CEP;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.apache.flink.cep.PatternStream;
import org.apache.flink.cep.functions.PatternProcessFunction;
import org.apache.flink.cep.pattern.Pattern;
import org.apache.flink.cep.pattern.conditions.SimpleCondition;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.util.Collector;
public class DemandSupplyPattern {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
DataStream<String> keyedInputStream = env.fromElements("AAA","BBB","CCC");
Pattern<String, ?> dspattern = Pattern.<String>begin("start");
PatternStream<String> patternStream = CEP.pattern(keyedInputStream, dspattern);
DataStream<String> matechedStream = patternStream.process(new PatternProcessFunction<String, String>() {
@Override
public void processMatch(Map<String, List<String>> map, Context context, Collector<String> collector) throws Exception {
collector.collect(map.get("start").toString());
}
});
matechedStream.print();
env.execute("DemandSupply-CEP");
}
} <dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime</artifactId>
<version>${flink.version}</version>
<!--<scope>test</scope>-->
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope>-->
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-avro</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope>-->
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-statebackend-rocksdb</artifactId>
<version>${flink.version}</version>
<!--<scope>test</scope>-->
</dependency>
<dependency>
<groupId>org.asynchttpclient</groupId>
<artifactId>async-http-client</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-cep</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope>-->
</dependency>
</dependencies>
... View more
Labels:
- Labels:
-
Apache NiFi