Member since
12-03-2017
157
Posts
26
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2747 | 11-03-2023 12:17 AM | |
| 5082 | 12-12-2022 09:16 PM | |
| 2077 | 07-14-2022 03:25 AM | |
| 2886 | 07-28-2021 04:42 AM | |
| 4071 | 06-23-2020 10:08 PM |
12-12-2022
09:16 PM
1 Kudo
@steven-matison - thank you. Found out the reason for NoClassdef error it was due to the missing nar dependency in my NAR generation project pom. Need to add "nifi-mongodb-nar" dependency to make use of these mongo related classes as they are not part of standard nar. But I am already using "nifi-standard-nar" as dependency in NAR project I cant use one more, so had drop down this idea and write my own Mongo controller service.
... View more
12-12-2022
03:16 AM
Hello Experts, I have custom processor code where I am using 2 controller services as input property. DBCPService (jdbc connection pool) works fine without any issue, but when I try to take input as MongoDBClientService (mongodb connection pool controller service) I get MongoDBClientService NoClassDefFoundError error during project build. public class TrackRequestStatus extends AbstractProcessor { public static final PropertyDescriptor DBDC_SERVICE = new PropertyDescriptor.Builder() .name("JDBC Connection Pool") .description("JDBC Connection Pool") .required(false) .dynamic(true) .identifiesControllerService(DBCPService.class) //.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .build(); static final PropertyDescriptor CLIENT_SERVICE = new PropertyDescriptor.Builder() .name("mongo-client-service") .displayName("MongoDB Client Service") .description("If configured, this property will use the assigned client service for connection pooling.") .required(false) .identifiesControllerService(MongoDBClientService.class) .build(); <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-standard-bundle --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-standard-bundle</artifactId> <version>${nifi16.version}</version> <type>pom</type> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-dbcp-service-nar --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-dbcp-service</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-dbcp-service-api --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-dbcp-service-api</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> <!--<dependency> <groupId>org.mongodb</groupId> <artifactId>mongodb-driver-legacy</artifactId> <version>4.7.2</version> </dependency>--> <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-mongodb-bundle --> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-mongodb-bundle</artifactId> <version>${nifi16.version}</version> <type>pom</type> </dependency> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-mongodb-client-service-api</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-mongodb-services</artifactId> <version>${nifi16.version}</version> <scope>provided</scope> </dependency> [INFO] Generating documentation for NiFi extensions in the NAR... [INFO] Found NAR dependency of org.apache.nifi:nifi-standard-nar:nar:1.16.3:compile [INFO] Found NAR dependency of org.apache.nifi:nifi-standard-services-api-nar:nar:1.16.3:compile [INFO] Found NAR dependency of org.apache.nifi:nifi-jetty-bundle:nar:1.16.3:compile [INFO] Found a dependency on version 1.16.3 of NiFi API [WARNING] Could not generate extensions' documentation java.lang.NoClassDefFoundError: org/apache/nifi/mongodb/MongoDBClientService at com.o9solutions.nifi.processors.o9_custom_processors.TrackRequestStatus.<clinit> (TrackRequestStatus.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance (Constructor.java:423) at java.lang.Class.newInstance (Class.java:442) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:629) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:605) at org.apache.nifi.NarMojo.generateDocumentation (NarMojo.java:577) at org.apache.nifi.NarMojo.execute (NarMojo.java:499) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) Caused by: java.lang.ClassNotFoundException: org.apache.nifi.mongodb.MongoDBClientService at java.net.URLClassLoader.findClass (URLClassLoader.java:387) at java.lang.ClassLoader.loadClass (ClassLoader.java:418) at java.lang.ClassLoader.loadClass (ClassLoader.java:351) at com.o9solutions.nifi.processors.o9_custom_processors.TrackRequestStatus.<clinit> (TrackRequestStatus.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance (Constructor.java:423) at java.lang.Class.newInstance (Class.java:442) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:629) at org.apache.nifi.NarMojo.writeDocumentation (NarMojo.java:605) at org.apache.nifi.NarMojo.generateDocumentation (NarMojo.java:577) at org.apache.nifi.NarMojo.execute (NarMojo.java:499) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) [INFO] Building jar: D:\Project\Platform\Source\IntelliJ-O9Git\Nifi\nifi-dev-repo\atocl\o9_custom_processors\nifi-o9_custom_processors-nar\target\nifi-o9_custom_processors-nar-nifi16-1.0-SNAPSHOT.nar [INFO] [INFO] --- maven-site-plugin:3.11.0:attach-descriptor (attach-descriptor) @ nifi-o9_custom_processors-nar --- [INFO] Skipping because packaging 'nar' is not pom. [INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ nifi-o9_custom_processors-nar --- Can some one please help to resolve this ? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
11-22-2022
09:43 PM
@Green_ Thank you so much ! It worked as expected.
... View more
11-16-2022
08:36 PM
Thanks @Green_ . That did not work for me
... View more
11-15-2022
10:26 AM
Hello experts, I am using below api to create and update variable registry variables.. /nifi-api/process-groups/" + processGroupId + "/variable-registry/update-requests But I am unable to delete any already created variable from specific processor group variable registry using nifi api. Which api I am suppose to use to delete a variable registry variables? Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
08-09-2022
08:54 PM
Hello Experts, I am new to Flink CEP and trying to test basic things - In below code my expectation is all the input should matched in patter and should print as matched result. But somehow nothing is matching ('matechedStream.print()') any idea about the reason ? Any suggestion/help would be much appreciated. package com.o9.flink;
import com.o9.flink.asyncio.DemandSupply;
import org.apache.flink.cep.CEP;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.apache.flink.cep.PatternStream;
import org.apache.flink.cep.functions.PatternProcessFunction;
import org.apache.flink.cep.pattern.Pattern;
import org.apache.flink.cep.pattern.conditions.SimpleCondition;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.util.Collector;
public class DemandSupplyPattern {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
DataStream<String> keyedInputStream = env.fromElements("AAA","BBB","CCC");
Pattern<String, ?> dspattern = Pattern.<String>begin("start");
PatternStream<String> patternStream = CEP.pattern(keyedInputStream, dspattern);
DataStream<String> matechedStream = patternStream.process(new PatternProcessFunction<String, String>() {
@Override
public void processMatch(Map<String, List<String>> map, Context context, Collector<String> collector) throws Exception {
collector.collect(map.get("start").toString());
}
});
matechedStream.print();
env.execute("DemandSupply-CEP");
}
} <dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime</artifactId>
<version>${flink.version}</version>
<!--<scope>test</scope>-->
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope>-->
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-avro</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope>-->
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-statebackend-rocksdb</artifactId>
<version>${flink.version}</version>
<!--<scope>test</scope>-->
</dependency>
<dependency>
<groupId>org.asynchttpclient</groupId>
<artifactId>async-http-client</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-cep</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope>-->
</dependency>
</dependencies>
... View more
Labels:
- Labels:
-
Apache NiFi
08-01-2022
09:07 PM
1 Kudo
@MattWho - We have separated out ebs volumes of each repos and also 3 ebs volumes each for content and provenance repos. Now looks like issue is pretty much resolved ! Thanks for all your suggestions, helped a lot Thanks Mahendra
... View more
07-29-2022
07:29 PM
Thank you @MattWho . - 0 flow file in outbound connection. - no backpressure - nifi 1.12.1 version we are using - Don't see any logs related to this - We are using 500 gb disk and only 20gb was utilized at that time But all repos (flowfile, content and prov) are located in same ebs volume of 500gb, so is this could be something due to lack of iops availability?
... View more
07-28-2022
12:28 AM
Hello Experts, We have HandleHttpRequests processor running with 16 concurrency. Container queue size configured is 1000. While running a load test suddenly HandleHttpRequests stopped emitting flowfiles recieved and started throwing error show in below image - So we stopped the load test and now even after 30 mins when we send requests it will just emit 2 or 3 flow files and again shows same error and does not release/emit any request flowfiles. Ideally when no request is getting hit to that port (which HttpRequestListener is configured) container should get cleared and flow files should get created from queue and emit right? What could be the reason this processor is just trhowing container full error and not emitting any flow files ? It started working only when we manually cleanup all repositories (content, provenance, ff) and restart the nifi ! Any suggestion would be appreciated. Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-25-2022
09:40 PM
Experts, I am continuously seeing this log in Nifi app log - 2022-07-25 06:12:40,788 INFO [Timer-Driven Process Thread-219] o.a.n.c.repository.FileSystemRepository Unable to write to container default due to archive file size constraints; waiting for archive cleanup Any idea what is the reason and will there be any impact of this ? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi