Member since
01-12-2018
10
Posts
1
Kudos Received
0
Solutions
01-18-2018
08:56 AM
Hi, If my flowfile contains json, which at JsonPath does not have array, it's routed to failure. But lets assume, that I don't know, whether I get single record or array of recods. Would it make sense to create property to control behavior in such case? User could then decide, whether it's an error, or whether unchanged flow should be routed to original relationship (or any other relationship) Or is there any other way, how to test flowfile, whether there is at given JsonPath array or something else? Thanks! (if change described in first paragraph is necessary and would be accepted, I can do the coding, np).
... View more
01-15-2018
08:59 AM
Thanks a lot! This helped me, based on this hint everything works as it should. Thanks again. Martin.
... View more
01-12-2018
04:52 PM
after immense trial&error rounds, we moved forward and currently face this. If we do not provide this dependency in pom.xml <dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record-serialization-service-api</artifactId>
<scope>compile</scope>
</dependency>
then nar cannot be deployed, because those interfaces are missing. If we leave out compile, nar cannot be deployed, since interfaces are not on classpath. If we keep compile there, interface will be there, but it will be considered to be different version and implementators should extend our interface defined in out built nar. Is this a bug or what? I assume I should be able to add just: <dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record-serialization-service-api</artifactId>
</dependency>
or what am I doing wrong? EDIT: current pom.xms: • packaging pom does not have anything interresting • pom.xml of nar module <?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
...
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>???</groupId>
<artifactId>???</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<artifactId>???</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>nar</packaging>
<properties>
<maven.javadoc.skip>true</maven.javadoc.skip>
<source.skip>true</source.skip>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record-serialization-service-api</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-schema-registry-service-api</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>cz.embedit</groupId>
<artifactId>nifi-fdp-processors</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
</project>
• pom.xml of processors module <?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
...
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>???</groupId>
<artifactId>???</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<artifactId>???</artifactId>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-processors</artifactId>
<version>${nifi.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record-serialization-service-api</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-avro-record-utils</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-schema-registry-service-api</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-utils</artifactId>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</dependency>
</dependencies>
</project>
... View more
01-12-2018
03:15 PM
Hi, we have issues with standard nifi processors, and have to fix them ourselves. I don't want to delve into details why, please take it as an axiom, imagine that I'm creating something from scratch instead. So I took nifi 1.4.0 archetype to generate project. I created some custom procesor on my own, which does not have any extra dependencies, and validated, that it works in nifi. All good. Now lets move on to nifi code. I took ValidateRecord file, and copy-pasted it into our project. I copy pasted also all dependencies from pom.xml module ValidateRecord is in. Our maven project and ValidateRecord maven project has now same parent, same dependencies. Both builds just fine. When I tried to deploy it, it fails, because of missing dependencies, for example on nifi-record. I build our project using mvn clean install in root project. Built nar file does NOT contain respective jar (nifi-record). When I build nifi, also nifi-standard-processors does not contain this jar. I'd consider it normal, since maven parent prescribes provided. We have this dependency in pom.xml: <dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record</artifactId>
<scope>compile</scope>
</dependency> (we added compile in despair, without it it does not work equally) Question one: how do you declare dependency in nifi/nar if you want it to be available and nar can be deployed?? Now into more wild stuff: if I run mvn nifi-nar:nar it will generate nar files in all 3 target dirs. In root dir with pom packaging it's pure nonsense, in processor's module, with jar packaging there is nar file, which contains blend of normal build and nar build, and it CONTAINS nifi-record library. And finally in packaging module there is nar without nifi-record; same as from mvn clean install. To conclude: provided, that I work with official archetype and official source code, while using identical dependencies definition, it's surprising that I'm not able to get working code. What am I doing wrong? (aside of copypasting, but I don't have any other choice) Thanks!
... View more
Labels:
- Labels:
-
Apache NiFi