Member since
12-03-2017
156
Posts
26
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2074 | 11-03-2023 12:17 AM | |
4137 | 12-12-2022 09:16 PM | |
1562 | 07-14-2022 03:25 AM | |
2389 | 07-28-2021 04:42 AM | |
3316 | 06-23-2020 10:08 PM |
07-28-2021
04:42 AM
Update : Below commands worked : git config user.name "myemailuser" git config user.email "myemail@myorg.com" Need to run this inside the cloned repo folder and then start nifi registry.
... View more
07-28-2021
02:17 AM
Hello @alim , Thanks for the detailed explanation. Do you know any solution for this problem - https://community.cloudera.com/t5/Support-Questions/NiFi-registry-commits-templates-to-git-with-default-email/td-p/321648 Thanks Mahendra
... View more
07-27-2021
10:55 PM
Hello Experts, I have configured Git as FlowPersisteceProvider for nifi registry. Tested with test repo and worked fine, but now pointed to my company repo and getting error in push/commit. The error is due to the policy is set in my org git repo, it accepts push/commit from the accout with email as *@myorg.com only. But Even though I configured my credentials (user & PAT) which belongs to my org for Git access from registry, NiFi registry pushes the code to Git with email as "nifiregistry@<hostname>". As its org level git repo ops team cant change this policy, so is there any way I can override this default "nifiregistry@<hostname>" to "my-user@myorg.com" so that git push will not be rejected ? This is how nifi reg is pushing and comitting, Authored and Committed Identity should be changed from default Tried this : https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.2.0/administering-nifi-registry/content/git-user-configuration.html But did not solve Error in registry log : Caused by: org.eclipse.jgit.errors.TransportException: https://o9git.visualstudio.com/CoreDev/_git/nifi-registry-dev: error occurred during unpacking on the remote end: error VS403702: The push was rejected because one or more commits contain author email 'nifiregistry@nifi-test-poc.*****.com' which does not match the policy-specified patterns. at org.eclipse.jgit.transport.BasePackPushConnection.readStatusReport(BasePackPushConnection.java:382) at org.eclipse.jgit.transport.BasePackPushConnection.doPush(BasePackPushConnection.java:222) at org.eclipse.jgit.transport.TransportHttp$SmartHttpPushConnection.doPush(TransportHttp.java:1112) at org.eclipse.jgit.transport.BasePackPushConnection.push(BasePackPushConnection.java:170) at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:172) at org.eclipse.jgit.transport.Transport.push(Transport.java:1344) at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:169) ... 8 common frames omitted Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
07-08-2021
10:51 PM
@Ven5 - Login to git in browser and create branch called 'develop'. - Login to server(node) where registry is running, got to '/versioned_flows' directory - Clone git repo with newly created specific branch >git clone -b develop https://github.com/myrepo-name.git Thats it. you should see your templates committing to 'develop' branch instead of default master branch.
... View more
06-01-2021
08:29 AM
Hello experts, I have nifi registry with MySQL db as metadata store and FileSystem as Persistence provider. And this registry has 10+ flows with multiple versions. Now I am trying to integrate registry to GIT, that is trying to change persistence provider from FileSystem to GIT. I referred some docs (below urls) and steps to switch persistence provider etc, those all help for fresh GIT integration set and they wont carry/migrate already created buckets/flows/versions from FileSystem to GIT https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#switching-from-other-flow-persistence-provider https://community.cloudera.com/t5/Community-Articles/Storing-Apache-NiFi-Versioned-Flows-in-a-Git-Repository/ta-p/248713 Any help on this would be much appreciated. Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
06-01-2021
07:45 AM
Thanks for the great explanation Matt, much appreciated.
... View more
05-26-2021
09:26 PM
Hello, I am trying to migrate my Nifi registry from one server to another. Used niFi toolkit CLI for the same. Below are the commands used - On source registry server : registry list-buckets registry list-flows On target registr server: registry create-flow registry sync-flow-versions Flows with versions got migrated properly, but one issues noticed is the version date on target server. Instead of actual date of version it shows migrated date for all versions in target server after migration. Find the attached screnn shot for ref. Is there any way to fix this? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
05-24-2021
10:07 AM
Hello, I am trying to use nifi toolkit cli to list the bucket in Nifi registry using below command - /usr/hdf/3.4.1.1-4/nifi-toolkit/bin/cli.sh registry list-buckets -u http://<ip>:<port> But getting below error : /usr/hdf/3.4.1.1-4/nifi-toolkit$ /usr/hdf/3.4.1.1-4/nifi-toolkit/bin/cli.sh registry list-buckets -u http://<ip>:<port> cli.sh: JAVA_HOME not set; results may vary Exception in thread "main" java.lang.NoClassDefFoundError: javax/annotation/Priority at org.glassfish.jersey.model.internal.ComponentBag.modelFor(ComponentBag.java:551) at org.glassfish.jersey.model.internal.ComponentBag.lambda$registerModel$10(ComponentBag.java:477) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:229) at org.glassfish.jersey.model.internal.ComponentBag.registerModel(ComponentBag.java:469) at org.glassfish.jersey.model.internal.ComponentBag.register(ComponentBag.java:380) at org.glassfish.jersey.model.internal.CommonConfig.register(CommonConfig.java:454) at org.glassfish.jersey.client.ClientConfig$State.register(ClientConfig.java:242) at org.glassfish.jersey.client.ClientConfig.register(ClientConfig.java:619) at org.apache.nifi.registry.client.impl.JerseyNiFiRegistryClient.<init>(JerseyNiFiRegistryClient.java:110) at org.apache.nifi.registry.client.impl.JerseyNiFiRegistryClient.<init>(JerseyNiFiRegistryClient.java:54) at org.apache.nifi.registry.client.impl.JerseyNiFiRegistryClient$Builder.build(JerseyNiFiRegistryClient.java:226) at org.apache.nifi.toolkit.cli.impl.client.NiFiRegistryClientFactory.createClient(NiFiRegistryClientFactory.java:96) at org.apache.nifi.toolkit.cli.impl.client.NiFiRegistryClientFactory.createClient(NiFiRegistryClientFactory.java:39) at org.apache.nifi.toolkit.cli.impl.command.registry.AbstractNiFiRegistryCommand.doExecute(AbstractNiFiRegistryCommand.java:55) at org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74) at org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:251) at org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:232) at org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:187) at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145) at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72) Caused by: java.lang.ClassNotFoundException: javax.annotation.Priority at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base@9-internal/BuiltinClassLoader.java:366) at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base@9-internal/ClassLoaders.java:184) at java.lang.ClassLoader.loadClass(java.base@9-internal/ClassLoader.java:419) ... 22 more u This machine has Java 9 openJDK. java -version openjdk version "9-internal" OpenJDK Runtime Environment (build 9-internal+0-2016-04-14-195246.buildd.src) OpenJDK 64-Bit Server VM (build 9-internal+0-2016-04-14-195246.buildd.src, mixed mode) Any help would be appreciated. Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
06-23-2020
10:08 PM
2 Kudos
@AJ121 - Please try this : ${datestring:toDate("yyyyMMdd","GMT"):format("yyyy-MM-dd")} Please accept if it helped/solved your problem.
... View more
06-17-2020
11:43 PM
1 Kudo
Hi @SirV , I see there are 2 possible options : 1. Merge two flow files based on common key ('FALLA_ID') using MergeContent processor : - Use EvaluateJsonPath first to get 'FALLA_ID' value to flow file attribute. - Use MergeContent processor to merge master-detail flow files, you need to use above step extracted FALLA_ID value in 'Correlation Attribute Name' filed of MergeContent processor, so that it always merge flow files based on common FALL_ID value, so that you can get single merged file for each FALL_ID. - Use JOLTTransformJson to transform your merged json to desired format of output json. 2. Cache the first flow file content in to cache with key as 'FALLA_ID' value and merge when second flow file arrives : - Use NiFi DistributedMap Cache (or any other external cache like Ignite) to cache the first flow. (It will be key-value pair in cache, so use key as FALL_ID and value as whole flow file content.) Before caching the FF just check if that key is already present in cache, if already present means first (master/details) file has already arrived, so you can read that file and dont need to cache the current flow file. - Now you have 1 file in FF Content and a file in FF Attribute (which is read from cache), now you can use ExceuteScript and write simple script (of your choice - python/groovy) to club FF content and attributes to form a desired output json Note : This cache approach has to be picked carefully based on your file volumes and content size etc else it may fill up your memory. Also if you are in multi node cluster mode, NiFi DistributedCache will be independent for each node and does not interact with other nodes so if master file and details files get picked by different nodes then logic will fail ! Please ACCEPT if it helps/resolves your problem. Thanks Mahendra
... View more