Member since
07-19-2018
613
Posts
101
Kudos Received
117
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5686 | 01-11-2021 05:54 AM | |
| 3811 | 01-11-2021 05:52 AM | |
| 9485 | 01-08-2021 05:23 AM | |
| 9283 | 01-04-2021 04:08 AM | |
| 38599 | 12-18-2020 05:42 AM |
12-22-2020
11:42 AM
Amazing work here sir!
... View more
12-18-2020
05:42 AM
1 Kudo
Excellent news. Can you accept the first 2 responses to close this solution?
... View more
12-18-2020
05:40 AM
1 Kudo
@hakansan The error is stating your hard drive is full: could not write to file "pg_logical/replorigin_checkpoint.tmp": No space left on device "no space left on device" The solution you need is to investigate cleaning out some files to free up space, expanding disk, etc. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
12-17-2020
01:33 PM
Thats weird.. did you scroll the attributes list down (just in case its that simple)? i cant see the full modal so not sure. The config of each attribute looks okay so does the json.
... View more
12-17-2020
05:58 AM
1 Kudo
@justenji Great response! @GMAN I have some templates that may help you get a head start: https://github.com/steven-matison/NiFi-Templates There are 2 InvokeHttp Examples.
... View more
12-04-2020
05:16 AM
@SandeepG01 Ahh no fun with bad filenames. Space in filename is highly not recommended in these days and times. That said, a solution you might try is to \ (backslash) the space. Especially in the context of passing the filename in flowfile attributes. If you still need to allow spaces and cannot resolve upstream (do not use spaces), i might suggest submitting your experience over on the NiFI jira as a bug: https://issues.apache.org/jira/projects/NIFI/issues If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
12-01-2020
08:28 AM
1 Kudo
The problem is that you need something to store the dynamic schemas in. That is where the Schema Registry comes in as it provides a UI and api to add/update/delete schemas. These can then be refrenced from NiFi. It looks like AvroSchemaRegistry allows you to do the similar, minus the ui/api. So you would need to create your schema in your flow, as attribute, and send that to AvroRecorderReader configured against AvroSchemaRegistry. You could use some other data store to store these schemas, but you would need to pull them out into an attribute of the same name configured in the Reader and Registry. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-registry-nar/1.12.1/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html The latter method does not give you a way to manage all the schemas, which is why I reference the Hortonworks Schema Registry which does include ability to manage, version actual schemas.
... View more
12-01-2020
07:49 AM
You can leverage "Attributes to send" or if you stop processor and click + you can add custom attributes right on bottom of processor config. If you are not getting anything out of response (failure, retry, no-retry, etc) then you definitely have a connectivity issue from Nifi outbound...
... View more
12-01-2020
05:09 AM
@Ksad Excellent work to completely show us what you have. Also, excellent work to test and confirm your request work in postman first. This is always one of the first things I do to make sure i have a valid test connection and all settings to connect to the API before attempting with InvokeHttp. When you take this route, and you cannot get a response, this indicates a networking issue with Nifi to [salesforce domain]. You should test command line from NiFi node to [salesforce domain] using curl, wget, telnet, etc. Next if you can confirm connectivity, attempt to adjust the processor time outs. Some systems need longer than defaults. For example I sometimes set to 50 and 150 by just adding a 0 to the 2 values (connection and read timeout). f it did time out it should throw an error. You can also set the processor log level to debug to expose more verbose output in the NiFi UI. Last but not least, tail the nifi-app.log file while doing all nifi flow debugging. Sometimes more useful information is found there. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
11-30-2020
05:02 AM
@naga_satish yes, what you are looking for is the Schema Registry: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.0.0/bk_schema-registry-user-guide/content/ch_integrating-schema-registry.html The schema registry can be configured in NiFI, then the schema you create there are available in NiFi Record Readers and Writers. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more