Member since
07-19-2018
613
Posts
100
Kudos Received
117
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3701 | 01-11-2021 05:54 AM | |
2586 | 01-11-2021 05:52 AM | |
6978 | 01-08-2021 05:23 AM | |
6432 | 01-04-2021 04:08 AM | |
29505 | 12-18-2020 05:42 AM |
08-21-2018
11:16 AM
You are going to have complete the setup of the standardSSLContextService. You can find a writeup about Google HTTPS and using the jvm cacerts for the truststore here: https://community.hortonworks.com/articles/61180/streaming-ingest-of-google-sheets-into-a-connected.html If this answer is helpful, please choose ACCEPT to mark your question answered.
... View more
08-20-2018
12:03 PM
@Veerendra Nath Jasthi In PutEmail do you have AttachFile true or false? That attribute is underneath the scroll in the processor modal window so maybe you missed that setting? If you are trying to get the content of the flowfile into the message body, without an attachment, you will need to use the ExtractText processor to get the text from the content of a flow file into an attribute. You should be able to add a property like "message.body" with a value of "(.*)" to ExtractText to get it to extract everything. Then use the PutEmail Attribute Include All Attributes In Message set to True.
... View more
08-20-2018
11:16 AM
This should work (no brackets): /homepath/customer_*/inbox/ Another solution would be to create a separate folder for example: /listFiles/. Then configure your processor to hit that folder with Recurse Subdirectories TRUE. Then inside that folder create symlinks to the real folders you want: ln -s /homepath/customer_A/inbox/ /listFiles/customer_A_inbox/ ln -s /homepath/customer_B/inbox/ /listFiles/customer_B_inbox/ ln -s /homepath/customer_C/inbox/ /listFiles/customer_C_inbox/ ln -s /homepath/customer_D/inbox/ /listFiles/customer_D_inbox/ If this answer is helpful, please choose Accept to mark your question as answered.
... View more
08-17-2018
12:00 PM
@Errico
Malatesta good job, glad you were able to work it out!
... View more
08-17-2018
11:59 AM
1 Kudo
@Dhruvil Dhankani I think you are looking for the processor EvaluateJsonPath where you can define payload.after.id and payload.op as follows when sending json content file from kafka: payload.after.id: $.payload.after.id payload.op: $.payload.op Applying an avro schema to the json and using record readers is another beast so let us know if that is what you are looking for. If this answer is helpful, please choose accept to mark it as answered.
... View more
08-17-2018
11:52 AM
@Guven
Guvenal this is a great custom processor, you should create a HOWTO Article here about how to use it and or create one about how to make Custom Processors.
... View more
08-16-2018
05:20 PM
Accept the answer to make it answered for others. You are correct, the particular implementation for your use case scenario can be accomplished in many ways. The morale should be that NiFi and other hadoop based components are true winners. Also working with new components keeps things exciting. You would not be making an unwise decision to embrace NiFi.
... View more
08-16-2018
04:53 PM
The solution here is to use 2 nodes for 2.6 and 2 more nodes for 3.0. 2 separate clusters. It is not possible to have 2 versions running on same 2 nodes nor any manner to swap between two. That is just not how this works.
... View more
08-16-2018
04:17 PM
@Gavin
Gilmour
I am currently using NiFi to solve your criteria. A NiFi Processor Group handles Elasticsearch (ScrollElasticsearchHttp) -> attribute/string of XML -> Parsed XML Object as Json -> Additional Data From SQL -> Merged Data -> Kafka Topic. The Kafka Topic is configured for Data Compaction (de-duplication) based on an index id. A Second NiFi Processor Group uses GetKafka and sends those unique results to PutElasticSearchHttp creating the final unique elastic document. There are many ways to skin this cat, but to answer your questions: 1 NiFi is excellent for multiple data sources and the approach is extremely feasible. 2. Data compaction and "exactly once" via kafka is just some basic topic settings. 3. The NiFi Scroll ElasticSearch processor has a state allowing it to roll through an index indefinitely never skipping any ids. If this answer is helpful, please choose Accept.
... View more
08-16-2018
03:21 PM
@Veerendra Nath Jasthi In one of my NiFi Data Flows (multiple process groups) I create a Fail Output port in each group, and route that to a Notification Process Group's input port. which uses PutEmail Processor. If you are working in a single Process Group, just route all of your Failures to the same PutEmail Processor. If this answer is helpful, please click ACCEPT.
... View more