Member since
09-29-2015
871
Posts
723
Kudos Received
255
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4264 | 12-03-2018 02:26 PM | |
| 3206 | 10-16-2018 01:37 PM | |
| 4312 | 10-03-2018 06:34 PM | |
| 3172 | 09-05-2018 07:44 PM | |
| 2426 | 09-05-2018 07:31 PM |
01-03-2017
04:06 PM
PutSplunk has two modes of operating, it can send the entire content of the flow file as a single message, or it can stream the content of a flow file and separate it based on a delimiter. The way it chooses between these modes is based on whether or not the "Message Delimiter" property is set in PutSplunk. In your case I am assuming you have multiple JSON documents in a flow file, so you probably want to set the "Message Delimiter" to whatever is separating them, likely a \n.
... View more
12-21-2016
08:01 PM
1 Kudo
Shouldn't the FetchSFTP Remote File be set to ${path}/${filename} which are the attributes produced by ListSFTP?
... View more
12-21-2016
05:36 PM
1 Kudo
@Theena R @Anoop Nair I submitted a pull request for a FetchHBaseRow processor that may solve your use case: https://github.com/apache/nifi/pull/1349 Keep in mind this is "use at your own risk" right now as it has not been reviewed or merged into the NiFi codebase and is subject to change at any time. If you have any feedback, please comment on the JIRA: https://issues.apache.org/jira/browse/NIFI-1784
... View more
12-21-2016
02:27 PM
How many NiFi nodes are there? Can you see what is in /var/log/nifi/nifi-app.log for each of the NiFi nodes when this is happening?
... View more
12-21-2016
01:55 PM
@Theena R can you provide some input about your use-case... Are you receiving messages from somewhere (Kafaka/JMS/etc) and those messages have a row key that you want to fetch? Are you looking to fetch the whole row (i.e. multiple cells) or do you know a specific col-family:col-qualifier and only want to fetch a single cell? What type of output would you expect? something similar to the JSON representation that GetHBase produces?
... View more
12-20-2016
04:09 PM
There are two concepts, "indexed" and "stored"... If a field is only indexed and not stored, then you can search on it but can't get the original value of the field back for search results. If a field is only stored and not indexed, then you can't search on it, but you can use it as a return field for search results. If a field is both then it can be searched and can also be used as a return field as search results.
... View more
12-20-2016
03:39 PM
How are you planning to move the data between the servers? SFTP? Lets say you have NiFi running on Server A, and you use GetFile to pick up the files from your directory, and PutSFTP to push the files to Server B. GetFile writes the "filename" and "absolute.path" on each flow file it picks up, and you can then reference those in the PutSFTP Directory property as "${absolute.path}/${filename}" and then set "Create Directory" to true. This assumes Server B is allowing SFPT.
... View more
12-20-2016
02:54 PM
1 Kudo
Pretend you didn't have NiFi, and all you had was Solr, how would you do it? NiFi is not doing anything special here, it is streaming the content of the flow file (your .msg files) to Solr's /update/extract handler which is doing the extraction. This would be the same as you doing a curl command from a terminal to post a .msg file to Solr. Reading Solr's documentation for the request handler (https://wiki.apache.org/solr/ExtractingRequestHandler), it says... You may notice that although you can search on any of the text in the sample document, you may not be able to see that text when the document is retrieved. This is simply because the "content" field generated by Tika is mapped to the Solr field called "text", which is indexed but not stored. This is done via the default map rule in the /update/extract handler in solrconfig.xml and can be easily changed or overridden. For example, to store and see all metadata and content, execute the following: curl "http://localhost:8983/solr/update/extract?literal.id=doc1&uprefix=attr_&fmap.content=attr_content&commit=true" -F "myfile=@tutorial.html"
The uprefix=attr_ param causes all generated fields that aren't defined in the schema to be prefixed with attr_ (which is a dynamic field that is stored). The fmap.content=attr_content param overrides the default fmap.content=text causing the content to be added to the attr_content field instead.
... View more
12-16-2016
03:40 PM
The flow in the picture is the correct flow for load balancing, and it is what was meant by "with the proper dataflow configuration - load-balance it across the rest of the nodes" in the admin guide link above.
... View more
12-16-2016
03:38 PM
When you start the RPG it goes to the URL you entered and asks it for the info of all the cluster nodes one time, after that it is talking to all the nodes directly. So it doesn't matter if that node goes down while it is running because it already knows about all the nodes. It only matters if it goes down at the moment you start the RPG, which could be when your NiFi instance restarts or when the user clicks start in the UI.
... View more