Member since
09-11-2015
60
Posts
72
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1796 | 04-11-2017 06:18 PM | |
1442 | 02-21-2017 03:04 PM | |
2070 | 12-28-2016 10:21 PM | |
4557 | 11-30-2016 04:17 AM | |
6753 | 11-08-2016 02:42 PM |
11-30-2016
04:17 AM
1 Kudo
I've seen this before when you try to do a maven build while NiFi is running leaving a lingering process. Try running "jps" to see if there is a "NiFi" process still running. If so then run "kill X" where X is the PID of the process. After killing the process you should be able to delete the files and they will stay deleted.
... View more
11-08-2016
02:45 PM
1 Kudo
@Greg Keys great listing of articles. I'd suggest adding the "Apache NiFi in Depth" doc to that list as well: https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html
... View more
11-08-2016
02:42 PM
2 Kudos
Ingesting data from Facebook using NiFi before has been asked before. I'd suggest you start there: https://community.hortonworks.com/questions/43184/how-to-get-facebook-feeds-into-nifi.html As for detecting change, that is a whole other question/problem. It would require information about where you're eventually putting the Facebook data, how much data you're track, etc. in order to know how to query it to determine if it changed. I'd suggesting asking another question with those details (and any more you can share).
... View more
10-26-2016
09:46 PM
Ah you are right about the "toRadix" I was using it in a way that was wrong but still functioned correctly. It should be replaced with "toNumber()". What that expression is doing is formatting it as will be accepted as a "decimal" and then converting it to a whole number. For NIFI-2950 I am also adding a "fromRadix" function that will convert from one base to base 10. When you tried the expression, did you do it using a latest build off master or a released NiFi version?
I just posted a Pull Request (PR) adding proper support for whole number hex values and a fromRadix() function. Please, feel free to test and post feedback on the PR to github. This should allow you to simply do "${'$2':fromRadix(16)}". Here is the link: https://github.com/apache/nifi/pull/1161
... View more
10-26-2016
04:27 PM
Working with hexadecimal numbers is not something that is easily done in a current release of NiFi. In order to get it to work you'd need to use one of the scripting processors ExecuteScript or InvokeScriptedProcessor.
That said, doing numeric evaluations is one of my focuses in this upcoming release (which is currently being curated to be finalized) and I've been able to create a solution involving just the ReplaceText processor. I used the following configuration: Search Value: ^(\w*)\ *(\w*)\ *(\d*)\ *(\w*)$
Replacement Value: $1|${'$2':prepend('0x'):append('p0'):toNumber()}|$3|$4
Replacement Strategy: Regex Replace
Evaluation Mode: Line-by-line
The rest is up to your use-case (ie. which ever character set it is in). The search value will create capture groups for each of the sections. Then in the replacement value I utilize the second (the one for the hex digit) in an Expression language function to convert to base 10. The purpose of the "append" and "prepend" is that on the current master only decimals/double accept hex numbers (I need to improve that) so I just make it format it as a double. So it is unfortunate this use-case isn't currently handled out of the box, it soon will be!
... View more
10-24-2016
02:12 PM
By default the LogAttribute processor logs to the NiFi App log. It does this because processors are configured to log there by the default configuration in the logback.xml. In order to have a new file that just the LogAttribute Processor uses you can add a new Appender and Logger like so: <appender name="STATUS_LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logs/nifi-status.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>./logs/nifi-status_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>5</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<logger name="org.apache.nifi.processors.standard.LogAttribute" level="INFO" additivity="false">
<appender-ref ref="STATUS_LOG_FILE" />
</logger>
... View more
10-13-2016
09:12 PM
According to the curl man page, the "-d" denotes the data. So everything you have in the quotes after "-d" would have to be the content of the FlowFile (which gets posted as the body).
... View more
10-03-2016
03:31 AM
The user that corresponds to the cert you are using doesn't have access to the UI. An admin has to add the user to the "view the UI" policy first. You can read more about the Multi-tenant authorization in the NiFi docs: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#UI-with-multi-tenant-authorization
... View more
07-19-2016
02:34 PM
1 Kudo
When developing the MQTT processors I used Mosquitto to test. I found it to be a very easy to use and simple to configure broker that handled a decently high throughput even on my laptop. That said, the NiFi MQTT processors should be able to communicate with any broker that handles the vanilla MQTT Api.
... View more
07-18-2016
03:14 AM
2 Kudos
I believe you're missing a ":" after hive2 in your connect URL.
... View more