1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1999 | 04-03-2024 06:39 AM | |
| 3171 | 01-12-2024 08:19 AM | |
| 1725 | 12-07-2023 01:49 PM | |
| 2505 | 08-02-2023 07:30 AM | |
| 3517 | 03-29-2023 01:22 PM |
10-20-2016
08:05 PM
1 Kudo
Here is an example of using the SQOOP API https://gist.github.com/umbertogriffo/5ddb75bda34a99a427ff Just use NIFI, why write a whole Java app? Or just schedule SQOOP with OOZIE
... View more
10-20-2016
01:05 PM
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ListenHTTP/ that is what you want it will get anything they send as a file
... View more
10-19-2016
08:00 PM
2 Kudos
Apache Phoenix https://phoenix.apache.org/faq.html SparkSQL is probably what you are looking for. http://spark.apache.org/sql/ Hortonworks is 100% open source apache. It can run Apache Impala, Apache Drill, Hive LLAP, Phoenix, Apache HAWQ, SparkSQL, Presto.
... View more
10-19-2016
07:48 PM
1994148034455292.txt
... View more
10-19-2016
07:34 PM
I'll take a look at the exact messages sent, but it's what's pulled in by ConsumePOP3.
... View more
10-19-2016
07:33 PM
https://community.hortonworks.com/questions/58848/rfc-2822-validation-error.html http://apache-nifi.1125220.n5.nabble.com/Regarding-ConsumeIMAP-Processor-td12249.html
... View more
10-19-2016
07:07 PM
When I route to ExtractEmailHeaders and Attachments, they always fail. Is there a specific type of email server or extra processing required? Example email with no attachments
Apache NiFi Tutorials
The following tutorials show you how to work with different
processors, create dataflows and display data in various dashboards.
There’s also an intro to MiniFi which is a smaller NiFi to run on
embedded or tiny devices.
Using Apache NiFi with Apache Kafka Best Practices and Tutorial
Using Apache NiFi with Sensors Tutorial
Change Data Capture (CDC) with Apache NiFi
An Introduction to Apache NiFi 1.0.0
Apache NiFi 1.0.0 Crash Course
DataFlow with Apache NiFi (includes Video)
Real-Time Streaming Recommendations with NiFi, Kafka and Spark
Setting up Apache NiFi 1.0.0 Zero Master Clusters
Getting Started with HDF 2.0
Creating a Kibana Dashboard of Twitter Data Pushed with NiFi
NiFi For Geoenrichment and Routing of Log Messages
Running Google Vision From Apache NiFi
Getting Started with MiniFi
How to Add Kerberos Authentication to Apache NiFi 1.0.0
Converting CSV Data to AVRO with Apache NiFi
nifi-app.log:2016-10-19 18:56:09,054 ERROR [Timer-Driven Process Thread-6] o.a.n.p.email.ExtractEmailAttachments
nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments$1.process(ExtractEmailAttachments.java:138) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments.onTrigger(ExtractEmailAttachments.java:124) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log:2016-10-19 18:56:09,054 ERROR [Timer-Driven Process Thread-1] o.a.n.p.email.ExtractEmailHeaders
nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders$1.process(ExtractEmailHeaders.java:160) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders.onTrigger(ExtractEmailHeaders.java:146) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log:2016-10-19 18:56:09,056 ERROR [Timer-Driven Process Thread-10] o.a.n.p.email.ExtractEmailAttachments ExtractEmailAttachments[id=31a043b1-d7a6-1806-80ef-3040b8bc0bce] Could not parse the flowfile StandardFlowFileRecord[uuid=a963b2b3-4cb1-4155-a251-b0ae2089299a,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1476903369035-1627, container=default, section=603], offset=16199, length=45886],offset=0,name=1994148034455292,size=45886] as an email, treating as failure: javax.mail.MessagingException: Message failed RFC2822 validation
nifi-app.log:2016-10-19 18:56:09,056 ERROR [Timer-Driven Process Thread-4] o.a.n.p.email.ExtractEmailHeaders ExtractEmailHeaders[id=31a043b2-d7a6-1806-e673-2e2b86f5c83e] Could not parse the flowfile StandardFlowFileRecord[uuid=eed4f0e7-d63f-4f5a-9b1c-986087bc896e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1476903369035-1627, container=default, section=603], offset=16199, length=45886],offset=0,name=1994148034455292,size=45886] as an email, treating as failure: javax.mail.MessagingException: Message failed RFC2822 validation
nifi-app.log:2016-10-19 18:56:09,057 ERROR [Timer-Driven Process Thread-10] o.a.n.p.email.ExtractEmailAttachments
nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments$1.process(ExtractEmailAttachments.java:138) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments.onTrigger(ExtractEmailAttachments.java:124) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log:2016-10-19 18:56:09,057 ERROR [Timer-Driven Process Thread-4] o.a.n.p.email.ExtractEmailHeaders
nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders$1.process(ExtractEmailHeaders.java:160) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders.onTrigger(ExtractEmailHeaders.java:146) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
Example partial email with encoding --001a11404b903cb34b053f3b322c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
- Hortonworks DataFlow Webinar Series
<http://hortonworks.com/blog/hortonworks-dataflow-webinar-series/>Octobe=
r
18, 2016
HDF makes streaming analytics faster and easier, by enabling accelerated
data collection, curation, analysis and delivery in real-time, on-premis=
es
or in the cloud through an integrated solution with Apache NiFi, Kaf
... View more
Labels:
- Labels:
-
Apache NiFi
10-19-2016
06:03 PM
I like Bryan's suggestion. That's a good model for IoT as well with remote notes messaging in. You could have send messages between outside cluster and an inside secure cluster via MQTT, JMS, Kafka, SiteToSite. Then just one port and one controller set of IPs communicating with each other. An IoT or security gateway.
... View more
10-19-2016
05:46 PM
http://www.tutorialspoint.com/unix_commands/hexdump.htm could be called from executestreamcommand
... View more
10-18-2016
07:51 PM
looks like a permissions issue make sure that this exists and the current user has access hdfs dfs -mkdir /tmp hdfs dfs -chmod -R 777 /tmp are you in as root? http://kitesdk.org/docs/1.0.0/Using-the-Kite-CLI-to-Create-a-Dataset.html Could be a kite error. http://mail-archives.apache.org/mod_mbox/crunch-dev/201303.mbox/%3CBLU0-SMTP1468E6756741B738043FB98ADF90@phx.gbl%3E Nifi, Hive or Pig may be a better option. http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/ Looks like it may be Kite, may need to upgrade that https://issues.cloudera.org/browse/KITE-874
... View more