Created 10-19-2016 07:07 PM
When I route to ExtractEmailHeaders and Attachments, they always fail. Is there a specific type of email server or extra processing required?
Example email with no attachments
Apache NiFi Tutorials The following tutorials show you how to work with different processors, create dataflows and display data in various dashboards. There’s also an intro to MiniFi which is a smaller NiFi to run on embedded or tiny devices. Using Apache NiFi with Apache Kafka Best Practices and Tutorial Using Apache NiFi with Sensors Tutorial Change Data Capture (CDC) with Apache NiFi An Introduction to Apache NiFi 1.0.0 Apache NiFi 1.0.0 Crash Course DataFlow with Apache NiFi (includes Video) Real-Time Streaming Recommendations with NiFi, Kafka and Spark Setting up Apache NiFi 1.0.0 Zero Master Clusters Getting Started with HDF 2.0 Creating a Kibana Dashboard of Twitter Data Pushed with NiFi NiFi For Geoenrichment and Routing of Log Messages Running Google Vision From Apache NiFi Getting Started with MiniFi How to Add Kerberos Authentication to Apache NiFi 1.0.0 Converting CSV Data to AVRO with Apache NiFi
nifi-app.log:2016-10-19 18:56:09,054 ERROR [Timer-Driven Process Thread-6] o.a.n.p.email.ExtractEmailAttachments nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments$1.process(ExtractEmailAttachments.java:138) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments.onTrigger(ExtractEmailAttachments.java:124) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log:2016-10-19 18:56:09,054 ERROR [Timer-Driven Process Thread-1] o.a.n.p.email.ExtractEmailHeaders nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders$1.process(ExtractEmailHeaders.java:160) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders.onTrigger(ExtractEmailHeaders.java:146) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log:2016-10-19 18:56:09,056 ERROR [Timer-Driven Process Thread-10] o.a.n.p.email.ExtractEmailAttachments ExtractEmailAttachments[id=31a043b1-d7a6-1806-80ef-3040b8bc0bce] Could not parse the flowfile StandardFlowFileRecord[uuid=a963b2b3-4cb1-4155-a251-b0ae2089299a,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1476903369035-1627, container=default, section=603], offset=16199, length=45886],offset=0,name=1994148034455292,size=45886] as an email, treating as failure: javax.mail.MessagingException: Message failed RFC2822 validation nifi-app.log:2016-10-19 18:56:09,056 ERROR [Timer-Driven Process Thread-4] o.a.n.p.email.ExtractEmailHeaders ExtractEmailHeaders[id=31a043b2-d7a6-1806-e673-2e2b86f5c83e] Could not parse the flowfile StandardFlowFileRecord[uuid=eed4f0e7-d63f-4f5a-9b1c-986087bc896e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1476903369035-1627, container=default, section=603], offset=16199, length=45886],offset=0,name=1994148034455292,size=45886] as an email, treating as failure: javax.mail.MessagingException: Message failed RFC2822 validation nifi-app.log:2016-10-19 18:56:09,057 ERROR [Timer-Driven Process Thread-10] o.a.n.p.email.ExtractEmailAttachments nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments$1.process(ExtractEmailAttachments.java:138) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailAttachments.onTrigger(ExtractEmailAttachments.java:124) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log:2016-10-19 18:56:09,057 ERROR [Timer-Driven Process Thread-4] o.a.n.p.email.ExtractEmailHeaders nifi-app.log:javax.mail.MessagingException: Message failed RFC2822 validation nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders$1.process(ExtractEmailHeaders.java:160) ~[nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579] nifi-app.log: at org.apache.nifi.processors.email.ExtractEmailHeaders.onTrigger(ExtractEmailHeaders.java:146) [nifi-email-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
Example partial email with encoding
--001a11404b903cb34b053f3b322c Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable - Hortonworks DataFlow Webinar Series <http://hortonworks.com/blog/hortonworks-dataflow-webinar-series/>Octobe= r 18, 2016 HDF makes streaming analytics faster and easier, by enabling accelerated data collection, curation, analysis and delivery in real-time, on-premis= es or in the cloud through an integrated solution with Apache NiFi, Kaf
Created 10-19-2016 07:16 PM
Timothy
It appears that the incoming email doesn't contain 'from' and/or 'sentDate'. Basically there is explicit validation for those fields inside the processor and if both missing the exception is triggered. The argument there is that it violates 'RFC2822' (Internet Message Format).
While it may be correct (I have to look at the doc itself) I am not sure that the goal for this processor is to enforce the mentioned spec.
Anyway, would you mind creating JIRA in https://issues.apache.org/jira/browse/NIFI and I can give it a closer look. Also, if you can find more details as to why the incoming message missing those field.
Cheers
Oleg
Created 10-19-2016 07:16 PM
Timothy
It appears that the incoming email doesn't contain 'from' and/or 'sentDate'. Basically there is explicit validation for those fields inside the processor and if both missing the exception is triggered. The argument there is that it violates 'RFC2822' (Internet Message Format).
While it may be correct (I have to look at the doc itself) I am not sure that the goal for this processor is to enforce the mentioned spec.
Anyway, would you mind creating JIRA in https://issues.apache.org/jira/browse/NIFI and I can give it a closer look. Also, if you can find more details as to why the incoming message missing those field.
Cheers
Oleg
Created 10-19-2016 07:34 PM
I'll take a look at the exact messages sent, but it's what's pulled in by ConsumePOP3.
Created 10-19-2016 07:37 PM
Yes, I understand and that's why I am having second thoughts about the check in the processor. In other words we may need to remove it
Created 10-19-2016 07:48 PM
Created 10-19-2016 07:33 PM
Created 10-24-2016 06:32 PM
nifi-bootstrap.log:2016-10-24 18:30:33,712 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: connecting to host "pop.x.com", port 110, isSSL false nifi-bootstrap.log:2016-10-24 18:30:33,936 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: PIPELINING enabled nifi-bootstrap.log:2016-10-24 18:30:33,936 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: authentication command trace suppressed nifi-bootstrap.log:2016-10-24 18:30:34,090 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: authentication command succeeded nifi-bootstrap.log:2016-10-24 18:30:38,072 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 1 nifi-bootstrap.log:2016-10-24 18:30:38,156 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 2 nifi-bootstrap.log:2016-10-24 18:30:38,229 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 3 nifi-bootstrap.log:2016-10-24 18:30:38,304 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 4 nifi-bootstrap.log:2016-10-24 18:30:38,380 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 5 nifi-bootstrap.log:2016-10-24 18:30:38,531 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 6 nifi-bootstrap.log:2016-10-24 18:30:38,607 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 7 nifi-bootstrap.log:2016-10-24 18:30:38,690 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 8 nifi-bootstrap.log:2016-10-24 18:30:38,769 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 9 nifi-bootstrap.log:2016-10-24 18:30:38,846 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 10 nifi-bootstrap.log:2016-10-24 18:30:38,925 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 11 nifi-bootstrap.log:2016-10-24 18:30:39,014 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 12 nifi-bootstrap.log:2016-10-24 18:30:39,090 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 13 nifi-bootstrap.log:2016-10-24 18:30:39,167 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 14 nifi-bootstrap.log:2016-10-24 18:30:39,242 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 15 nifi-bootstrap.log:2016-10-24 18:30:39,329 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 16 nifi-bootstrap.log:2016-10-24 18:30:39,395 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 17 nifi-bootstrap.log:2016-10-24 18:30:39,473 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 18 nifi-bootstrap.log:2016-10-24 18:30:39,551 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 19 nifi-bootstrap.log:2016-10-24 18:30:39,627 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 20 nifi-bootstrap.log:2016-10-24 18:30:39,701 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 21 nifi-bootstrap.log:2016-10-24 18:30:39,781 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 22 nifi-bootstrap.log:2016-10-24 18:30:39,856 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 23 nifi-bootstrap.log:2016-10-24 18:30:40,121 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 24 nifi-bootstrap.log:2016-10-24 18:30:40,196 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: streaming msg 25 nifi-bootstrap.log:2016-10-24 18:30:40,437 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: mail.pop3.apop.enable: false nifi-bootstrap.log:2016-10-24 18:30:40,437 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: mail.pop3.disablecapa: false nifi-bootstrap.log:2016-10-24 18:30:40,437 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: connecting to host "pop.agilemobiledeveloper.com", port 110, isSSL false nifi-bootstrap.log:2016-10-24 18:30:40,663 INFO [NiFi logging handler] org.apache.nifi.StdOut DEBUG POP3: PIPELINING enabled