Member since
06-26-2015
515
Posts
138
Kudos Received
114
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2269 | 09-20-2022 03:33 PM | |
| 6036 | 09-19-2022 04:47 PM | |
| 3258 | 09-11-2022 05:01 PM | |
| 3727 | 09-06-2022 02:23 PM | |
| 5800 | 09-06-2022 04:30 AM |
03-07-2022
12:47 PM
@bmoisson , I believe the best alternative for you would be to use a fixed schema rather than "Infer Schema". Create a parameter with the schema that specifies the exact structure and data types that you want to use and configure your RecordReader setting that parameter in the "Schema Text" property of the RecordReader and setting the Schema Strategy to "Use Schema Text". Cheers, André -- Did this response answer your question? If so, please take a moment to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-06-2022
11:41 PM
Hi @Olivia_4692 , The only issue I can see here is that in your sample data "Before" has a capital B, while you your EvaluateJSONPath configuration it's all in lowercase. This will certainly create a problem for your logic. I tried your data (converting "Before" to "before") and it worked correctly in my test. Please my test flow attached. Cheers, André -- Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-06-2022
08:49 PM
@Olivia_4692 , Would you have a sample of the input data? André
... View more
03-06-2022
02:33 PM
@Olivia_4692 , Your approach is correct and it should work as you expect. There are a few things that I'd like to check, though: 1. In the JSON Path expression "a_before = $.['a']['before'][]['ip']", what's the reason for the empty "[]" just before "['ip']"? Could you please share a sample of your data? 2. Have you checked for leading or trailing spaces in either one of the attributes? 3. This is probably a copy&paste mistake, but I noticed that your RouteOnAttribute expression is missing a "}" at the very end. Cheers, Andre -- Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-06-2022
02:18 PM
@OmarElSeihy , Note that the error message says that the filesystem that's low on space is /tmp. The df result that you listed in your post is for the root filesystem (/). Please run a "df -h /tmp" and check that filesystem for content that could be cleaned. Cheers, André -- Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-03-2022
04:27 PM
@Jarinek , This is total volume, in bytes, received from the network by all the brokers across the cluster. Replication traffic is counted by this counter as well. Cheers, André -- Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-03-2022
04:19 PM
Hi, @templarian , From your log I can see that thread 13 got stuck right after this line: 2022-03-02 16:31:37,479 DEBUG [Timer-Driven Process Thread 13 ] jakarta.mail getProvider() returning jakarta.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle] Looking at other threads, the line that should've come right after that would be one like this: com.sun.mail.smtp connected to host "smtp.office365.com", port: 587 It seems to me that the thread got stuck during the attempt to connect to the SMTP server. I was able to reproduce this locally and confirmed that after the socked connection is established, if the server response is lost for some reason (or if the server doesn't respond at all) the processor thread gets stuck forever. Apparently this is kind of by design, since the default value of the mail.smtp.timeout property is infinite, as you can see here. I had a look at the PutEmail processor source code and unfortunately I don't think there is a way currently to set this in the processor configuration. I'd suggest that you open a NiFi Jira in https://issues.apache.org/ asking for the configurable timeout to be added. If you have support with Cloudera, please open a ticket and we should be able to help with that. Cheers, André -- Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-03-2022
02:36 PM
1 Kudo
@pandu2022 , When using Kerberos and/or TLS, please make sure that the hostname is specified as a fully qualified name (e.g. hostname.acm.com), instead of a short name. Are you using a fully qualified name? If not, could you please try again using one? Also, are you using a load balancer? Regards, André
... View more
03-03-2022
02:30 PM
Ah! Thanks a lot for taking the time to post the solution, @spserd ! I suspect that there was one detail that wasn't mentioned: did you also upgrade Java from 8 to 11 between the two clusters? In Java 8 the default keystore/truststore type used to be JKS and in Java 11 the default changed to PKCS12. If the store type isn't set explicitly, this error would certainly happen when upgrading Java. Cheers, André
... View more
03-03-2022
01:39 AM
@BIDEV3421 This property is for the opposite of what you want. The value of this property is what hive shows when the source day has a null value. I don't think there's a property to do what you want, but you can use an expression like the following in your select: case when mycol = ' ' then NULL else mycol end Cheers Andre
... View more