About ischuldt24

ischuldt24 · ‎07-23-2021

I believe what we determined was that most browsers will typically trust a website if you call it using https, but if you try and make an https call from nifi, you need to have a trust store built in order to let nifi know you trust that site. I think alternatively you can set a property in the invokeHttp processor called trusted host name that will allow you to get around it.

ischuldt24 · ‎10-01-2020

So I can't see what specifically is wrong here, but I can tell you I've done the same thing using MergeContent. I use the bin packing algorithm with binary concatenation. Just another thought, you might be able to change the run schedule of the merge processor to only run once a minute or something. That might insure that all the successful flowfiles will have queued up behind the processor before it executes.

ischuldt24 · ‎10-01-2020

Have you tried using MergeContent instead of MergeRecord? I've noticed a lot of the processors dealing with records can be kind of picky about certain things. I believe merge content would accomplish what you're looking for and it might be easier to use. Also, do you have some kind of common identifier that you can use to tie the record back together? Maybe you could try posting a screen shot of your merge processor settings.

ischuldt24 · ‎10-01-2020

I am working on a highly secured network. We are required to white list any hosts that we plan on attempting to connect to. I have done so for three different end points that I am attempting to connect to via NiFi. Each of these end points work perfectly fine via a standard browser, but when I attempt the same call using InvokeHTTP in NiFi they fail. One is using an SSL connection. I've confirmed that my keystores and truststores work via another secured network to the same host via nifi. The other two have no real additional security on the other end that could be blocking them so I assume its an issue with this networks configurations. I'm trying to figure out if there are any potential settings in NiFi that could be changed that would make an GET HTTP call behave differently in NiFi than it would in a standard browser like Chrome or Firefox. Is it possible the call is getting directed through some unusual port? We are running NiFi on port 9090. Could that make a difference for any reason? I also attempted to run these connections in Postman and I'm getting similar behavior as I do in NiFi. When I execute the request in Postman, I get three extra headers in my return on this system that I don't get on my old network. They are as follows: X-Cache -> MISS from proxyint01 X-Cache-Lookup -> MISS from proxyint01:3128 via 1.1 proxyint01 (squid/3.5.20)

ischuldt24 · ‎06-15-2020

@stevenmatison I still haven't had any luck with this at all. Do you know if an error on one specific file could cause the entire process to crash and not complete or does it attempt to skip over bad files?

ischuldt24 · ‎06-01-2020

@hegdemahendra Hi heg, This appears to require some kind of AWS account. I'm not sure that is going to work for what I'm trying to do. Thanks

ischuldt24 · ‎06-01-2020

@stevenmatison Hi Steven, Thanks for the reply. I tried increasing the run schedule. I haven't seen any success yet. For the record, I'm using nifi 1.9.2. You were correct that the run schedule was set to zero. Another thing I've noticed is that I tend to get at least one file that errors out somewhere due to bad permissions or a bad filename. Could that cause the entire process to fail? If so how would I avoid that? I'm pulling the data off a removable hard drive that I have mapped a network drive to. The exact same set up works just fine on relatively small folders with 20 or so files in it. It's hard for me to post screenshots, but if you're interested in properties, I'm tracking timestamps, recursing subdirectories, I've tried an input directory of both local and remote. The file filter is wide open. Path filter is unchanged(although I tried it with the same regex as the file filter a couple of times), not including file attributes, I tried bumping the max directory listing time up to a couple of hours.

ischuldt24 · ‎05-27-2020

Hi, I'm trying to do a recursive list file on a massive directory. We're talking close to 100 GB. It seems like ListFile is walking through every single solitary file before it will spit out the first flow file. I was assuming it would spit out files one at a time as it finds them. It seems as though there's simply too much data. It just kind of sits there and spins. I left it run over the weekend and it never got through everything or spit anything out. I can't tell if its still working or if it's overloaded or what is going on. How would I go about walking through a directory like this returning one file at a time without duplicates?

ischuldt24 · ‎04-25-2019

Hi, Let's pretend that I'm being forced against my will to run Nifi on Windows. The registry isn't supported on Windows and I can't run a VM. Can I track certain Nifi files in a regular old Git repository and expect to be able to switch between branches with a simple checkout? If so which files do I need to track? And how do I refresh the Nifi work space to reflect those changes?

Online	Offline
Last Visited	‎10-01-2020 04:06 PM

Member Since	‎04-24-2019 01:34 PM
Last Visited	‎10-01-2020 04:06 PM
Posts	14

Cloudera Community

Re: HTTP request works from a browser, but not fro...

Re: NiFi merge records after filtering out some

Re: NiFi merge records after filtering out some

HTTP request works from a browser, but not from ni...

Re: ListFile on a huge directory

Re: ListFile on a huge directory

Re: ListFile on a huge directory

ListFile on a huge directory

Re: Storing Apache NiFi Versioned Flows in a Git R...