About saikrishna_tara

saikrishna_tara · ‎08-31-2016

Hi ,Can we flatten complex JSON files (with lists,arrays etc)with NIFI.?

saikrishna_tara · ‎08-30-2016

@mclark @Bryan Bende I tried the same thing , I had the List-->Fetch-->Output port inside a process group on remote machine. My local NiFi was not able to find that port , But when I removed it from the process group and copied on to main canvas then my local NiFi was able to find it. Thank you both.

saikrishna_tara · ‎08-30-2016

I have my NIFI running on a remote machine which i access from my local machine using http://remoteurl:9090/nifi i could fetch files from HDFS using List\FetchHDFS and if i want to PutFiles on to my local machine , it will not work as NiFi is running on remote machine and it wont have access to my local folders. i think we can achive this by running another NiFi instance locally on my machine and thru Remote process groups. but our firewall is only open oneway from local to remote. i could call a remote process on remote machine from my local nifi , but i cannot call a remote process on local machine from remote nifi. any ideas on how to solve?

saikrishna_tara · ‎08-29-2016

@mclark, Ok , but RouteOnContent checks for the string in the whole file. where as i want to compare only the firstline . if i have my RouteOnContent like below..it would route files to "Header" even if the data satisfies the RegEx.

saikrishna_tara · ‎08-29-2016

Hi, I need to merge contents based on .CSV file headers. Lets say if i have 10 files in a folder and 5 of them with same header Name,Age,Gender.I want to merge all those 5 together and send rest to failures. How can i do that.?

saikrishna_tara · ‎08-25-2016

@Bryan Bende , RouteOnAttribute will know the value of header.? Don't i have to read the value of first line to know the header.? if i have to read first line then should i use SplitText which i read creates a flow file for each line which i want to avoid as i am trying to move thousands of files . below is a simple flow ..i created 5 files , 4 with header foo and one with a different header. i want to merge all the 4 files with foo as header. tried the approach suggested by you above , but its routing everything to unmatched.

saikrishna_tara · ‎08-25-2016

@Bryan Bende , How do I send files with same header (hardcoded??) to MergeContent process and files that do not match the header I am planning to send those to failures. can this be done .?

saikrishna_tara · ‎08-24-2016

@mpayne, is there a way i can check for headers from CVS files and merge all files with similar headers and route other files to failures.??

saikrishna_tara · ‎08-23-2016

Its working now after I changed it to localhost, i think as you mentioned it was resolving to server’s IP address earlier and now with “localhost” as the hostname , the Tunnel is routing it to the 9091 port.

saikrishna_tara · ‎08-19-2016

Hi Bryan, here are my config files and settings.. local instance of nifi # Site to Site properties nifi.remote.input.socket.host= nifi.remote.input.socket.port= nifi.remote.input.secure=false --Remote instance of nifi nifi.remote.input.socket.host= nifi.remote.input.socket.port=9091 nifi.remote.input.secure=false SSH Tunnel on local server (where local instance of NiFi is running) and here is how my local dataflow looks.. remote

Online	Offline
Last Visited	‎10-26-2018 04:07 PM

Member Since	‎07-08-2016 07:25 PM
Last Visited	‎10-26-2018 04:07 PM
Posts	260
Kudos received	44

Cloudera Community

Re: I am using unpack processor in NIFI to unzip a...

Re: Changing NiFi Templates

Re: Hive Connection Pool Kerberos Issue

Re: NiFi user getting Tez exception

Re: Help with Hive Regex extract.

Can we flatten complex JSON file using NIFI.?

Re: How can we fetch files from a HDFS to local ma...

How can we fetch files from a HDFS to local machin...

Re: Merge files Based on file headers.?

Merge files Based on file headers.?

Re: How to Merge files together by file attribute ...

Re: How to Merge files together by file attribute ...

Re: How to MergeContent with single header from mu...

Re: Error accessing NIFI remote port thru SSH tunn...

Re: Error accessing NIFI remote port thru SSH tunn...