- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
drop a column from csv
- Labels:
-
Apache Hadoop
-
Apache NiFi
Created on
02-03-2020
05:47 AM
- last edited on
02-03-2020
06:36 AM
by
cjervis
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
how can i drop column from csv
i only want to keep one selected feild
Created 02-07-2020 08:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Let's assume your input FlowFile content looks like this:
Date, IP, Description
01-30-2020, 10.0.0.1, server1
01-30-2020, 10.0.1.2, server2
01-30-2020, 10.0.3.4, server3
01-30-2020, 10.0.4.6, server4
01-30-2020, 10.0.10.2, server5
You would configure your ReplaceText processor as follows:
The Search Value contains a java regular expression which will match on your entire line and uses one capture group. The Replacement Value says to replace everything matched by the Search Value Regex with only the value from capture group 1. We then configure ReplaceText to evaluate that Regex against each line.
The resulting FlowFile output to the success relationship will have content that looks like this:
IP
10.0.0.1
10.0.1.2
10.0.3.4
10.0.4.6
10.0.10.2
Hope this helps,
Matt
Created 02-03-2020 06:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@nishank_paras Can you provide some additional details for those who may be able to assist you. Specifically, what components (Spark, NiFi, etc.) are you trying to do this with?
Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Created on 02-03-2020 06:24 AM - edited 02-03-2020 06:31 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i am new to this
i have a csv file with three column name, IP, description
i want to keep ip only and drop other
guide me to the best possible way
listfile ---> fetch file ----> ?? ----->putfile
also tell me the configuration settings if any
Technology - nifi
Created 02-03-2020 06:30 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
nifi
Created 02-03-2020 06:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for the additional information. I have updated the labels for your question so those with an interest in Nifi will be more likely to find it. While you are waiting for someone to answer. Allow me to direct you to something you may not have noticed.
On the right side of this page you should see a list of recommendations.
At the top for me is one that may be of interest with the subject of How to delete a row/drop a column in nifi.
There may be other links of assistance listed as well.
Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Created on 02-06-2020 01:44 AM - edited 02-06-2020 01:48 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to delete a row/drop a column in nifi
this is not helping as the configuration are not clear i'm new to this kindly guide if there is any better way
this solution is referencing to the nifi documentation
Created 02-06-2020 07:28 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Without an example input and desired output file, It is difficult to provide an exact solution.
However, you may want to consider using the ReplaceText processor in NiFi.
This processor can be configured with a java regular expression that uses java capture groups to parse your csv file line by line and then use the capture group to replace that lien with only the desired column value.
Just make sure you configure the processor to use "Line-by-Line" instead of the default "Entire text".
Hope this helps you get to a solution that works for you,
Matt
Created on 02-06-2020 09:15 PM - edited 02-07-2020 01:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
what regex should i use in replace text i have csv in given format
Date, IP, Description
and i want to keep IP only
Created 02-07-2020 08:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Let's assume your input FlowFile content looks like this:
Date, IP, Description
01-30-2020, 10.0.0.1, server1
01-30-2020, 10.0.1.2, server2
01-30-2020, 10.0.3.4, server3
01-30-2020, 10.0.4.6, server4
01-30-2020, 10.0.10.2, server5
You would configure your ReplaceText processor as follows:
The Search Value contains a java regular expression which will match on your entire line and uses one capture group. The Replacement Value says to replace everything matched by the Search Value Regex with only the value from capture group 1. We then configure ReplaceText to evaluate that Regex against each line.
The resulting FlowFile output to the success relationship will have content that looks like this:
IP
10.0.0.1
10.0.1.2
10.0.3.4
10.0.4.6
10.0.10.2
Hope this helps,
Matt
