Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

drop a column from csv

Solved Go to solution
Highlighted

drop a column from csv

how can i drop column from csv 

i only want to keep one selected feild

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: drop a column from csv

Master Guru

@nishank_paras 

 

Let's assume your input FlowFile content looks like this:

Date, IP, Description
01-30-2020, 10.0.0.1, server1
01-30-2020, 10.0.1.2, server2
01-30-2020, 10.0.3.4, server3
01-30-2020, 10.0.4.6, server4
01-30-2020, 10.0.10.2, server5

 

You would configure your ReplaceText processor as follows:
Screen Shot 2020-02-07 at 11.19.05 AM.png

The Search Value contains a java regular expression which will match on your entire line and uses one capture group.   The Replacement Value says to replace everything matched by the Search Value Regex with only the value from capture group 1.  We then configure ReplaceText to evaluate that Regex against each line.

The resulting FlowFile output to the success relationship will have content that looks like this:

IP
10.0.0.1
10.0.1.2
10.0.3.4
10.0.4.6
10.0.10.2


Hope this helps,

Matt

View solution in original post

8 REPLIES 8
Highlighted

Re: drop a column from csv

Community Manager

@nishank_paras Can you provide some additional details for those who may be able to assist you. Specifically, what components (Spark, NiFi, etc.) are you trying to do this with?

 

 



Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum
Highlighted

Re: drop a column from csv

i am new to this 

i have a csv file with three column name, IP, description 

i want to keep ip only and drop other

guide me to the best possible way

listfile ---> fetch file ----> ?? ----->putfile

also tell me the configuration settings if any

 

Technology - nifi

Re: drop a column from csv

nifi

Highlighted

Re: drop a column from csv

Community Manager

Thank you for the additional information. I have updated the labels for your question so those with an interest in Nifi will be more likely to find it. While you are waiting for someone to answer. Allow me to direct you to something you may not have noticed. 

 

On the right side of this page you should see a list of recommendations.

Screen Shot 2020-02-03 at 9.38.49 AM.png

At the top for me is one that may be of interest with the subject of How to delete a row/drop a column in nifi.

There may be other links of assistance listed as well. 



Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum
Highlighted

Re: drop a column from csv

How to delete a row/drop a column in nifi

this is not helping as the configuration are not clear i'm new to this kindly guide if there is any better way

this solution is referencing to the nifi documentation

 

Highlighted

Re: drop a column from csv

Master Guru

@nishank_paras 

 

Without an example input and desired output file, It is difficult to provide an exact solution.
However, you may want to consider using the ReplaceText processor in NiFi.

 

This processor can be configured with a java regular expression that uses java capture groups to parse your csv file line by line and then use the capture group to replace that lien with only the desired column value.

Just make sure you configure the processor to use "Line-by-Line" instead of the default "Entire text".

Hope this helps you get to a solution that works for you,

Matt

Highlighted

Re: drop a column from csv

what regex should i use in replace text i have csv in given format

Date, IP, Description

 

and i want to keep IP only

@MattWho 

Highlighted

Re: drop a column from csv

Master Guru

@nishank_paras 

 

Let's assume your input FlowFile content looks like this:

Date, IP, Description
01-30-2020, 10.0.0.1, server1
01-30-2020, 10.0.1.2, server2
01-30-2020, 10.0.3.4, server3
01-30-2020, 10.0.4.6, server4
01-30-2020, 10.0.10.2, server5

 

You would configure your ReplaceText processor as follows:
Screen Shot 2020-02-07 at 11.19.05 AM.png

The Search Value contains a java regular expression which will match on your entire line and uses one capture group.   The Replacement Value says to replace everything matched by the Search Value Regex with only the value from capture group 1.  We then configure ReplaceText to evaluate that Regex against each line.

The resulting FlowFile output to the success relationship will have content that looks like this:

IP
10.0.0.1
10.0.1.2
10.0.3.4
10.0.4.6
10.0.10.2


Hope this helps,

Matt

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here