Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI - link 3 columns

Solved Go to solution

NIFI - link 3 columns

New Contributor

Hello, I need link 3 column in all rows of the file, for example:

column1;column2;column3;2018;09;27

column1_1;column2_1;column3_1;2018;09;27

column1_2;column2_2;column3_2;2018;09;27

the output file must contains only 4 columns:

column1;column2;column3;20180927

column1_1;column2_1;column3_1;20180927

column1_2;column2_2;column3_2;20180927

The input file is .txt and the output file should be .txt

Thaks!!


1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NIFI - link 3 columns

Super Guru

@Pepelu Rico

You can achieve this by using ReplaceText processor (or) UpdateRecord processor

Method1:Using UpdateRecord processor:

If you are working on larger number of rows then use UpdateRecord processor and define CSV Reader controller service with ; as value seperator.

Define CSV writer controller service add new property in the Processor as

/link3column as

concat(/col_4,/col5,/col6)

Refer to this link for configure/Usage of UpdateRecord processor.

Method2:Using ReplaceText processor:

Use this method if you are dealing with less number of records

92591-screen-shot-2018-09-27-at-112953-pm.png

Search Value

^((?:[^;]+;\s*){3})(.*)

Replacement Value

$1${'$2':replace(";","")}

Character Set

UTF-8

Maximum Buffer Size

1 MB //change this value according to your flowfile size

Replacement Strategy

Regex Replace

Evaluation Mode

Line-by-Line

In this method we are using regex to match until third ; to first capture group($1) and rest of the data into $2 capture group.

Making use of NiFi expression language we are replacing ";" in $2 capture group with ""

OutputFlowfile:

column1;column2;column3;20180927column1_1;column2_1;column3_1;20180927column1_2;column2_2;column3_2;20180927

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

2 REPLIES 2
Highlighted

Re: NIFI - link 3 columns

Super Guru

@Pepelu Rico

You can achieve this by using ReplaceText processor (or) UpdateRecord processor

Method1:Using UpdateRecord processor:

If you are working on larger number of rows then use UpdateRecord processor and define CSV Reader controller service with ; as value seperator.

Define CSV writer controller service add new property in the Processor as

/link3column as

concat(/col_4,/col5,/col6)

Refer to this link for configure/Usage of UpdateRecord processor.

Method2:Using ReplaceText processor:

Use this method if you are dealing with less number of records

92591-screen-shot-2018-09-27-at-112953-pm.png

Search Value

^((?:[^;]+;\s*){3})(.*)

Replacement Value

$1${'$2':replace(";","")}

Character Set

UTF-8

Maximum Buffer Size

1 MB //change this value according to your flowfile size

Replacement Strategy

Regex Replace

Evaluation Mode

Line-by-Line

In this method we are using regex to match until third ; to first capture group($1) and rest of the data into $2 capture group.

Making use of NiFi expression language we are replacing ";" in $2 capture group with ""

OutputFlowfile:

column1;column2;column3;20180927column1_1;column2_1;column3_1;20180927column1_2;column2_2;column3_2;20180927

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

Re: NIFI - link 3 columns

New Contributor

Thanks @Shu , It has been very helpful!!