Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

nifi-replace file headers using replacetext processor

avatar
New Contributor

i got a bunch of files with differert headers 

for files of one type,their may be two kinds of headers,i.e. the following two type(call it type1 and type 2) of files's headers

type1: a,b,c34,d     or     a,b,c,d

type2: d,e,f,kgg      or     d,e,f,g

notice that the third field names of type1 are c or c34,the forth field names of type2 are kgg or g,the field name c34 and kgg are wrong, and need to be changed to c and g

i want to change the headers only(that is, the first line) using  kv pairs, key is the string to be replaced and value is the  replacement value ,such as 

{"c34":"c"

"kgg":"g"

"hqq":"h"

....

}

i want to do this using  replacetext but the processor replacetext could only set one search and  replacement value,that's the problem!!

any ideas would be welcome

kami_0-1665557020574.png

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Perhaps JoltTransformRecord could help you.

 

The jolt transformation "shift" let's you rename fields - in a csv file's case, headers.

If you know all your replaces ahead of time, you could define a transformation like:

{

operation: shift

spec: {

    "kgg":"g",

    "c34":"c",

    "*":"&"

    }

}

 

Note the last shift, "*":"&" ,  which would transfer over the rest of the headers you didn't specifically rename. 

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

Perhaps JoltTransformRecord could help you.

 

The jolt transformation "shift" let's you rename fields - in a csv file's case, headers.

If you know all your replaces ahead of time, you could define a transformation like:

{

operation: shift

spec: {

    "kgg":"g",

    "c34":"c",

    "*":"&"

    }

}

 

Note the last shift, "*":"&" ,  which would transfer over the rest of the headers you didn't specifically rename. 

avatar
New Contributor

thanks ! that's a good idea
i'll have a try

avatar
New Contributor

thanks ,Green!

your idea helped me solve the problem