Support Questions
Find answers, ask questions, and share your expertise

What is wrong in reading Nifi flowfile and creating python(pandas)dataframe?

Solved Go to solution

What is wrong in reading Nifi flowfile and creating python(pandas)dataframe?

Rising Star

I have scenario where I am getting the file(stream:flowfile of NIFI) as of type csv file, then creating the dataframe and dumping it thats it. But after creating the dataframe the structure of the file got disturbed, if I open the same flowfile on my disk i could see clear structure with columns separated with tab, but with python dataframe I am not getting the same structure, if I get same structure i can perform row manipulation.

 

Here What I am doing:
1: using ExecuteSQL processor, I am getting database record,

2: then passing this record to ConvertRecord processor  to convert this avro record type csv file separated by tab.
convertRecordSetWriter settings...
Capture.PNG

The output of the flowfile is ...

Capture.PNG


3: Then the reading flowfile(step 2 folowfile) as python data using ExecuteStreamCommand, coz I am want to perform some action on the database record, to do this my record structure has been changed in data frame.
Capture.PNG

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: What is wrong in reading Nifi flowfile and creating python(pandas)dataframe?

Rising Star

it did work after adding '\t' to read_csv as 2nd arg.

View solution in original post

1 REPLY 1
Highlighted

Re: What is wrong in reading Nifi flowfile and creating python(pandas)dataframe?

Rising Star

it did work after adding '\t' to read_csv as 2nd arg.

View solution in original post