I have scenario where I am getting the file(stream:flowfile of NIFI) as of type csv file, then creating the dataframe and dumping it thats it. But after creating the dataframe the structure of the file got disturbed, if I open the same flowfile on my disk i could see clear structure with columns separated with tab, but with python dataframe I am not getting the same structure, if I get same structure i can perform row manipulation.
Here What I am doing:
1: using ExecuteSQL processor, I am getting database record,
2: then passing this record to ConvertRecord processor to convert this avro record type csv file separated by tab.
convertRecordSetWriter settings...
![Capture.PNG Capture.PNG](https://community.cloudera.com/t5/image/serverpage/image-id/27275i6A2DA622BC10192B/image-size/large?v=v2&px=999)
The output of the flowfile is ...
![Capture.PNG Capture.PNG](https://community.cloudera.com/t5/image/serverpage/image-id/27276i5EB43E66B96B3607/image-size/large?v=v2&px=999)
3: Then the reading flowfile(step 2 folowfile) as python data using ExecuteStreamCommand, coz I am want to perform some action on the database record, to do this my record structure has been changed in data frame.
![Capture.PNG Capture.PNG](https://community.cloudera.com/t5/image/serverpage/image-id/27277i3CA6DE6C69BEE45C/image-size/large?v=v2&px=999)