- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
What is wrong in reading Nifi flowfile and creating python(pandas)dataframe?
- Labels:
-
Apache NiFi
Created 04-28-2020 07:47 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have scenario where I am getting the file(stream:flowfile of NIFI) as of type csv file, then creating the dataframe and dumping it thats it. But after creating the dataframe the structure of the file got disturbed, if I open the same flowfile on my disk i could see clear structure with columns separated with tab, but with python dataframe I am not getting the same structure, if I get same structure i can perform row manipulation.
Here What I am doing: 1: using ExecuteSQL processor, I am getting database record, 2: then passing this record to ConvertRecord processor to convert this avro record type csv file separated by tab. convertRecordSetWriter settings...
The output of the flowfile is ...
3: Then the reading flowfile(step 2 folowfile) as python data using ExecuteStreamCommand, coz I am want to perform some action on the database record, to do this my record structure has been changed in data frame.
Created 04-28-2020 11:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
it did work after adding '\t' to read_csv as 2nd arg.
Created 04-28-2020 11:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
it did work after adding '\t' to read_csv as 2nd arg.