- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Read SAS files into parquet using nifi
Created 02-25-2025 10:48 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We are using Cloudera Private Cloud, and our customer wants to process SAS files using NiFi. However, I couldn't find a built-in NiFi processor that supports this.
Is there a known solution or workaround for handling SAS files in NiFi? Any suggestions would be greatly appreciated.
Thanks
Created 02-26-2025 05:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@dsender
Apache NiFi is a data agnostic service. It can move any data format through a dataflow because the content is treated as just bytes inside a FlowFile. The only time the content needs to be read is if there is need to manipulate it, extract from it, etc. Then you would need to use a processor that understand the data format. While it does not appear that Cloudera Flow Management offers any SAS specific processor components.
So some custom processor would need to be developed or perhaps you can use one of the available scripting processors? You would still need to write a custom script to ingest and/or process the SAS files. So this starts with the question of how would you pull these SAS files from command line outside of using NiFi? Then figure out how to turn that success into a custom script or processor that does the same thing.
You could also reach out to your Cloudera Account owner and discuss possible professional service offering that maybe able to help you here with your custom needs.
Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 02-26-2025 05:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@dsender
Apache NiFi is a data agnostic service. It can move any data format through a dataflow because the content is treated as just bytes inside a FlowFile. The only time the content needs to be read is if there is need to manipulate it, extract from it, etc. Then you would need to use a processor that understand the data format. While it does not appear that Cloudera Flow Management offers any SAS specific processor components.
So some custom processor would need to be developed or perhaps you can use one of the available scripting processors? You would still need to write a custom script to ingest and/or process the SAS files. So this starts with the question of how would you pull these SAS files from command line outside of using NiFi? Then figure out how to turn that success into a custom script or processor that does the same thing.
You could also reach out to your Cloudera Account owner and discuss possible professional service offering that maybe able to help you here with your custom needs.
Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
