Support Questions

Find answers, ask questions, and share your expertise

Read SAS files into parquet using nifi

avatar
New Contributor

Hi,

We are using Cloudera Private Cloud, and our customer wants to process SAS files using NiFi. However, I couldn't find a built-in NiFi processor that supports this.

Is there a known solution or workaround for handling SAS files in NiFi? Any suggestions would be greatly appreciated.

Thanks

1 ACCEPTED SOLUTION

avatar
Master Mentor

@dsender 

Apache NiFi is a data agnostic service.  It can move any data format through a dataflow because the content is treated as just bytes  inside a FlowFile.  The only time the content needs to be read is if there is need to manipulate it, extract from it, etc.   Then you would need to use a processor that understand the data format.  While it does not appear that Cloudera Flow Management offers any SAS specific processor components. 

So some custom processor would need to be developed or perhaps you can use one of the available scripting processors?  You would still need to write a custom script to ingest and/or process the SAS files.  So this starts with the question of how would you pull these SAS files from command line outside of using NiFi?  Then figure out how to turn that success into a custom script or processor that does the same thing.

You could also reach out to your Cloudera Account owner and discuss possible professional service offering that maybe able to help you here with your custom needs.

Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View solution in original post

1 REPLY 1

avatar
Master Mentor

@dsender 

Apache NiFi is a data agnostic service.  It can move any data format through a dataflow because the content is treated as just bytes  inside a FlowFile.  The only time the content needs to be read is if there is need to manipulate it, extract from it, etc.   Then you would need to use a processor that understand the data format.  While it does not appear that Cloudera Flow Management offers any SAS specific processor components. 

So some custom processor would need to be developed or perhaps you can use one of the available scripting processors?  You would still need to write a custom script to ingest and/or process the SAS files.  So this starts with the question of how would you pull these SAS files from command line outside of using NiFi?  Then figure out how to turn that success into a custom script or processor that does the same thing.

You could also reach out to your Cloudera Account owner and discuss possible professional service offering that maybe able to help you here with your custom needs.

Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt