Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Local Data combined with HDFS

Solved Go to solution

Local Data combined with HDFS

New Contributor

Hi All,

 

Just looking through the CDSW documentation and have found the following : If you want to create a new project around one or more data files on your computer, select the Localoption when creating the project.

 

We're looking at creating projects that combine Local data, data from HDFS, etc. Is this possible? Or can you only use local files in a project that's marked as 'Local'?

 

Thanks a lot.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Local Data combined with HDFS

Master Collaborator

Really, this is just saying you can upload data at project creation time or later from your local computer to the local file system that the Python/R/Scala sessions see in their local file system. 


Those jobs then see those local files as simple files, and can do what they like with them.


But you can also within the same program access whatever data you want, anywhere you want; you just need to write code that does so. Via Spark or whatever library you want you can also access whatever data sources you want, as well.


There is no either/or here.

2 REPLIES 2
Highlighted

Re: Local Data combined with HDFS

Master Collaborator

Really, this is just saying you can upload data at project creation time or later from your local computer to the local file system that the Python/R/Scala sessions see in their local file system. 


Those jobs then see those local files as simple files, and can do what they like with them.


But you can also within the same program access whatever data you want, anywhere you want; you just need to write code that does so. Via Spark or whatever library you want you can also access whatever data sources you want, as well.


There is no either/or here.

Re: Local Data combined with HDFS

New Contributor

Hi, thanks for the information! This is exactly what I expected but just wanted to make sure.

Don't have an account?
Coming from Hortonworks? Activate your account here