Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

DEVPH Folder in Self paced learning VM

Solved Go to solution

DEVPH Folder in Self paced learning VM

Hi there:

I am an existing student of self paced learning. In one of the modules titled 'HDP Developer: Apache Pig and Hive Rev 6', I need to use a VM from the following link: https://s3.amazonaws.com/vm-us-east/data_analyst/HDP2.3-Pig-Hive-Rev6.zip./.

There is a folder titled DEVPH under /root. I can easily navigate to the folder using the command prompt, as shown below:

9403-p1.png

However I am struggling to navigate to DEVPH using the UI. I am trying to edit a file under DEVPH using GEDIT. However I can't see the folder under root, through the UI, as shown below:

9404-p1.png

Any help will be much appreciated.

Thanks and regards,

Anindya

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: DEVPH Folder in Self paced learning VM

@Anindya Chattopadhyay

You are right. Let me explain other way -

1. You have your local system which has ubuntu installed.

2. On Ubuntu you are running Sandbox [for which you ssh to Login]

3. Sandbox has also operating system [generally centos/ubuntu - depends upon which sandbox you downloaded].

4. And you have HDFS filesystem which is in Sandbox which is Hadoop Filesystem.

5. All hadoop config files are stored on Sandbox Local filesystem [ie. core-site.xml,hdfs-site.xml.etc..]

You can do ssh to sandbox and then access the hadoop config files using [vi or cat command].

But you cannot browse it from the UI as you shown in screenshot above, since the UI/browser which you are loading is of your Local Ubuntu machine and not of Sandbox.

if you need to do the same for Sandbox then you need to install GUI for Sandbox [As sandbox does not comes default with GUI]

Hope that makes clear now.

13 REPLIES 13

Re: DEVPH Folder in Self paced learning VM

Hi :

Any help with this one?

I just realised we can't really access the HDFS files the way we acces File System files. Normally we need to copy the file to the local File System, edit it and put it back to HDFS.

So I am not sure what is meant by the following steps in the Self paced learning lab:

9438-ss.png

Thanks and regards,

Anindya

Re: DEVPH Folder in Self paced learning VM

@Anindya Chattopadhyay

In the above image just put the value infront of "Location" as /root/devph/labs/Lab3.3

and enter. This will take you to the path.

Re: DEVPH Folder in Self paced learning VM

So it will look like -

Location: /root/devph/labs/Lab3.3

Re: DEVPH Folder in Self paced learning VM

@Sagar Shimpi, don't think so. That's how you access the local file system, not HDFS !! Look at the following screenshot.

9548-capture.png

Re: DEVPH Folder in Self paced learning VM

@Anindya Chattopadhyay

If this is HDFS path then you need to have NFS gateway in place. Check https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_hdfs_nfs_gateway/bk_hdfs_nfs_gateway-201...

From above logs the path seems to be on local path and not HDFS.

Re: DEVPH Folder in Self paced learning VM

@Sagar Shimpi, thanks for that PDF link. However I am just wondering how will I access files like core_site.xml, hdfs_site.xml etc.? Any help will be much appreciated.

Regards,

Anindya

Re: DEVPH Folder in Self paced learning VM

@Anindya Chattopadhyay core_site.xml, hdfs_site.xml etc are on local OS filesystem and you should be able to browse as per above screenshot you displayed.

Else go to terminal and do

$cat /etc/hadoop/conf/core-site.xml

You should have basic linux command knowledge.

Please accept the answer to close this thread if you feel initial question was answered. Will be happy to help you if any new question raised.

Re: DEVPH Folder in Self paced learning VM

@Sagar Shimpi, I am sort of across the linux commands (not an expert TBH). But I think I am understanding something really wrong here. My bad as I am very new to this HDP platform. My realization is core_site.xml, hdfs_site.xml etc. are not part of local file system. I can only see them after I run the SSH SANDBOX command through the terminal! See the following 2 screenshots: Screenshot 1 (without running the SSH SANDBOX command and I can't see any hadoop folder under etc)

9641-c1.png

Screenshot 2 (Running the SSH SANDBOX command and I can see hadoop folder under etc)

9642-c2.png

This makes me think those files are still part of HDFS and not local file system. So I still face the same situation?

Regards,

Anindya

Highlighted

Re: DEVPH Folder in Self paced learning VM

@Anindya Chattopadhyay

You are right. Let me explain other way -

1. You have your local system which has ubuntu installed.

2. On Ubuntu you are running Sandbox [for which you ssh to Login]

3. Sandbox has also operating system [generally centos/ubuntu - depends upon which sandbox you downloaded].

4. And you have HDFS filesystem which is in Sandbox which is Hadoop Filesystem.

5. All hadoop config files are stored on Sandbox Local filesystem [ie. core-site.xml,hdfs-site.xml.etc..]

You can do ssh to sandbox and then access the hadoop config files using [vi or cat command].

But you cannot browse it from the UI as you shown in screenshot above, since the UI/browser which you are loading is of your Local Ubuntu machine and not of Sandbox.

if you need to do the same for Sandbox then you need to install GUI for Sandbox [As sandbox does not comes default with GUI]

Hope that makes clear now.

Don't have an account?
Coming from Hortonworks? Activate your account here