Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oryx test run on hdfs file

Oryx test run on hdfs file

Explorer

Hi,

 

Am trying to run recommendation on a file in HDFS.

 

config file

**********

model=${als-model}
model.instance-dir=/Oryx_hdfs/Trade
serving-layer.api.port=8091
computation-layer.api.port=8092

 

file presents in  HDFS

 


[root@Namenode Oryx]# hadoop fs -ls /Oryx_hdfs/Trade
Found 1 items
drwx------   - root hdfs          0 2014-03-26 10:59 /Oryx_hdfs/Trade/00000
[root@Namenode Oryx]# hadoop fs -ls /Oryx_hdfs/Trade/00000
Found 1 items
drwx------   - root hdfs          0 2014-03-26 11:04 /Oryx_hdfs/Trade/00000/inbound
[root@Namenode Oryx]# hadoop fs -ls /Oryx_hdfs/Trade/00000/inbound
Found 1 items
-rwx------   3 root hdfs  319649757 2014-03-26 11:04 /Oryx_hdfs/Trade/00000/inbound/GroomedTradeDataForMahout.csv

 

but when i start the computation layer it fails with Port 8020, but port 8020 is running as well.

 

[root@Namenode Oryx]# netstat -nplt |grep 8020
tcp        0      0 10.128.2.101:8020           0.0.0.0:*                   LISTEN      20119/java
[root@Namenode Oryx]# ps -ef|grep 20119
hdfs     20119     1  1 11:11 ?        00:00:12 /usr/jdk/jdk1.6.0_31/bin/java -Dproc_namenode -Xmx1024m -Djava.net.preferIPv4Stack=true -

 

Error :

-*****

 

 

Mar 28, 2014 11:26:13 AM com.cloudera.oryx.computation.PeriodicRunner run
WARNING: Unexpected error in execution
java.io.IOException: Failed on local exception: java.io.IOException: Response is null.; Host Details : local host is: "Namenode/10.128.2.101"; destination host is: "Namenode":8020;
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:763)
        at org.apache.hadoop.ipc.Client.call(Client.java:1242)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)

 

 

Please help resolve this issue, am running the command in Namenode where i have Namenode and Jobtracker running.

 

Regards,

Suresh

3 REPLIES 3

Re: Oryx test run on hdfs file

Master Collaborator

The files look OK; I assume the CSV data is in the usual user,item,score format.

 

What are you referring to regarding port 8020? it's a port that Hadoop workers use but nothing in the app uses it.

 

Have you had a look at the Hadoop config files on the machine? are they in /etc/hadoop/conf, or a location pointed to by HADOOP_CONF_DIR?

 

Are you running an HA cluster? Host names like Namenode or "Namenode" otherwise look suspicous.

 

Highlighted

Re: Oryx test run on hdfs file

Explorer

Sean,

 

Appreciate your prompt responses as always, Yes, file is a CSV with user,item,score format..

 

8020  refers to hdfs service .

/etc/hadoop/conf.class is the location where i have all my hadoop config files.

[root@Namenode Oryx]# echo $HADOOP_CONF_DIR
/etc/hadoop/conf.class

Am running Hadoop 1.2.0 MRV1.

 

all am trying to do is read the input from hdfs file, for my example1(Collaborative Filtering).

 

Regards,

Suresh

Re: Oryx test run on hdfs file

Master Collaborator

Yes, I'm wondering what errors you are getting with regards to port 8020, because they are not related to the app. I think there is some issue with your cluster configuration, and that is a symptom then? I would look at your host names since "Namenode" doesn't sound right except possibly for HA. That is, i don't think there is an issue to do with the app here, not yet.