Reply
Highlighted
Contributor
Posts: 28
Registered: ‎03-04-2014

Oryx test run on hdfs file

Hi,

 

Am trying to run recommendation on a file in HDFS.

 

config file

**********

model=${als-model}
model.instance-dir=/Oryx_hdfs/Trade
serving-layer.api.port=8091
computation-layer.api.port=8092

 

file presents in  HDFS

 


[root@Namenode Oryx]# hadoop fs -ls /Oryx_hdfs/Trade
Found 1 items
drwx------   - root hdfs          0 2014-03-26 10:59 /Oryx_hdfs/Trade/00000
[root@Namenode Oryx]# hadoop fs -ls /Oryx_hdfs/Trade/00000
Found 1 items
drwx------   - root hdfs          0 2014-03-26 11:04 /Oryx_hdfs/Trade/00000/inbound
[root@Namenode Oryx]# hadoop fs -ls /Oryx_hdfs/Trade/00000/inbound
Found 1 items
-rwx------   3 root hdfs  319649757 2014-03-26 11:04 /Oryx_hdfs/Trade/00000/inbound/GroomedTradeDataForMahout.csv

 

but when i start the computation layer it fails with Port 8020, but port 8020 is running as well.

 

[root@Namenode Oryx]# netstat -nplt |grep 8020
tcp        0      0 10.128.2.101:8020           0.0.0.0:*                   LISTEN      20119/java
[root@Namenode Oryx]# ps -ef|grep 20119
hdfs     20119     1  1 11:11 ?        00:00:12 /usr/jdk/jdk1.6.0_31/bin/java -Dproc_namenode -Xmx1024m -Djava.net.preferIPv4Stack=true -

 

Error :

-*****

 

 

Mar 28, 2014 11:26:13 AM com.cloudera.oryx.computation.PeriodicRunner run
WARNING: Unexpected error in execution
java.io.IOException: Failed on local exception: java.io.IOException: Response is null.; Host Details : local host is: "Namenode/10.128.2.101"; destination host is: "Namenode":8020;
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:763)
        at org.apache.hadoop.ipc.Client.call(Client.java:1242)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)

 

 

Please help resolve this issue, am running the command in Namenode where i have Namenode and Jobtracker running.

 

Regards,

Suresh

Cloudera Employee
Posts: 366
Registered: ‎07-29-2013

Re: Oryx test run on hdfs file

The files look OK; I assume the CSV data is in the usual user,item,score format.

 

What are you referring to regarding port 8020? it's a port that Hadoop workers use but nothing in the app uses it.

 

Have you had a look at the Hadoop config files on the machine? are they in /etc/hadoop/conf, or a location pointed to by HADOOP_CONF_DIR?

 

Are you running an HA cluster? Host names like Namenode or "Namenode" otherwise look suspicous.

 

Contributor
Posts: 28
Registered: ‎03-04-2014

Re: Oryx test run on hdfs file

Sean,

 

Appreciate your prompt responses as always, Yes, file is a CSV with user,item,score format..

 

8020  refers to hdfs service .

/etc/hadoop/conf.class is the location where i have all my hadoop config files.

[root@Namenode Oryx]# echo $HADOOP_CONF_DIR
/etc/hadoop/conf.class

Am running Hadoop 1.2.0 MRV1.

 

all am trying to do is read the input from hdfs file, for my example1(Collaborative Filtering).

 

Regards,

Suresh

Cloudera Employee
Posts: 366
Registered: ‎07-29-2013

Re: Oryx test run on hdfs file

Yes, I'm wondering what errors you are getting with regards to port 8020, because they are not related to the app. I think there is some issue with your cluster configuration, and that is a symptom then? I would look at your host names since "Namenode" doesn't sound right except possibly for HA. That is, i don't think there is an issue to do with the app here, not yet.