I have installed the HDP 2.6.1 virtual box installation which is a single node hadoop cluster. I am have been trying to run the a sample word count project for Spring Hadoop integration, taking code from the following url: https://github.com/spring-projects/spring-hadoop-samples. I get the following exception:
Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'runner': Invocation of init method failed; nested exception is java.lang.IllegalStateException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hadoop-yarn/staging/singhs2/.staging/job_1510100113370_0006/job.split could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1708) Can some one help me how to resolve this.
... View more
We have a Java (Struts-Hibernate-Spring) app using Oracle database to import data into oracle database by reading files and putting them in a staging table and then processing them from the staging table to permanent tables. This process takes a lot of time (8 hours for 70,000 records) to load it into the permanent tables due to ETL logic written in Java. Can I use hadoop to reduce this?
... View more