Member since
08-20-2017
6
Posts
0
Kudos Received
0
Solutions
12-12-2017
04:32 PM
@gnovak Thanks for quick response.my hdfs uri looks like "hdfs://demo" I'm getting below exception java.net.UnknownHostException: demo java.lang.IllegalArgumentException: java.net.UnknownHostException: demo
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:438)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:688)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:629)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2761)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2795)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2777)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:386)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:179)
at com.mckinsey.hcva.aaas.execute.ReportsGeneration$1.run(ReportsGeneration.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: demo
... View more
12-12-2017
09:47 AM
Currently I'm doing it using hdfs dfs -cp -f file:"/tmp/files" hdfs:/user/username in script. I want to do it in java now
... View more
12-12-2017
09:40 AM
I'm trying to move files from one location to another location while executing java application on execution server Example:From one location(/tmp/files folder) to another location(hdfs://user/username directory) using java application.
... View more
Labels:
- Labels:
-
Apache Hadoop
08-23-2017
03:33 AM
@Geoffrey Shelton Okot Perfect!.It worked for me, I used WinSCP to upload the jar to tmp folder.And using -CopyFromLocal command moved the jar from tmp to the maria_dev user and able execute the jar. Thanks a lot for quick response
... View more
08-22-2017
03:21 AM
@Geoffrey Shelton Okot Thanks for quick response I have jar in my local directory at F:/dev/jars/sparktest.jar and I would like to move to maria_dev directory which is present in hdfs. $hdfs dfs F:/dev/jars sparktest.jar /user/maria_dev I tried the above command it is not working
... View more
08-20-2017
07:17 AM
I have created a simple java+spark project to read and perform a calculation on JavaRDD. I have setup HDP-Oracle VM on my machine.My question is where to upload .jar(ex: Aambari file view) and execute the jar
... View more
Labels:
- Labels:
-
Apache Spark