Cloudera Community
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
bhaskerbobby
user rank
Explorer
My Accepted Solutions
Show All

Re: how to copy file from mount point to hdfs (Web...

by Explorer in Support Questions
‎04-10-2019 05:20 PM
‎04-10-2019 05:20 PM
I figured out the issue was not with target directory, it was with source directory where the files were coming from , hence i fixed the permissions on those directories and now all got succeeded here is further commands i wrote to grand permissions (Please becarefull with 777 as it is not a good practice) [hdfs@sandbox-hdp ~]$ hdfs dfs -chmod 777 /user/hadoop [hdfs@sandbox-hdp ~]$ hdfs dfs -chmod 777 /user/hadoop/silverPop 0: jdbc:hive2://10.88.79.112:10000> LOAD DATA INPATH '/user/hadoop/silverPop/testme.csv' OVERWRITE INTO TABLE dwstg.dummy; successful ... View more

Re: how to copy file from mount point to hdfs (Web...

by Explorer in Support Questions
‎04-10-2019 05:02 PM
‎04-10-2019 05:02 PM
i figured the issues was not with target directory , it was source DIR where permissions were lacking, hence I fixed using below commands (Please note 777 is not a good practice , you will need adjust per your production scenario) [hdfs@sandbox-hdp ~]$ hdfs dfs -chmod 777 /user/hadoop [hdfs@sandbox-hdp ~]$ hdfs dfs -chmod 777 /user/hadoop/silverPop ... View more

Re: how to copy file from mount point to hdfs (Web...

by Explorer in Support Questions
‎04-10-2019 05:02 PM
‎04-10-2019 05:02 PM
@shuaili gong, Thanks for your reply, I went through below steps and finally managed upload the file in the HDFS directory so Hive can access it and this was sucessful. Now m having permission issues from source directory to /warehouse directory. I manually changed to 777 on /warehouse so everybody have full access and yet m still getting same permission issue. i have also checked via Ambari on this /warehouse directory and i can see it has full access to everyone but still same issue. Any suggestions highly appreciated Below are the steps i followed on how i uploaded file etc and gave permissions as well. [root@sandbox-hdp ~]# cp /mnt/tmp/testme.csv testme.csv [root@sandbox-hdp ~]# ls -ltr [root@sandbox-hdp ~]# su hdfs [hdfs@sandbox-hdp root]$ cd [hdfs@sandbox-hdp ~]$ hdfs dfs -chmod 777 /user [hdfs@sandbox-hdp ~]$ exit exit [root@sandbox-hdp ~]# [root@sandbox-hdp ~]# su hdfs [hdfs@sandbox-hdp root]$ cd [hdfs@sandbox-hdp ~]$ ls -ltr [hdfs@sandbox-hdp ~]$ exit exit [root@sandbox-hdp ~]# hdfs dfs -mkdir /user/hadoop/ [root@sandbox-hdp ~]# hdfs dfs -mkdir /user/hadoop/silverPop [root@sandbox-hdp ~]# hdfs dfs -put testme.csv /user/hadoop/silverPop [root@sandbox-hdp ~]# hdfs dfs -ls /user/hadoop/silverPop Found 1 items -rw-r--r-- 1 root hdfs 12967 2019-04-08 20:41 /user/hadoop/silverPop/testme.csv [root@sandbox-hdp ~]# hdfs dfs -du /user/hadoop/silverPop/testme.csv 12967 12967 /user/hadoop/testme.csv [root@sandbox-hdp ~]# hadoop fs -chmod g+w /warehouse 0: jdbc:hive2://10.88.79.112:10000> LOAD DATA INPATH '/user/hadoop/silverPop/testme.csv' OVERWRITE INTO TABLE dwstg.dummy; ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. org.apache.hadoop.hive.ql.metadata.HiveException: Access denied: Unable to move source hdfs://sandbox-hdp.hortonworks.com:8020/user/hadoop/silverPop/testme.csv to destination hdfs://sandbox-hdp.hortonworks.com:8020/warehouse/tablespace/managed/hive/dwstg.db/temp_silverpop/base_0000012: Permission denied: user=hive, access=WRITE, inode="/user/hadoop/silverPop":root:hdfs:drwxr-xr-x INFO : Completed executing command(queryId=hive_20190409152723_ee9c9ad2-9213-407d-8edc-16df7e13a2e0); Time taken: 0.112 seconds ... View more

how to copy file from mount point to hdfs (WebShel...

by Explorer in Support Questions
‎04-09-2019 01:04 AM
‎04-09-2019 01:04 AM
Hi everyone , This is the Sandbox env., M trying to load data into Hive table called: dwstg.dummy; here are the steps i followed inside WebShellClient Step1)after connecting to JDBC Hive , below is the command i issued Input) LOAD DATA INPATH '/mnt/tmp/testme.csv' OVERWRITE INTO TABLE dwstg.dummy; Output) I get following error Error: Error while compiling statement: FAILED: SemanticException Line 1:17 Invalid path ''/mnt/tmp/testme.csv'': No files matching path hdfs://sandbox-hdp.hortonworks.com:8020/mnt/tmp/testme.csv (state=42000,code=40000) Step2)based on the above error , i researched & realized JDBC is not seeing the mount point as it should exist only in HDFS directory for it to see? Step2.1)So i ran below command --> Input) hadoop fs -cp /mnt/tmp/testme.csv hdfs:/ Output: cp: `/mnt/tmp/testme.csv': No such file or directory So what m trying to figure out is , how to copy file from Mount point to hdfs directory structure? Regards: Bhasker.V Jsglp ... View more
Labels:

how to reset root password

by Explorer in Support Questions
‎04-03-2019 04:43 PM
‎04-03-2019 04:43 PM
Hi Everyone, M logging into my hadoop env. after a long time, infact last time when i have changed the password for root login on web-shell-client i stored it and system doesn't accept that changed password any more not sure why and what happened. Now all m trying to figure out is how & where to go , to be able reset my root password Regards: Bhasker.V Jsglp ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎05-13-2019 08:45 PM
Community Statistics
Member Since ‎04-02-2019 05:38 PM
Last Visited ‎05-13-2019 08:45 PM
Posts 5