Hi @Ba  You can probably fetch this information from hdfs cli  rather than going to individual datanode disk.   you can run - hdfs fsck / -files -blocks -locations -->This will give you entire files blocks information with locations on which datanodes the blocks are stored. Just to segregatate wrt each datanodes you can apply some "grep" or "awk" filter   Hope this is what you were looking for. ... View more

@Ba  i didn't get you. can you please elaborate more. ... View more

Hi @HadoopHelp    For all questions then answer is -   If you are loading any file to hdfs then using below command you can check if the file is replicated across datanodes or not. The default replication factor is 3 so you can see 3 copied in below command command -   hdfs fsck /myfile.txt -files -blocks -location Eg. hosts is filename in my case hdfs fsck /tmp/hosts -files -blocks -locations /tmp/hosts 1157 bytes, 1 block(s): OK 0. BP-762887186-10.147.167.59-1521037753807:blk_1073748028_7830 len=1157 repl=4 [DatanodeInfoWithStorage[10.1.6.40:1019,DS-6cf46ebf-57fa-4d26-a0f8-f7b99f28424a,DISK], DatanodeInfoWithStorage[10.1.6.44:1019,DS-838d4d62-2069-4b73-b142-76ae1025ae6c,DISK], DatanodeInfoWithStorage[10.1.6.50:1019,DS-da75b9c5-5520-43f4-8e90-60d5982c714d,DISK], DatanodeInfoWithStorage[10.1.6.46:1019,DS-954af47c-1ba7-4057-aacd-1eae700d58cf,DISK]]   In above example you can see it shows blocks present on 3 datanodes ... View more

Hi @Hakim    To what i know you cannot use entire hdfs across 2 clusters. If you want you can use HDFS NFS gateway which will allow you to mount hdfs filesystem as mount  point on remote OS.   ... View more

Hi @Hakim  For bigdata technology you can adopt Hadoop platform for build distributed cluster. You can follow link below which will guide you to setup distributed cluster irrespective of number of nodes -   https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/installation.html   Note: You can test for trail subscription which has limited no of service features.   Also Cloudera Manager is Tool which allows you to manage this cluster. ... View more

Hi @mvsyomo1    Please check 2 things -   yarn.admin.acl property in Yarn configs [which is set to generally - dr.who,activity_analyzer,yarn] auto-to-local rules and proxyuser settings - In HDFS configs This will resolve the issue. ... View more

Hi @waelk    To create a case with cloudera you should opt for subscription/license. Probably you need to reach out to sales team at cloudera to explore more. Also pls do check below links -   https://www.cloudera.com/products/pricing.html https://www.cloudera.com/products/pricing/product-features.html   If you are currently running Hadoop for your own learning purpose then you can always post your queries on community and team will be helping you to fix the issues.   For now can you check the log messages for any errors displayed and can u post here. ... View more

Hi @Ba  Did you test ? Do you have any update?     ... View more

Hi @Ba As mentioned earlier,   If you are performing copying and remounting of Filesystem then during this activity namenode must be in safe mode and not serving any kind of operation.   If you add new mount to "dfs.name.data.dir", there there is no need to perform any steps only restart of hdfs will be required. No activity for datanodes. ... View more

Hi @Manoj690  You can clean the database and ambari will take care of this. You do not need to do it manually. ... View more

Hi @BaluD   If this is namenode mount point then make sure the data is backed properly before you move/delete the filesystem.   There will be no impact unless the mount point name is same. Just make sure while you do this activity the namenode was not functional to client. It should be stopped.   I see there is other option -   You can better add the new mount point in HDFS configs ->dfs.name.data.dir [as comma seperated value] Once you change the config and restart Namenode will start writing data to new mount. Once you see that all data is written to new mount comparing size with existing mount point then you can remove old mount from dfs.name.data.dir and restart hdfs   Make sure you do below first - 1. hdfs dfsadmin -safemode on 2. hdfs dfsadmin -saveNamespace ... View more

Hi @BaluD  Below will be the steps   1. Lets assume your exsiting data resides in /test [which is wrong mounted on "/"] 2. Create new mount point. eg /data01 [which is /dev/vdd1] 3. Mount the disk [which is wrong mount as off now] on the new mount created in steps1 ie. /data01 4. cp data from existing disk /test to /data01 5. Once copied, test data exist and then remove /test 6. unmount /data01 7. Create mount with /test 8. mount disk to new mount  - mount /dev/vdd1 on /test   Hope steps are clear now.   ... View more

Hi @Manoj690    You Database does not seems to be clean. It has already tables and schema existing.   Probably you might have already ran below command manually - /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql   Pls confirm.   If YES then you need to clean database and start/install via ambari again. ... View more

Hi @Ba The only way is to copy/move the data to new mount point and then rename. There is no short steps to faster the operation.   Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.   ... View more

Hi @Manoj690 Is this resolved ?   If so please accept the best answer. If you are still facing issue please let us know the error. We can check this out.   ... View more

Hi @BaluD    This is more of Unix question rathen than hadoop 🙂   Please try below command which will ignore all mounts and only give size details of filesystem/dirs which resides under "/"   for a in /*; do mountpoint -q -- "$a" || du -s -h -x "$a"; done  Let me know if that helps. ... View more

Hi @mike_bronson7    There is no generic/specific tool to monitor HDFS/KAFKA disk.Most client generally use the tool which is incorporated with OS vendor or either opt for thrid party tools. However there are multiple external tools available through which you can achieve this task -   Nagios / OpsView /HPOMI are popular options I've seen. In our environment KAFKA is extensively used and we have HPOMI and Prometheus installed for monitoring.   ... View more

Hi @Koffi    You can test this out. I tried this in past and it worked for me.   https://community.cloudera.com/t5/Support-Questions/How-to-Remove-all-External-Users-from-the-Ranger-Ranger/td-p/94987   Also do check https://issues.apache.org/jira/browse/RANGER-205   Hope that helps. ... View more