@Chen Yimu Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted! ... View more

I am not sure where you are getting this issue, but the number looks too long for an integer! If this is your hive table etc, try using BigInt and not int. Or else post some more info about the issue 🙂 ... View more

@Christian Lunesa Is this related to the earlier issue? ... View more

@Vincent van Oudenhoven Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted! ... View more

@swathi thukkaraju Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted! ... View more

@Aishwarya Sudhakar You need to understand the HDFS directory structure. This is the one which is causing issues to you. Follows some explanation. Let's say the username for these example commands is ash. So when ash tries to create a directory in HDFS with the following command /user/ashhadoop fs -mkdir demo //This creates a directory inside HDFS directory //The complete directory path shall be /user/ash/demo it is different than the command given below hadoop fs -mkdir /demo //This creates a directory in the root directory. //The complete directory path shall be /demo So a suggestion here is, whenever you try to access the directories, use the absolute path(s) to avoid the confusion. So in this case, when you create a directory using hadoop fs -mkdir demo and loads the file to HDFS using hadoop fs -copyFromLocal dataset.csv demo You file exists at /user/ash/demo/dataset.csv //Not at /demo So your reference to your spark code for this file should be sc.textFile("hdfs://user/ash/demo/dataset.csv") Hope this helps! ... View more

@Aishwarya Sudhakar You need to understand that the directory structure you are mentioning and trying to access is different. //This means there is a directory named demo on root and has a file named dataset.csv /demo/dataset.csv //This means there is a directory named demo in the user directory of user and has a file named dataset.csv demo/dataset.csv Now, you do try the following on your terminal to get the username. whoami Now use the output of this command to reach to your dataset.csv file. You will realize that hadoop fs -cat demo/dataset.csv is similar to hadoop fs -cat /user/<your username>/demo/dataset.csv You can evaluate that using the ls command on these directories. hadoop fs -ls demohadoop fs -ls /user/ash/demo Now to access these file(s), use the correct directory reference. scala> val data = MLUtils.loadLibSVMFile(sc, "demo/dataset.csv") Let know if that helps! ... View more

@Félicien Catherin Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted! ... View more

@Krishna R Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted! ... View more

@subbiram Padala Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted! ... View more