Member since
10-02-2017
14
Posts
0
Kudos Received
0
Solutions
09-04-2018
11:51 PM
Hi All- I appeared for hortonworks - HDPCD Spark exam.
In the whole exam, I was unable to launch the spark-shell and proctor was not able to provide any help with that. When i run the command "spark-shell" or even "pyspark"- it just gave an error - command not found.
Same thing happened when i tried to launch on AWS Spark-Practice instance as depicted on the website. I didnt even find sbt available.
If anyone has any insights on how to run this in AWS, please share your experience. Thanks, Sandeep 412-608-4461
... View more
Labels:
06-19-2018
12:34 AM
Could someone tell me the answer of below question, why and how? Q. How many partitions shall "intialiy" be created with the following command on spark shell-
sc.textfile("hdfs://user/cloudera/csvfiles")
There are 100 files in directory /user/cloudera/csvfiles and there are 10 nodes running Spark.
a.1
b.10
c.20
d.100
... View more
Labels:
03-08-2018
03:06 AM
hbase master.pid is not getting created, PID for other daemons is generated properly ERROR: no master to stop because no pid file /var/run/hbase/hbase-hbase-master.pid PID files for other hbase daemons:
hbase-hbase-regionserver.pid
hbase-hbase-rest.pid
hbase-hbase-thrift.pid
21391
21623
21955 When i run any command inside hbase shell it give error: Can't get master address from ZooKeeper; znode data == null However I am able to open zookeeper-client shell/prompt. Could you please tell me why hbase master pid file is not getting created? Can I manually create it and put the PID inside? If Yes, what is the PID for hbase-master? Thanks//Sandeep
... View more
Labels:
03-07-2018
07:09 PM
hbase master.pid is not getting created, PID for other daemons is generated properly ERROR: no master to stop because no pid file /var/run/hbase/hbase-hbase-master.pid PID files for other hbase daemons: hbase-hbase-regionserver.pid hbase-hbase-rest.pid hbase-hbase-thrift.pid 21391 21623 21955 When i run any command inside hbase shell it give error: Can't get master address from ZooKeeper; znode data == null However I am able to open zookeeper-client shell/prompt. Could you please tell me why hbase master pid file is not getting created? Can I manually create it and put the PID inside? If Yes, what is the PID for hbase-master? Thanks//Sandeep
... View more
12-22-2017
05:14 PM
@samurai I already tried these options, none of these work. when i do "su -" followed by password I get this su: cannot set groups: Operation not permitted ls -ltr /usr/bin/sudo -rwxrwxrwx. 1 root root 123832 /usr/bin/sudo 1. Not sure what is the "." at the end of permissions rwx. 2. I have to change the group of this sudo file to '0' but to do that also I should have sudo access, so its an endless loop. 3. I think there is no way to reset it while booting a cloudera machine either. Currently I am working on another VM but I will have to alter all the configurations settings again what I did so far, as when the error occurs.
... View more
12-21-2017
11:34 AM
Hi, @csgunathanks for your reply, this solution wont work. I have been using sudo for a long time but this occurred after a permission change using chmod 777 and now "sudo su" wont work, I am getting below error:- sudo: effective uid is not 0, is sudo installed setuid root?
... View more
12-10-2017
11:43 AM
Hi All- I did some chmod/chown changes on all and since then I am not able to login with root user. sudo command is not working. Tried all the options available online, still getting this error- "sudo: effective uid is not 0, is sudo installed setuid root?" I know that the uid for the file /usr/bin/sudo must be set to 0 but currently it is 1 and i am not able to change it. As soon as I launch the VM it opens with user "cloudera" by default. Is there any way to launch the VM with root user? Anyone know if this can be fixed while restarting the VM? Just want to fix this without re-installing the complete VM. Thanks, appreciate any help/insights on this.
... View more
Labels:
12-03-2017
10:50 AM
Hi Clover, Not sure if anybody found the answer to this. I am trying to upgrade to the latest version of Hive in cloudera. If anyone has any information on how to do this, please let me know. Would really appreciate it!
... View more
11-06-2017
02:03 AM
@Revathy Mourouguessane Not sure if you already solved it, I was facing the same issue today and finally was able to resolve it. You dont have to give .class extension in the argument, just give --class-name customerInfo This is the command I ran and its working fine, merging data from two directories into a new hdfs directory, keeping newer directory data as priority: sqoop merge --class-name customerInfo --jar-file customerInfo.jar --new-date /user/cloudera/new_dir/ --onto /user/cloudera/old_dir --merge-key id --target-dir /user/cloudera/new_emp_dir I kept the .jar and .class files in the bin directory of sqoop where i run commands, if you have it in some other paths - give the complete path and make sure it has the appropriate access. Hope this helps. I'd be interested and thrilled if we could help each other learn in future too!
... View more
10-22-2017
05:01 PM
Hey, thanks for the post. Adding these two services should fix the auxillary service exception issue. Dont forget to restart the yarn services. Thanks!
... View more
10-20-2017
05:20 PM
Hey buddy.........share across the solution you applied. Im facing the same issue and someone else too requested below. You will be helping 2 for sure, maybe more! Awaiting your reply, much appreciate your help!!!
... View more
10-07-2017
09:58 PM
I am trying to run a similar flume agent with HTTP source on the VM installed on my local computer. Not sure if the path and port is correct. When I ran my agent like this: ./flume-ng agent -n HttpAgent -c conf -f flume.conf it is stuck at the point http-source started. Not sure what's happening and not able to see the flume logs either! HttpAgent.sinks.hdfs-sink.hdfs.path = hdfs://quickstart.cloudera:8020/user/cloudera/output/http_flume Also if you can explain what will my output file be like when this works? Any suggestions?
... View more
10-07-2017
08:33 PM
@Deepesh @shilpa kumar I am trying to run a similar flume agent on the VM installed on my local computer, my conf file is also same as mentioned in your question Shilpa.
The only difference is, agent name=HttpAgent and the below property is different- HttpAgent.sinks.hdfs-sink.hdfs.path = hdfs://quickstart.cloudera:8020/user/cloudera/output/http_flume Also if you can explain what will my output file be like when this works? When I ran my agent like this: ./flume-ng agent -n HttpAgent -c conf -f flume.conf it is stuck at the point http-source started. Not sure what's happening and not able to see the flume logs either! Any suggestions? Could you help please.
... View more
10-02-2017
06:58 PM
Hi Penta, Did it work? Actually Im facing the same issue and this is what I have used: a1.sources.Twitter.consumerKey=XXX a1.sources.Twitter.consumerSecret=XXX a1.sources.Twitter.accessToken=XXX a1.sources.Twitter.a ccessTokenSecret=XXX I am trying to run the flume agent in cloudera VM. Please advice if you or anyone know the solution. Appreciate your suggestions/help!
... View more