Member since
02-02-2017
48
Posts
2
Kudos Received
0
Solutions
03-17-2017
12:19 PM
Hi dear, I agree with you but i do not get the result expected. On the picture flumeagent.png , in the first red rectangle , I have launch the cmd and the state for the port 44444 is 'LISTEN' When I created and started the agent a1, I noticed that the flume.conf file in the /conf/ directory of flume has not changed. Bit the directory a1 has been created contain a file flume.conf. Is this normal? I have used this second conf file (In a1 directory) to test the event. I opened a second terminal to telnet de port 44444 but i am still facing this problem of "connection refused" as you can see on the picture connectionrefused-44444-connection-refused.png What is wrong with my environment? What should i test to troubleshoot this problem? Thanks a lot for your response.
... View more
03-16-2017
11:44 AM
Hi , I tried the port 44444 as on the user guide but the connection is refused. I have just followed the flume user guide https://flume.apache.org/FlumeUserGuide.html#starting-an-agent I copy paste The flume.conf file in the flume service in ambari( as you can see on the picture flumeconf.png) I have 2 git bash terminal, one for starting the agent and see the log display buy the agent's sink. The other used to telnet localhost. I have already read another post facing the same problem and resolved it with the port 8050 or 8070. Me i am still facing problem with these both ports ( as you can see on the joined picture connectionrefused-44444-8050-8070.png) Thanks for your answer.
... View more
Labels:
- Labels:
-
Apache Flume
03-13-2017
11:05 AM
sqoophive2.png Hi, I would like to connect to Hiveserver2 of my sandbox. But I am facing a problem with the connection string. I have a java exception saying that there is an error when creating database manager as you can see on the joined picture. Can anyone help me with the rigth string? Thanks!
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Sqoop
03-13-2017
10:51 AM
Thanks @jay! I am able to list the tables using mysql database server and its databases and also tables in each data base.
... View more
03-01-2017
04:06 PM
Hi @Jay . Thanks for your response. You are right concerning hive so i remove the extra "//" as you see on the pic but I face a problem of "parsing arguments" Concerning the command "msql -u root -p, i always face the problem of access denied Thanks for your feedbacksqoopproblem1.png
... View more
03-01-2017
02:05 PM
Hi, I am learning sqoop using the sandbox and blocked on the first step concerning connection to the database server. I ssh the VM with git bash I have tested 2 cases : mysql and hive For the first case , I type the command "mysql -u root" and it returns "access denied" as you can see on the picture in the red rectangle For the second case, I tried to connect to Hive server - database "default". I used the jdbc url i have seen on ambari /Hive service as you can see on picture ambariHive Does anyone can help me troubleshooting this problem? Thanks ambarihive.pngsqoopproblem.png
... View more
- Tags:
- connection-refused
- MySQL
- Sandbox & Learning
- Sqoop
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
- Labels:
-
Apache Sqoop
02-28-2017
09:21 AM
Thanks a lot @mqureshi for this clear answer. I see that it is not possible to have this type of platform. I will prepare my certtification and find some uses cases on line . do you have some websites which can be helpful? Thanks.
... View more
02-27-2017
02:40 PM
1 Kudo
Excuse me but maybe my question is out of scope. I am looking for a platform where firms post their projects and data in order to have solutions. Something similar today is kaggle but I see that it is specialized in machine learning. I am not a machine learner because I am not at ease with Maths. I will like to apply what I have learned on this hadoop platform on a real entreprise project in order to mark it as a valuable experience on my CV. For the moment I am preparing HDPCD exam certification, but it cannot replace the entreprise experience. For the moment , I am not able to work in an entreprise for personnal reasons. Thanks for your feedback.
... View more
- Tags:
- hadoop
- Hadoop Core
Labels:
- Labels:
-
Apache Hadoop
02-23-2017
11:45 AM
I tried for the first time the command "spark-shell" on git bash in on order to learn how to manage data with spark and hive. I have faced a problem a memory as you can see on the picture.spark-shell.png I am on sandbox docker hdp 2.5 trying to learn with the "hello world" case. I have allocated 5GO of ram for my sandbox and my PC have 8GO of RAM in total. Until now , I have not faced a memory problem Is it compulsory to add RAM or is there a simple solution for this problem.
... View more
Labels:
- Labels:
-
Apache Spark
02-23-2017
10:57 AM
No problem ! i could just click on "vote" but unable to select the answer as the best one.how to do?
... View more
02-23-2017
10:44 AM
Thanks a lot @Jay! The problem resolved with a small notice of @Daniel Kozlowski concerning interpreter biding. The spark interpreter was "white" , I should have kept it "blue" Thanks a lot !!
... View more
02-23-2017
10:41 AM
Oh! @Daniel ! you are wright with this "blue" detail!!!! I though for the interpreter binding , spark should be "white"! Please see this fantastic screen! thanks a lot ! and keep helping newbies of newbies as me. !scversion3.png
... View more
02-23-2017
10:08 AM
Thanks @Daniel, I have already took @jay notice into account. I have no more paste the code since yesterday but type it by hand. What i have done now is create a new notebook file type %spark - press enter - type print(sc.version)- press enter , run it and I have a "prefix not found" display. I have tried to open the log file from /var/log/zeppelin/zeppelin-interpreter-sh-zeppelin-sandbox.hortonworks.com.log (as you can see on the picture)scversion2.pngscversion1.pnglogfile.png
I have also retstart spark interpreter again but still have the same "prefix not found" error.
... View more
02-23-2017
09:59 AM
Thanks @jay. I have already noticed my error.
... View more
02-23-2017
09:33 AM
Also facing the same problem with sc.version or print(sc.version)
scversion.png I have just install the sandbox on my laptop to do the "hello world" case. I can I know if it is kerberized?
... View more
02-23-2017
09:19 AM
localrepo.pngHi @Daniel The %sh whoami works well. It returns zeppelin. the problem I have is with %spark as said to @Bernhard. I don't know what is happening this morning but not able to see the directoty hdp as you can see. Yesterday I could 😞 The proble is resolved, The ssh port was not correct so please see the result of the command on the pic "localrepo"
hdpdirectorydisappear.png
... View more
02-23-2017
08:54 AM
Hi @Bernhard, Excuse me not to have respected what you said before concerning the fisrt line. the interpreter %jdbc(hive) and %sql are working well because the "show tables" display the result expected. The problem i have is with %spark(respecting the first line) and I have errors as you can see on the joined picture. In the interpreter binding, I have selected spark and save sparkproblem.png
sparkproblem1.png
sparkconfiguration.png I joined also the spark interpreter configuration
... View more
02-22-2017
02:59 PM
Hi @Bernhard, I have tried both but facing the same problem. Maybe jar files missing in my spark interpreter? sqlnotwork.pngsqlcontexttest.png
... View more
02-22-2017
02:47 PM
Hi @Jay, I checked but no extra line before "%spark" I manually write but still facing the problemhicecontextdisplay-2.png
... View more
02-22-2017
02:12 PM
Hi, I am facing a strange behaviour in sparK; I am following the tutotial (lab 4 -spark analysis) but when executing the command the script is returned as you can see on the picture(red rectangle) I have tables in My hive database default. I checked the jar files in my spark interpreter and found jar files as you can see on picture sparkjarfiles.png (Are some jar files missing please? Any suggestion?hivedisplay.pnghicecontextdisplay.png
... View more
- Tags:
- Data Science & Advanced Analytics
- Hive
- HIVECONTEXT
- Spark
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
- zeppelin
- zeppelin-notebook
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
-
Apache Zeppelin
02-22-2017
12:58 PM
I found the solution here ! really helpful link ! https://community.hortonworks.com/answers/62776/view.html Thanks!
... View more
02-22-2017
12:56 PM
really helpful @Ash! Thanks a lot !
... View more
02-22-2017
12:03 PM
jdbconfig.png Hi @Kshitij I have tried %jdbc(hive) jdbcinterpreter.pngbut have the same problem with this config jdbconfig.png Concerning my environment I am learning with Hadoop tutorial so my environment is a sanbox running HDP 2.5 I have intalled git bash to ssh the server. I sometimes use the web ssh through the port 4200 I am on the step :' lab 4 - spark risk analysis...' and face this problem of interpreter. I have not upddate jdbc config , line "hive url" , I verify in ambari and this url is the same as in the interpreter jdbc
... View more
02-22-2017
11:04 AM
Hi @Kshitij
I have tried %jdbc but have the same problem with this config jdbconfig.png Concerning my environment I am learning with Hadoop tutorial so my environment is a sanbox running HDP 2.5 I have intalled git bash to ssh the server. I sometimes use the web ssh through the port 4200 I am on the step :' lab 4 - spark risk analysis...' and face this problem of interpreter. I have not upddate jdbc config , line "hive url" , I verify in ambari and this url is the same as in the interpreter jdbc hiveserver2-url.png Thanks for your answer
... View more
02-21-2017
02:12 PM
Thanks, I have already install again and evrything is going well! I am now facing a problem with zeppelin connection on tutorial/lab5. I have already posted the question.
... View more
02-21-2017
02:06 PM
zeppelinenotebook1.png jdbconfig.png Hi, I am following tutorial- lab5 on spark and I am on the zeppelin step where i should create a notebook and execute the "show tables" from hive. I am facing a problem of interpreter not found. I check the JDBC interpreter which contains hive for me it is OK. I have just change the port il the "hive.url" parameter but still facing the same problem as shown on the picture could you please confirm me if the configuration In jdbc intepreter is correct? Thanks a lot !
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Zeppelin
02-21-2017
12:30 PM
Thanks Artem for your response. Few days before , i could do the same command as you can see the joined screen, using the same shell. How can I locate the docker please? Thanks.
beelineconnectiontohive2.png
... View more
02-21-2017
11:31 AM
I am facing a problem using beeline to acces the hiveserver2. i used git bash to ssh the sandbox in the virtuabox version hdp 2.5 When i type "sudo su hive", The shell display that the hive user does not exist. any suggestion? thanks. userhive.png
... View more
Labels:
- Labels:
-
Apache Hive