Cloudera Community
azeltov
user rank
Guru

Re: Spark SQL - Update Command

by Guru in Support Questions
‎03-21-2016 05:21 PM
2 Kudos
‎03-21-2016 05:21 PM
2 Kudos
@Sridhar Babu M it depends on the datasource you are updating, not all sources can be updated. What is the backend ur using for the Dataframe People? For example for hive: it’s possible to update data in Hive using ORC format https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-sql-hive-orc-example.html ... View more

Re: Apache Zeppelin and SparkR

by Guru in Community Articles
‎03-18-2016 07:04 PM
1 Kudo
‎03-18-2016 07:04 PM
1 Kudo
@Neeraj Sabharwal was able to get passed the dependencies issue with R 3.2.3 by explicitly specifigying the repo: repos="http://xyz.xxx.abc.edu") install.packages("evaluate", repos="http://xyz.xxx.abc.edu") ... View more

Re: Apache Zeppelin and SparkR

by Guru in Community Articles
‎03-17-2016 05:55 PM
1 Kudo
‎03-17-2016 05:55 PM
1 Kudo
@Neeraj Sabharwal just tried doing the sparkR zeppelin install on HDP2.4, but had problems with dependencies in R (package ‘evaluate’ is not available (for R version 3.2.3)) Which causes the maven build to fail with this message: ERROR: dependency ‘evaluate’ is not available for package ‘rzeppelin’ * removing ‘/root/Zeppelin-With-R/R/lib/rzeppelin’ Where u able to install the dependencies, I am thinking i have to downgrade my R version. Which version did u install. The gist takes the latest version: yum install R-* > install.packages("evaluate", dependencies = TRUE) Installing package into ‘/usr/lib64/R/library’ (as ‘lib’ is unspecified) --- Please select a CRAN mirror for use in this session --- Error in download.file(url, destfile = f, quiet = TRUE) : unsupported URL scheme HTTPS CRAN mirror 1: 0-Cloud [https] 2: Austria [https] 3: Chile [https] 4: China (Beijing 4) [https] 5: Colombia (Cali) [https] 6: France (Lyon 2) [https] 7: France (Paris 2) [https] 8: Germany (Münster) [https] 9: Iceland [https] 10: Mexico (Mexico City) [https] 11: Russia (Moscow) [https] 12: Spain (A Coruña) [https] 13: Switzerland [https] 14: UK (Bristol) [https] 15: UK (Cambridge) [https] 16: USA (CA 1) [https] 17: USA (KS) [https] 18: USA (MI 1) [https] 19: USA (TN) [https] 20: USA (TX) [https] 21: USA (WA) [https] 22: (HTTP mirrors) Selection: 21 Warning: unable to access index for repository https://xyz.xxxx.org/src/contrib: unsupported URL scheme Warning message: package ‘evaluate’ is not available (for R version 3.2.3) > quit; ... View more

Re: best way to install/integrate numpy scikit to ...

by Guru in Support Questions
‎03-08-2016 04:32 PM
2 Kudos
‎03-08-2016 04:32 PM
2 Kudos
@Artem Ervits your suggestion worked. This is what i ran to get it to run on my sandbox : yum install -y numpy ... View more

best way to install/integrate numpy scikit to an e...

by Guru in Support Questions
‎03-08-2016 02:25 PM
2 Kudos
‎03-08-2016 02:25 PM
2 Kudos
Hello Experts! I noticed that the default zeppelin install on hdp sandbox does not come with numpy and scikit learn packages. I can install pip manager and install the packages manually, but i want to make sure the zeppelin installation will pickup those packages. Anyone added these packages to their cluster? This is the error that i am getting: Traceback (most recent call last): File "/tmp/zeppelin_pyspark.py", line 162, in <module> eval(compiledCode) File "<string>", line 1, in <module>ImportError: No module named numpy the zeppelin property in ambari zeppelin.install_python_packages is set to false. I tried switching it to true but it does not do anything, its an install only read property i am assuming than. Thanks! ... View more
Labels:

Re: Is there a Spark Maven archetype for Java?

by Guru in Support Questions
‎02-01-2016 05:30 PM
‎02-01-2016 05:30 PM
@Vladimir Zlatkin that would be awesome if we had and HDP Maven archetype. Following the issue in case we find a custom archetype. ... View more

Re: Spark History UI crashing often?

by Guru in Support Questions
‎02-01-2016 05:22 PM
1 Kudo
‎02-01-2016 05:22 PM
1 Kudo
@Joshua can you increase the SPARK_DAEMON_MEMORY to 4g SPARK_DAEMON_MEMORY Memory to allocate to the history server (default: 1g). http://spark.apache.org/docs/latest/monitoring.html ... View more

Re: Index Documents using HDPSearch in HDP 2.3

by Guru in Community Articles
‎12-29-2015 09:50 PM
‎12-29-2015 09:50 PM
Thanks @Ali Bajwa ! Per our conversation, clean out the directories, stop solr, and if you got into bad state create the solr collection using a new name , that will do the trick ... View more

Re: Index Documents using HDPSearch in HDP 2.3

by Guru in Community Articles
‎12-29-2015 05:33 PM
‎12-29-2015 05:33 PM
Sorry had to break up the comment because of 600 character limit ... View more

Re: Index Documents using HDPSearch in HDP 2.3

by Guru in Community Articles
‎12-29-2015 05:33 PM
‎12-29-2015 05:33 PM
<directoryFactory name="DirectoryFactory"/> I tried replacing with <directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory"> But that did not help got the same error ... View more
Contact Me
Online
Offline
Last Visited
‎08-14-2019 06:45 PM
Community Statistics
Member Since ‎09-29-2015 01:18 AM
Last Visited ‎08-14-2019 06:45 PM
Posts 155
Kudos received 171