Member since
03-03-2017
17
Posts
3
Kudos Received
0
Solutions
03-08-2017
03:17 PM
Now when i type :
%spark
hiveContext.sql("create table riskfactor as select * from final")
it says :
org.apache.spark.sql.AnalysisException: `default`.`riskfactor` already exists.;
at org.apache.spark.sql.hive.execution.CreateTableAsSelect.run(CreateTableAsSelect.scala:86)
at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58)
at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56)
... View more
03-08-2017
03:12 PM
Im taking this tutorial https://community.hortonworks.com/questions/ask.html?space=81&topics=tutorial-100&topics=hdp-2.5.0, I went step by step , but in this line zepplin says :
%spark
hiveContext.sql("create table riskfactor as select * from finalresults")
java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.
This stopped SparkContext was created at:
org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
first i`ve created the table riskfactor in the beggining of the zepplin script with this command :
%hive
CREATE TABLE riskfactor (driverid string,events bigint,totmiles bigint,riskfactor float) STORED AS ORC And then im calling this at the end :
%spark hiveContext.sql("create table riskfactor as select * from finalresults")
Could anyone help me ?!
... View more
Labels:
- Labels:
-
Apache Zeppelin
03-07-2017
02:57 PM
such a compiler !
... View more
03-07-2017
01:33 PM
free -m shows 115 free
i have 6 running services everyday. ill try to increase ram there
... View more
03-07-2017
01:20 PM
2 Kudos
This is my script and i have also a table called riskfactor in Hive .
a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by event != 'normal';
c = foreach b generate driverid, event, (int) '1' as occurance;
d = group c by driverid;
e = foreach d generate group as driverid, SUM(c.occurance) as t_occ;
g = LOAD 'drivermileage' using org.apache.hive.hcatalog.pig.HCatLoader();
h = join e by driverid, g by driverid;
final_data = foreach h generate $0 as driverid, $1 as events, $3 as totmiles, (float) $3/$1 as riskfactor;
store final_data into 'riskfactor' using org.apache.hive.hcatalog.pig.HCatStorer();
When i execute this pig script it shows me that error and not saving any data.
I did step by step everything it says here :
https://hortonworks.com/hadoop-tutorial/hello-world-an-introduction-to-hadoop-hcatalog-hive-and-pig/#section_5
... View more
Labels:
03-07-2017
12:45 PM
i just have to figure out that i use oracle vm virtualbox , so is this issue related with that , or network ,or laptop , or what ?!
... View more
03-07-2017
12:44 PM
1 Kudo
I am a begineer to Hadoop and im taking Hortonworks Sandbox tutorials . Ambari server runs so slow , hive , pig , every job every service , it loads and run very very slow , example if i run a simple select query of 10 rows , it takes tike to execute . My question is : is it okay with this , or do i have to install specific tools for different tasks ,like i`ve seen a tutorial when it is saying that qyering with beeline tool is faster , or do i have to execute everything with Putty , or to continue working in Ambari server http://localhost:8080/ . I appriciate every explanation . Thanks for time .
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
03-07-2017
08:50 AM
now it is ok
... View more
03-06-2017
03:29 PM
Ambari version 2.4.0
... View more