Member since
09-27-2016
73
Posts
9
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1215 | 09-15-2017 01:37 PM | |
2259 | 09-14-2017 10:08 AM |
11-06-2017
04:36 PM
My response went inline above - not sure why. Included is an example of the submit that I am using.
... View more
09-15-2017
01:37 PM
In fact I realized that I had to set this properties in the "Custom spark-default" section in Ambari. This way, they are written to spark-defaults.conf configuration file and things work fine
... View more
09-14-2017
10:08 AM
I figured out what was wrong : In fact my class has to extends Configured and implements Tools in order to parse the confirguration properties from the command line. Works fine now ! I even figured out that I could set the property in ambari : label "MR Map Java Heap Size" actually maps the "mapreduce.map.java.opts" property, which is pretty confusing ...
... View more
08-09-2017
07:35 AM
1 Kudo
End of the story : In fact, the problem was related to https://issues.apache.org/jira/browse/HADOOP-10786 I moved to hadoop-common 2.6.1 and used AuthUtil class: http://hbase.apache.org/1.2/devapidocs/org/apache/hadoop/hbase/AuthUtil.html And everything started to work fine 🙂 Thanks for your help
... View more
07-06-2017
06:25 AM
Hi Josh, You are right, I gave a try using another principal for my client to match the realm of the principal used by PQS and it works fine now... Thanks a lot for your help
... View more
07-02-2018
04:07 PM
Hello, I'm facing a similar issue with metrics not storing in HBase METRIC_RECORD table. I can however see the metrics being tracked since they are being returned by the /ws/v1/timeline/metrics/metadata endpoint. I have set: timeline.metrics.service.outofband.time.allowance.millis=600000 Just to give a background, I'm using StormTimelineMetricsSink to push custom topology metrics to the metrics collector & see the log statements to show that metrics are being emitted properly. Although the metric names show up in the dropdown of Grafana, there are no values to plot. I do see other other metric values showing up in the default graphs for AMS_HBASE, HOST etc. I did rm -rf hbase-tmp/ folders couple of times & started clean. I also verified there is plenty of space on the disk. Could you please help with identifying the missing connection to see the custom data pushed to the embedded HBASE?
... View more
06-15-2017
02:34 PM
Thanks a lot, it fixed the problem. Dont know why this property was set like this "out of the box" in the sandbox...
... View more
05-30-2017
08:25 PM
To answer your question regarding Zookeeper. HBase needs Zookeeper. If you didn't set up Zookeeper yourself, HBase spins up an "internal" Zookeeper server, which is great for testing, but shouldn't be used in production scenarios.
... View more
06-29-2017
09:45 AM
@Sebastien Chausson You can do this by adding --files in the spark-opts tag of your spark action. <spark-opts>--executor-memory 20G --num-executors 50 --files hdfs://(complete hdfs path)</spark-opts> As an alternative you could use a shell action and pass your spark submit command directly to it.
... View more