Member since
02-11-2016
53
Posts
21
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2726 | 11-03-2016 01:22 PM | |
4030 | 10-31-2016 07:12 PM | |
2352 | 02-12-2016 02:40 PM |
11-03-2016
01:22 PM
1 Kudo
@Timothy Spann I was finally able to get running again: - Removed references to lzo from all configurations (using Ambari) - Manually removed all RPM packages on all machines that match *lzo* Then I re-read the Ambari instructions for about the 40th time and realized where communication was breaking down. The only installation of packages I ever observed from Ambari was during initial installation and all of it was triggered from dialogs. It may be obvious to some folks that Ambari uses the presence of the code in the io.codecs list as a trigger for silent package install on restart, but it certainly wasn't to me (since NOTHING else I've encountered in the system works in that manner. All other installs have a progress indication and function test). Once I added the configuration (without having manually installed packages first), it indeed installed them itself during restart and everything worked when complete. I would strongly suggest adding a small paragraph to the lzo configuration page to explicitly and clearly explain that this process physically installs the packages with no visual indication that this is occurring.
... View more
11-02-2016
05:54 PM
@Timothy Spann - Ambari on our system does not provide any facility to install lzo. You keep referring to this, but it isn't there. If you believe it should be, please tell me where I might find the dialog? - I followed ALL the steps you outlined above, except for Hive. I DO NOT WANT LZO COMPRESSION ON HIVE. If that's not optional, then it should be documented as such. - I did have things stopped when I installed the RPMs and updated configuration. We're in a real mess here and currently trying to find someone to help us recover. I wish your company provided per-incident support, but that doesn't seem to be the case.
... View more
11-02-2016
05:09 PM
@Timothy Spann The cluster itself was installed through Ambari and has been running about a year. One of my users needed LZO compression enabled several days ago. You web site told me that Ambari does not install or configure LZO, so I followed the instructions as you entered them above. I added two changes to core-site.xml that were similarly documented in the HDP 2.3.2 web pages. After fixing an initial typo, we had working LZO and could explicitly invoke LzoIndexer on files in HDFS. Shortly after that I started receiving reports about Hive being broken. Originally it was complaining that it could not find the LzoCodec. I never told it to use the LzoCodec. I did not change Hive configuration. After removing the entries in core-site.xml, the Hive problems continued but it now tells me it cannot find "com" - a nonsense class name. I did restart everything that needed to be restarted - several times, in fact. The only thing amiss in the Hive logs is the same traceback the user gets on a failed query: 2016-11-02 12:57:18,278 WARN [HiveServer2-Handler-Pool: Thread-2740]: thrift.ThriftCLIService (ThriftCLIService.java:FetchResults(681)) - Error fetching results:
org.apache.hive.service.cli.HiveSQLException: java.io.IOException: java.lang.RuntimeException: Error in configuring object
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:352)
at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:221)
at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:685)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:454)
at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:672)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:347)
... 13 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hive.common.util.ReflectionUtil.setJobConf(ReflectionUtil.java:115)
at org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:103)
at org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:207)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:361)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:295)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
... 17 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor194.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hive.common.util.ReflectionUtil.setJobConf(ReflectionUtil.java:112)
... 23 more
Caused by: java.lang.IllegalArgumentException: Compression codec com not found.
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:179)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 27 more
Caused by: java.lang.ClassNotFoundException: Class com not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
... 29 more
2016-11-02 12:57:18,281 INFO [HiveServer2-Handler-Pool: Thread-2740]: exec.ListSinkOperator (Operator.java:close(613)) - 10800 finished. closing...
We are running HDP-2.3.2 on Centos 6.7. I do not know where to start troubleshooting this, particularly since it's not deterministic. Only some queries are blowing up with no obvious common denominator across them. Again, we made no changes to Hive and my users have made no changes in the way they are querying it.
... View more
11-02-2016
03:10 PM
@Timothy Spann And that is precisely what I had done - to the letter. If there was an option for installation from Ambari, it is not evident. Where exactly is this "wizard" you refer to? Your web documentation states clearly that Ambari neither installs nor configures LZO. The proximate issue is that Hive is totally broken now - even after removal of the two changes made to core-site.xml. Why is Hive even TRYING to use LZO? I did not configure that - I did not so much as touch Hive.
... View more
11-02-2016
02:13 PM
2 Kudos
After following the directions here (I'm on Linux, but could not locate the page pertinent to the Linux HDP): http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2-Win/bk_HDP_Install_Win/content/LZOCompression.html All attempts at inserting into existing Hive tables (which are NOT setup for LZO compression) yield a long traceback featuring this : Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.LzoCodec not found.
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139) at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:179)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 21 more
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.LzoCodec not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
... 23 more Why on earth is Hive even trying to use LZO? Very frustrating to find this level of fragility. Any way to get LZO to coexist with a functional Hive? Update: I removed any and all mention of LZO from core-site.xml and Hive is still blowing up while search for codecs. Looks like we now have a completely hosed cluster.
... View more
Labels:
- Labels:
-
Apache Hive
10-31-2016
07:12 PM
I was missing com.hadoop.compression.lzo.LzopCodec in the compression codecs listing... Grrr. The error message proved to be utterly misleading.
... View more
10-28-2016
09:01 PM
I'm trying to get LZO compression to work on our HDP 2.3.2 cluster and getting nowhere. Here's what I've done: - Installed the hadooplzo and hadoop-lzo-native RPMs - Made the documented changes to add the codec and the lzo class spec to core-site.xml When I try to run a job thusly: yarn jar /usr/hdp/2.3.2.0-2950/hadoop/lib/hadoop-lzo-0.6.0.2.3.2.0-2950.jar com.hadoop.compression.lzo.LzoIndexer /path/to/lzofiles It tells me: [hirschs@sees24-lin ~]$ yarn jar /usr/hdp/2.3.2.0-2950/hadoop/lib/hadoop-lzo-0.6.0.2.3.2.0-2950.jar com.hadoop.compression.lzo.LzoIndexer /xxxx/yyy
16/10/28 16:44:56 ERROR lzo.GPLNativeCodeLoader: Could not load native gpl library
java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:32)
at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:71)
at com.hadoop.compression.lzo.LzoIndexer.<init>(LzoIndexer.java:36)
at com.hadoop.compression.lzo.LzoIndexer.main(LzoIndexer.java:134)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
16/10/28 16:44:56 ERROR lzo.LzoCodec: Cannot load native-lzo without native-hadoop
16/10/28 16:44:57 INFO lzo.LzoIndexer: LZO Indexing directory /xxxxx/yyyyy...
16/10/28 16:44:57 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file hdfs://correct_path_to_file, size 1.08 GB...
16/10/28 16:44:57 INFO compress.LzoCodec: Bridging org.apache.hadoop.io.compress.LzoCodec to com.hadoop.compression.lzo.LzoCodec.
16/10/28 16:44:57 ERROR lzo.LzoIndexer: Error indexing hdfs://correct_path_to_file
java.io.IOException: Could not find codec for file hdfs://correct_path_to_file - you may need to add the LZO codec to your io.compression.codecs configuration in core-site.xml
at com.hadoop.compression.lzo.LzoIndex.createIndex(LzoIndex.java:212)
at com.hadoop.compression.lzo.LzoIndexer.indexSingleFile(LzoIndexer.java:117)
at com.hadoop.compression.lzo.LzoIndexer.indexInternal(LzoIndexer.java:98)
at com.hadoop.compression.lzo.LzoIndexer.indexInternal(LzoIndexer.java:86)
at com.hadoop.compression.lzo.LzoIndexer.index(LzoIndexer.java:52)
at com.hadoop.compression.lzo.LzoIndexer.main(LzoIndexer.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
I get the feeling I'm missing a step somewhere. The shared libraries appear to be in place: [hirschs@sees24-lin native]$ rpm -ql hadoop-lzo-native
/usr/hdp/current/share/lzo/0.6.0/lib/native
/usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64
/usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.a
/usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.la
/usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.so
/usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.so.0
/usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.so.0.0.0
/usr/hdp/current/share/lzo/0.6.0/lib/native/docs
In core-site.xml: <property>
<name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec</value>
</property>
In hdfs-site.xml: <property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
What more do I need to do in order for this to run? Even a guess would be helpful at this point.
... View more
Labels:
09-15-2016
06:36 PM
@ripunjay godhani I want to be sure I understand your post. Are you saying that modifying a single Ambari property will relocate logs for all components on a restart? If so, can share what the name of that property is? The page you linked to does not have a single mention of log location. In a perfect world, I would have left plenty of room under /var for logging, but we have a heavily used cluster with a lot of data and constant crashes from full /var on many of the machines. I need to move everything to a new location.
... View more
08-01-2016
12:53 PM
@Benjamin Leonhardi Thanks for the explanation. We're having significant scaling issues with our 32-host Hortonworks HDP-2.3.2.0-2950 installation. How do I determine which version of ATS is installed? I do not see it listed in the 'Stacks and Versions' page in Ambari. Assuming we're running one of the troublesome versions, what's the most expedient way to disable reporting to ATS from Hive? Since it is useful for debugging, I'm hoping there's a session parameter we can set at query time to suppress reporting when performance is an issue.
... View more
07-28-2016
08:14 PM
I'm curious what the advantage of suppressing Hive ATS reporting might be. From an esthetic standpoint we really don't want the UI filled up with myriads of successful, short-running queries, but it would be nice to switch it on in a case by case basis for debugging purposes. Beyond that, would turning it off improve query latency?
... View more
06-30-2016
03:47 PM
1 Kudo
I wanted to post a quick followup on this thread. We recently found ourselves in a situation where we needed to deploy the hbase client code on an arbitrary number of machines and did not want the overhead of using Ambari. It was very straightforward to setup the Hortonworks repository reference and pull down hbase, however even after adding Phoenix the hbase shell would fail at startup with the dreaded (and spectacularly uninformative) exception: NativeException: java.io.IOException: java.lang.reflect.InvocationTargetException
initialize at /usr/hdp/2.3.2.0-2950/hbase/lib/ruby/hbase/hbase.rb:42
(root) at /usr/hdp/2.3.2.0-2950/hbase/bin/hirb.rb:131 After almost 1/2 day of hair-pulling, I ran strace against the shell startup on a working node and compared it to the trace from the failing one. It turns out that the shell absolutely requires this directory path to exist (it can be empty): /hadoop/hbase/local/jars Once I created that hierarchy the shell was able to start successfully: $ mkdir /hadoop
$ chmod 1777 /hadoop
$ mkdir -p /hadoop/hbase/local/jars
$ chmod -R 755 /hadoop/hbase Hopefully this will save someone else the time and aggravation.
... View more
03-16-2016
01:27 PM
@stevel Fantastic! That's a great example of useful and practical documentation. I'll let you know what I turn up from making the REST calls.
... View more
03-15-2016
04:59 PM
@stevel I made those two changes and restarted Spark. A job submitted with '--master yarn-client' still behaves as before, with the history server not correctly tracking the job. A job submitted with '--master yarn-cluster' does get picked up as a completed job in history, but when I drill in there is absolutely no information available relative to the job. The environment tab is populated, but not with anything obviously job-specific. The 'executors' tab has the following: executors.png which is suspiciously devoid of any actual activity. 'Stages', 'Storage' and 'Jobs' are completely blank. I understand in the abstract what you're asking for in terms of querying the ATS server, but it's going to take me some time to determine the required web-service calls and put that together. It's something I probably need to know about, but won't have the time to dig in for a day or so. Thanks for your help to this point! I'll try to get the rest of the information later this week.
... View more
03-14-2016
06:49 PM
1 Kudo
@stevel I moved the yarn timeline directory to /hadoop/yarn and restarted. I'm no longer seeing the 500 error from the Spark History UI, but it continues to list completed Spark jobs in 'incomplete' telling me that there are hundreds of tasks remaining to be run. The YARN history UI does correctly report that the job is complete. incomplete.png The developer who owns the application tells me that it appears to be returning proper results.
... View more
03-14-2016
06:13 PM
1 Kudo
@stevel I am seeing that same 500 error when working directory from the browser. Moving the timeline storage path is not a problem. I've read some suggestions about moving it to HDFS, but I'm not sure what other ramifications that may have so I'll stick with machine-local storage for now. Not sure if you saw one of my earlier posts where I mentioned that the Spark daemon log is filling with errors whose meaning is not clear (see beginning of thread above). Perhaps that will go away when I relocate the log directory.
... View more
03-14-2016
04:05 PM
@stevel Hi. We are using HDP-2.3.2.0-2950, with all nodes running Centos 6.7. Not sure I know how to answer the question about logs. For starters, it's not easy to understand where these would be. If I assume to history server to be the machine that I connect to for the Spark History UI and I assume that job-related logs would be under /tmp, then there's nothing relevant on that box. If I look on the namenode I can see /tmp/hadoop/yarn/timeline with populated subdirectories. Are those what you are referring to? I restarted the history server and now things are utterly non-functional. The Spark History UI shows nothing under either complete or incomplete and displays an error: Last Operation Failure: java.io.IOException: Bad GET request: status code 500 against http://bigfoot6.watson.ibm.com:8188/ws/v1/timeline/spark_event_v01?fields=PRIMARYFILTERS,OTHERINFO; {"exception":"WebApplicationException","message":"java.io.IOException: org.iq80.leveldb.DBException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /tmp/hadoop/yarn/timeline/leveldb-timeline-store.ldb/005567.sst: No such file or directo Indeed, there is no file by that particular name, but there are dozens of other .sst files present. What is causing it to look for that specific file and, further, why is it giving up completely after not finding it? We are using the YARN history service as the backend. FYI: After restarting the history server, I'm getting this is in the daemon logs on the history server host: spark-spark-orgapachesparkdeployhistoryhistoryserv.txt It looks very unhappy. All of this had been working fine as recently as late January, and I have not (knowingly) made any changes whatsoever to the Spark history configuration. Please let me know if you need any further information. I've looked through the Hortonworks courses on Hadoop management, but haven't seen any syllabus that claims to cover troubleshooting at a sufficiently low level. If that's not the case, can you advise which of them would provide enough background to be able to help in a case such as this?
... View more
03-09-2016
02:57 PM
1 Kudo
Thanks, but we do not have a support agreement. We'll just have to live with it. I've provided all the information I have.
... View more
03-08-2016
10:34 PM
2 Kudos
I'm starting to get concerned about this issue. We have run about 50 jobs in Spark that return results without any exceptional conditions, and which the YARN UI reports as complete. All of them are languishing in the Spark UI incomplete job listing with over 150 steps (it claims) left to go. The offending operation is either 'toPandas' or 'treeAggregate at GradientDescent.scala:189'. I do not see any sign that these processes are actually alive. Why are they not being reported as done?
... View more
03-08-2016
02:17 PM
2 Kudos
The Tez view appears to be working correctly. That endless cascade of exceptions from the history server must be pointing to something specific, but I unfortunately do not know how to interpret it. One of our users mentioned to me that the lingering jobs in the Spark UI are all using a Python method called 'toPandas', while the few that do get properly noted as complete do not. Is that a useful clue? The Spark "incomplete" history continues to pile up dozens of jobs that are reported on the console (and by YARN) as being finished.
... View more
03-07-2016
09:37 PM
2 Kudos
More information: Around the time the history server stopped working correctly, a cascade of exceptions appeared in the spark logs: 2016-02-04 14:55:09,035 WARN timeline.TimelineDataManager (TimelineDataManager.java:doPostEntities(366)) - Skip the timeline entity: { id: tez_con
tainer_e07_1453990729709_0165_01_000043, type: TEZ_CONTAINER_ID } org.apache.hadoop.yarn.exceptions.YarnException: The domain of the timeline entity { id: tez_container_e07_1453990729709_0165_01_000043, type: TEZ_
CONTAINER_ID } is not allowed to be changed from Tez_ATS_application_1453990729709_0165 to Tez_ATS_application_1453990729709_0165_wanghai_201602041
45330_86a58f3a-0891-4c24-bf0f-0375575077da:1 Does that shed any light on the underlying problem? The log contains > 50 MB of such messages.
... View more
03-07-2016
09:23 PM
2 Kudos
For the past month or so, all Spark jobs are either not appearing in the Spark History UI or showing as incomplete. YARN is correctly reporting all jobs, but Spark claims there are more steps yet to be run. A little background: At one point the logs started filling with errors from the Spark history service about a non-existent file. I ended up stopping the Spark history server and deleting everything in the directory it was yelling about, then restarting. I suspect I damaged something in the process and could use some advice on reinitializing the service.
... View more
Labels:
- Labels:
-
Apache Spark
02-15-2016
10:02 PM
The behavior I'm looking for is something like this: Export all deltas for all configuration files beyond settings that are built-in defaults for Hortonworks installation. I believe this would be defined as version numbers > 1 (is that correct?) Import these into a newly-built cluster and be prompted for manual intervention when a delta includes a machine name, IP address or port (the latter can probably be determined by a regex match on the property name). As an audit tool, the presentation of environment (shell) scripts should be in the form of unified diffs rather than a dump of the entire file at each revision. Just a few ideas off the top of my head. There's no way this process can be totally automated, but I think it's possible to get very close.
... View more
02-15-2016
08:00 PM
1 Kudo
Thanks, Jonas. As I mentioned in my last post, I'm in search of a way to generate a set of deltas that can propagate my tuning and tweaks to a new cluster. I try to keep notes, but there have been many occasions where I was in the midst of troubleshooting and failed to write down what I did. Exporting configuration in bulk isn't really what's needed to propagate diffs to a new cluster, since it will have machine names (and perhaps port addresses) that won't apply to the new target. Your script looks promising for a starting point, but I'd have to flesh it out with a framework that traverses all configuration files at a minimum. From a quick look it also appears that the change tracking on shell environment files will require a lot of manual work, since there's no attempt at differencing - the entire file is dumped.
... View more
02-15-2016
03:09 PM
1 Kudo
Ah - thanks. I am far from an expert on Python and did not realize this was a non-Hadoop package. A quick 'yum install python-requests' did the trick. FYI: I'm not sure if you're implying that 'audit.py' should be present on the system, but it certainly isn't here. I grabbed it from GitHub. This is a very valuable utilitity, and would be even more so if it had the ability to crawl all configuration files and generate a master listing. My proximate problem is how to effectively clone a running configuration to a new cluster. I'm surprised to find so little discussion of this task. Perhaps I'm missing something obvious, but exporting and copying configurations does not correct the myriad of machine names and IP addresses that will certainly be inappropriate in a new setting. Ambari badly needs the ability to generate bulk delta files (differences from defaults at install time) that can be imported "smartly" into a new cluster - e.g. spotting anything that looks like a URL or port address and prompting for manual intervention.
... View more
02-15-2016
02:46 PM
1 Kudo
Looks useful, but exactly which python binary is this intended to work with? It fails immediately with a complaint about not being able to find the 'request' package. I grepped the /usr/hdp tree, but cannot find a 'request.py' module.
... View more
02-12-2016
02:40 PM
1 Kudo
One of our users pointed out that Phoenix was not installed on the new client. For whatever reason, it was never presented as an option in Ambari. I had installed it manually on the six other cluster machines and forgot to do this on the new one. After pulling it down with yum, everything started cooperating. The error reporting from jruby is not particularly helpful. Hopefully this will be addressed at some point in the future?
... View more
02-11-2016
08:38 PM
1 Kudo
I added an edge node to our cluster and am running into problems with hbase shell:
[hirschs@bigfoot5 ~]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
NativeException: java.io.IOException: java.lang.reflect.InvocationTargetException
initialize at /usr/hdp/2.3.2.0-2950/hbase/lib/ruby/hbase/hbase.rb:42
(root) at /usr/hdp/2.3.2.0-2950/hbase/bin/hirb.rb:131
This works properly from the ambari server machine, but I'd like to move users off to a separate box. It appears that something may have been overlooked when I used ambari to setup the client-only machine, but what?
... View more
Labels:
- Labels:
-
Apache HBase