About pcoates

pcoates · ‎03-17-2016

I don't think it helps us here, but I'm glad to know it. Thanks

pcoates · ‎03-17-2016

Ambari can send SNMP traps to alert downstream systems. Is there a way to get Oozie to send similar alerts when jobs fail?

pcoates · ‎03-01-2016

This might be a good workaround. All we want is the information--it doesn't have to be a blueprint per se. Thanks!

pcoates · ‎02-29-2016

I got the following answer to this from another source so I'll post it here. The expected behavior is for Ambari to dump all the configurations into the blueprint, so the result I'm getting is indeed not what it should normally be expected. However, this is such an old Ambari version that blueprints were not well supported yet, and the recommended solution (at least the one I recommended to me was "you're going to have to upgrade.") This is a really old cluster---the blueprints functionality seems fine in more recent Ambari versions.

pcoates · ‎02-29-2016

Browser is normal, with all configurations are as expected. Here's what a host group looks like. As you see, it just names the services, but nothing else. Am I missing a parameter in the cURL call? It's a 1.6/2.1.5 cluster. { "name" : "host_group_3", "components" : [ { "name" : "HBASE_REGIONSERVER" }, { "name" : "SUPERVISOR" }, { "name" : "NODEMANAGER" }, { "name" : "DATANODE" }, { "name" : "GANGLIA_MONITOR" } ], "cardinality" : "10" },

pcoates · ‎02-29-2016

I'm trying to obtain configuration information from an existing Ambari 2.1.5 cluster (preparatory to an upgrade.) Rather than go through the GUI I dumped a blueprint a follows: curl -H "X-Requested-By: ambari" -X GET -u admin:admin http://my.fqdn.net:8080/api/v1/clusters/mycluster?format=blueprint But the result yields only the bare lists of the services that go in each of host_group_1 through host_group_6. It does not contain any of the configuration information that I expected find, similar to say, the sandbox blueprint. Only 253 lines and zero configuration settings. In comparison I get 2218 lines of JSON from the sandbox which includes many specifics for each listed service. Why am I not seeing them and how can I get them?

pcoates · ‎02-23-2016

Is there any built-in way to query Ambari for (a) when a service was last in maintenance mode, (b) the last error for a service plus why, etc. Accessing this information via the REST API would be ok if it is possible. Accessing it via Phoenix could also work, but is there a working example?

pcoates · ‎02-18-2016

Thanks for gettng back. Yes--I'm aware of the -m option, but it appears from the documentation that the mappers get a list of HDFS level files and work on these. I'm trying to find out if my understanding is accurate: that unlike a typical map reduce job that deals in single blocks or splits, the distcp maps each get the URI of an entire file or files to copy. Therefore, you might have hundreds of blocks, but if it's all one file, the same mapper will handle all. Is this the case?

pcoates · ‎02-18-2016

We see very few mappers created for discp copies. Are these mappers being allocated at the block level or at the file level? I.e., does a mapper copy a physical block or does it copy an entire logical file?

pcoates · ‎02-12-2016

All---thanks for the very helpful answers. The real issue here is that values get changed after the original correct installation. Then you get nailed by surprise later because arbitrarily much time can go by before processes are restarted (That's what happens repeatedly here.) It would be wonderful if Ambari could have an option to execute the same script it executes to do install-time checks periodically to catch this kind of thing.

pcoates · ‎02-05-2016

Aha. The problem turns out to be with the multiple directories named in the file naming the sources. You can have many sources, but only one target. The behavior I was looking for would be for distcp to make a separate tree for each input directory under the target. This seems not to be the way distcp works, but it's easy to script around it.

pcoates · ‎02-03-2016

I must have been unclear. We definitely want to use discp and cannot use Falcon for admin reasons. The problem is that I can't get the fully recursive behavior with discp. There's probably a way to do it, but I'm having trouble getting it to build the full depth of the directories on the target if it goes more than one level deep.

pcoates · ‎02-03-2016

Falcon is not available in my environment, unfortunately. Is there no way to do this without it? This must come up fairly often with partitioned HDFS files and ORC.

pcoates · ‎02-02-2016

I have a cluster with THP inadvertently left enabled. If I disable it, will the processes that are already running stop using it, or do they need to be restarted. Restarting is very inconvenient in this environment.

pcoates · ‎02-02-2016

I need to take a list of HDFS directories and copy the contents of those directories to another HDFS using discp. The problem is recursively creating the directories automatically. These are large partitioned files, and the available means seem to preserved structure only one level deep. Can anyone provide an example?

pcoates · ‎01-12-2016

One last detail---if the time runs out, and the blocks go on the queue for replication, what happens when the node comes back online and reports. Are they stricken from the queue? What if they've already been replicated?

pcoates · ‎01-11-2016

The three staleness properties control how long it will take for nodes that have not been heard from are regarded as stale, and whether to read or write to such nodes. I don't think that's what we're looking for. What I'm asking is whether it is necessary to avoid replicating blocks on nodes that are temporarily offline. I found the property dfs.namenode.replication.interval which is described as "controlling the periodicity with which the NN computed replication work for data nodes." It sounds like bumping it up temporarily might work. Opinion?

pcoates · ‎01-11-2016

I have a requirement to periodically restart all cluster nodes at the machine level. Assume I've done an FSCK before starting to confirm that all blocks are fully replicated. Question is, as I restart each node in turn, will the NameNode notice that any block on that node is under-replicated and put those blocks on the replication queue? If this does happen, will it automatically remove those blocks when the data node comes back online and reports it's blocks to the NN? Note, this is a hardware restart, so the Ambari rolling restart doesn't do the job.

pcoates · ‎01-06-2016

Within a cluster we have no trouble executing commands agains an HA NameNode using the NameServiceID. But it doesn't work when doing discp from one cluster to another because the clusters are unaware of each other's mapping of nodes to NameServiceID. How does one do this?

pcoates · ‎01-04-2016

We have two use cases--one is the normal slight imbalance that can creep up gradually and the other is when we add new nodes. Ten new nodes can be 100TB+ to move around--it can take a very long time with normal dfs.network.bandwidth.persecond setting. What's a good strategy? Is it reasonable to use chron to reset the value during off hours? What's the best practice? Also, does rebalancing defer to normal processing dynamically?

pcoates · ‎12-16-2015

Let me back up a step. Does experience tell you that a rolling restart is a sensible thing to do? Not everything that is possible is advisable.

pcoates · ‎12-15-2015

How can I automate a rolling restart of all the nodes in a cluster so it can be done without human intervention (all HA features are enabled)? I'm aware that you can do a rolling restart of the data nodes through Ambari, but we want to restart masters, edge nodes, etc. too. Organization requires restarting all nodes periodically and we want to avoid all manual steps.

pcoates · ‎12-09-2015

The documentation seems to suggest that the normal mode of use would be to have one reconstituted replica sitting around and that reconstituting an encoded block would be done only if this isn't the case. Keeping a block by default would eliminate most of the space savings because the data would expand from 1.6 to 2.6 times the raw file size. Why not have a policy that for leaves a single size copy for a limited time after a block is used? A "working set" as it were, so if you've used a block in the last X hours the decoded block won't be deleted.

pcoates · ‎12-08-2015

The admins want to know why every service has its own account ID, and is there any harm is using the same account for all? The cluster will be tightly secured. What is the best practice?

pcoates · ‎12-06-2015

Erasure Coding is apparently greatly speeded up by the ISA-L libraries. Will HDP be using these libraries? They are open-source, but I assume they have some optimizations for Intel processors, as they were written by Intel.

pcoates · ‎12-06-2015

Hadoop has long stressed moving the code to the data, both because it's faster to move the code than to move the data, and more importantly because the network is a limited shared resource that can easily be swamped. Erasure coding would seem to require that a large proportion of the data must move across the network because the contents of a single block will reside on multiple nodes. This would presumably apply not just the ToR switch, but the shared network as well, if the ability to tolerate the loss of a rack is preserved. Is this true and how are these principles reconciled?

pcoates · ‎11-23-2015

Your inode article is a great addition to David's answer. I'm puzzled though that any machine would run out of inodes before running out of disk space---it would require a strange configuration of the file system, wouldn't it? Was someone trying to save on inode allocation by assuming the average file would be larger? I can't think of any other reason to stray from the defaults. Any idea why?

pcoates · ‎11-21-2015

Thanks Neeraj and Deepesh---that's what I needed to know.

pcoates · ‎11-20-2015

Thanks Ancil. I'm still curious about what can be done from inside Hadoop. The federation of queries is particularly interesting becauese you don't always want to import the data into HDFS.

pcoates · ‎11-20-2015

Thanks for the reply. Yes, I read that page--the problem is trying to confirm whether the version of the connector in that tarball that this leads to, which seems to be for HDP 2.3 works with 2.2.4. Can't seem to locate one specifically for 2.2.4.

Online	Offline
Last Visited	‎03-30-2016 02:57 PM

Date Registered	‎10-06-2015 02:10 PM
Last Visited	‎03-30-2016 02:57 PM
Total Messages Posted	45
Total Kudos Received	48

Re: How can I get SNMP alerts for failed Oozie job...

How can I get SNMP alerts for failed Oozie jobs?

Re: Getting only skeletal configuration data from ...

Re: Getting only skeletal configuration data from ...

Re: Getting only skeletal configuration data from ...

Getting only skeletal configuration data from Amba...

How can I query Ambari for historical error and ma...

Re: Number of distcp mappers is small. Why?

Number of distcp mappers is small. Why?

Re: If THP is disabled after Java process start, d...

Re: How to discp a partitioned multi-level directo...

Re: How to discp a partitioned multi-level directo...

Re: How to discp a partitioned multi-level directo...

If THP is disabled after Java process start, do th...

How to discp a partitioned multi-level directory w...

Re: How does restarting a data node affect block r...

Re: How does restarting a data node affect block r...

How does restarting a data node affect block repli...

How to use Name Service ID between to Clusters

What are the best practices for HDFS rebalancing?

Re: Is there any way to do a fully automated rolli...

Is there any way to do a fully automated rolling r...

Re: How will Erasure Coding affect the principle o...

Why are there 21 separate service accounts?

Will Erasure Coding in HDP 3.0 use ISA-L, and to w...

How will Erasure Coding affect the principle of da...

Re: How many files is too many on a modern HDP clu...

Re: Connecting Teradata to HDP 2.2.4

Re: Access modes for teradata beyond Sqoop ingesti...

Re: Connecting Teradata to HDP 2.2.4