About mbigelow

mbigelow · ‎02-04-2017

Check the hue.ini to see if its blacklisted. Check in CM and also get the file from the actual server. Refer to the link I posted for the specific setting to blacklist apps.

mbigelow · ‎02-04-2017

Depending on your budget and feelings on opensource there are two options to push the accelerator. I haven't dug into either one to talk to how they do it. If you wanted to just roll your own I'd imagine it would be following the star schema methodology the Kimball is known for in Impala and then just leveraging Impala/HDFS caching to make it as fast as possible. http://kylin.apache.org/ or http://www.atscale.com/

mbigelow · ‎02-04-2017

I am going to go with bug and will try to test this out to confirm so an official bug report can be submitted. I think I ran into something similar. I wasn't upgrading; fresh install. I did restore a metastore from an older version and ran the same command to update NN URI for Hive. The tables were updated correctly if I recall correctly but the DB locations were not. They had the same double port entry made. I did the same method to fix by updating the DBS table in the metastore DB directly; I didn't no try the other command you listed. This was on CDH 5.8.2.

mbigelow · ‎02-04-2017

Does the group that the super user is in have the impala permission? Did you try to edit a group and find it in that list (you will need to scroll down)? Is it still not there? If yes, check the last bit about blacklisting apps. As it is most likely blacklisted (although that should blacklist it for the super user as well). http://gethue.com/how-to-manage-permissions-in-hue/

mbigelow · ‎02-03-2017

In Hue User Admin there are users and groups, and the groups have assigned access to the various apps. Check the group that the users is it to make sure it has access to the Impala app.

mbigelow · ‎02-03-2017

hdfs dfs -stat %r /path/to/file (and wildcards work). This just prints out the factor and not the file name.

mbigelow · ‎02-03-2017

The replication factor for a file is set on ingest. And simple, unless you mark it as 'final', a client can override any setting (well some are marked as final in the code base, replication factor is not one of them). In short, the client always defines the replication factor. As an example, the Terasort application provide with the distribution has the replication factor set to 1. Unless you change it the terasort output is written to HDFS with only one replica. The Namenode just keeps track of how many replicas there should be and are, etc.

mbigelow · ‎02-03-2017

The map container memory was set to 4 GB. Presumably the heap value was set to 3 GB (newer versions have a percentage and auto set the heap size of the container and the default percentage is 80%; 3/4 is 75%). The 6 GB comes from virtual memory, which I recommend just disabling as it can cause weird OOM issues. The default virtual memory ration is 2.1 which doesn't come out to 6 from 4. The log even states that the latter is the virtual memory size. yarn.nodemanager.vmem-check-enabled = false to disable.

mbigelow · ‎02-03-2017

Can you demonstrate? Show an empty directory, then upload a file, then run hdfs dfs -getrep on the file, and post the output here. Also post the hdfs-site.xml if possible from the client you are one or were you upload from? It just dawned on me that you stated that the NN and DN have replication set to 2 but if the clients uploading the data still have it set to 3 in their hdfs-site.xml then the file(s) will have a factor of 3.

mbigelow · ‎02-03-2017

https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cm_ig_feature_differences.html The main difference is that you do get a lot of features that would make management easier specifically around configuration versioning, encryption, security, etc. There will be not technical limitation on the services between the versions. Since you were told to get it from Apache it is worth mentioning that CDH is a package distribution that Cloudera integrates and tests. This means that you won't have to do it but it also means that you will have to go at Cloudera pace when adopting new projects or new versions (technically you can add your own as well but my view is that if you are going to be doing that anyway why not do it for all).

Online	Offline
Last Visited	‎03-25-2019 05:55 PM

Member Since	‎08-16-2016 08:51 PM
Last Visited	‎03-25-2019 05:55 PM
Posts	642
Kudos received	129

Cloudera Community

Re: Configuring the HDFS superuser in Kerberos

Re: Hive process crash

Re: Upgrade from CDH 5.11 Express to Enterprise

Re: Adding user to Cloudera Manager using REST AP...

Re: Running in non-interactive mode, and data appe...

Re: Unable to access Impala app from HUE Web UI

Re: how to create OLAP Cubes on HADOOP data - Imp...

Re: [CDH 5.10 upgrade] Wrong FS Hive tables

Re: Unable to access Impala app from HUE Web UI

Re: Unable to access Impala app from HUE Web UI

Re: NameNode alerting on Blocks under replicated e...

Re: NameNode alerting on Blocks under replicated e...

Re: Yarn Application failed on out of memory

Re: NameNode alerting on Blocks under replicated e...

Re: Need clarity on Cloudera's Hadoop Free Editio...