Hi I had a similar problem but not related to the Query size, but the query list. There were queries executed on the cluster and not returned by the API. I have found out that I need to timefilter and offeset to parse all the paginated data. ... View more

This just says you dont have a permission with mapred user to that directory. Without knowing any details, one of the solution could be to add mapred user to a supergroup group on every worker node.   ... View more

Edit: adding query timeout does not affect this behaviour:   Configured Hue to 30sec timeout, but the query is waiting to be closed for more than 2 minutes...   This is directly from the Query profile: Query Options (set by configuration): MEM_LIMIT=419430400,QUERY_TIMEOUT_S=30 Query Options (set by configuration and planner): MEM_LIMIT=419430400,QUERY_TIMEOUT_S=30,MT_DOP=0   ... View more

One more observation:  during the query "fetch time" the query on Impala daemons is reported as: "waiting to be closed" But has a state=FINISHED, First row fetched, Scan progress 100%.   So my additional question is why is Impala not closing automatically the queries when the state is in "FINISHED"? Is this a configurable behaviour?    Thanks! ... View more

I am glad that I could help T. ... View more

I did configure the c5d instance types, but it is still failing:   Process logs can be found at /root/.cloudera-director/logs/application.log Plugins will be loaded from /var/lib/cloudera-director-plugins Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0 Cloudera Director 2.8.0 initializing ... Connecting to http://localhost:7189 Current user roles: [ROLE_READONLY, ROLE_ADMIN] Configuration file passes all validation checks. Creating a new environment... Creating external database servers if configured... Creating a new Cloudera Manager... Unexpected internal error (see logs): { "validationErrors" : [ "Incompatible AMI virtualization type. Instance type c5d.2xlarge does not support hvm virtualization type of AMI ami-3548444c." ], "validationWarnings" : [ ], "conditions" : [ { "scope" : "aws.com.cloudera.director.aws.ec2.EC2Provider.template", "key" : "image", "exceptionInfo" : { "@class" : "com.cloudera.launchpad.common.error.ErrorInfo", "code" : [ "com.cloudera.launchpad.pluggable.common.ProviderErrorCode", "PROVIDER_EXCEPTION" ], "properties" : { "message" : "Incompatible AMI virtualization type. Instance type c5d.2xlarge does not support hvm virtualization type of AMI ami-3548444c." }, "causes" : [ ], "message" : "Incompatible AMI virtualization type. Instance type c5d.2xlarge does not support hvm virtualization type of AMI ami-3548444c." }, "message" : "Incompatible AMI virtualization type. Instance type c5d.2xlarge does not support hvm virtualization type of AMI ami-3548444c." } ], "formatWarnings" : [ ] } [centos@ip-10-197-19-68 ~]$ [centos@ip-10-197-19-68 ~]$ cat /var/lib/cloudera-director-plugins/aws-provider-1.5.0/etc/ec2.customvirtualizationmappings.properties hvm=c5d.xlarge,c5d.2xlarge,c5.large,m5.xlarge [centos@ip-10-197-19-68 ~]$ cat /var/lib/cloudera-director-plugins/aws-provider-1.5.0/etc/ec2.ephemeraldevicemappings.properties # (c) Copyright 2015 Cloudera, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # Comprehensive list of EC2 ephemeral device mappings at: # http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html#StorageOnInstanceTypes c5d.2xlarge=1 [centos@ip-10-197-19-68 ~]$ ... View more