Reply
Contributor
Posts: 83
Registered: ‎11-12-2015

CM API: Maximum number of requests

 

Hello,

 

I was using the CM API and I think that I reached the maximum number of requests. ¿What is the maximum of requests and how can I increase this value?

 

/api/v7/clusters/cluster/services/impala/impalaQueries?from=2018-05-31T0%3A0%3A0
{ "queries" : [ ], "warnings" : [ "Impala query scan limit reached. Last end time considered is 2018-05-31T16:21:46.409Z" ] }

 

 

Regards,

Joaquin

New Contributor
Posts: 9
Registered: ‎07-06-2018

Re: CM API: Maximum number of requests

Hi Team,

 

Any one has a solution to this question already? Please share if you have, thanks.

Posts: 688
Topics: 1
Kudos: 158
Solutions: 83
Registered: ‎04-22-2014

Re: CM API: Maximum number of requests

@Prav,

 

This is an internal limit on how searches are done that restricts "scanning" across partitions (based on time).

 

When you see this message, generally you can take the time in the warning and then use that to formulate a new query specifying that as the start time.

New Contributor
Posts: 9
Registered: ‎07-06-2018

Re: CM API: Maximum number of requests

@bgooley

 

Thanks let me try that approach then.

 

Would you be kind enough and share some thoughts on :

 

https://community.cloudera.com/t5/Cloudera-Manager-Installation/Way-to-capture-Impala-Pool-metrics-A...

 

Regards

New Contributor
Posts: 9
Registered: ‎07-06-2018

Re: CM API: Maximum number of requests

@bgooley

 

Is there a particular time format that it takes as an input. For ex: 

 

When I run :

 

https://hostname:7183/api/v17/clusters/cluster_name/services/impala/impalaQueries?from=2018-07-09

 

I get warning as:

 

 "warnings" : [ "Impala query scan limit reached. Last end time considered is 2018-07-09T17:04:32.776Z" ]

 

Now if I pick time from above warning and use as below :

 

https://hostname:7183/api/v17/clusters/cluster_name/services/impala/impalaQueries?from=2018-07-09T17:04:32.776Z

 

I still get the same warning:

 

"warnings" : [ "Impala query scan limit reached. Last end time considered is 2018-07-09T17:04:32.776Z" ]

Posts: 688
Topics: 1
Kudos: 158
Solutions: 83
Registered: ‎04-22-2014

Re: CM API: Maximum number of requests

@Prav,

 

I am not an expert in these queries, but ISO 8601 format is what is used (like 2018-07-09T17:04:32.776Z).

 

If you have a lot of impala queries in this cluster you may need to specify a "to" time as well.

I'm not sure why, when you specified the time liste din the warning, the same warning appeared... that does seem strange.

 

Do you see queries listed in the CM UI?  maybe try finding one query and try making the "from" and "to" in the API query encompass one or two queries that are displayed in CM.

New Contributor
Posts: 9
Registered: ‎07-06-2018

Re: CM API: Maximum number of requests

@bgooley

 

Thanks for quick response. I tried to include from and to in a filter but unable to work it out.Do you have an example you can share?

 

https://hostname:7183/api/v17/clusters/cluster_name/services/impala/impalaQueries?filter=(from=2018-07-09 and to=`date +"%Y-%m-%dT%T"`)

 

PS : Yes there are running queries and yes a lot of them.

 

Posts: 688
Topics: 1
Kudos: 158
Solutions: 83
Registered: ‎04-22-2014

Re: CM API: Maximum number of requests

@Prav,

 

I used the same one you used:

 

/api/v17/clusters/Cluster 1/services/IMPALA-1/impalaQueries?from=2018-08-01T17:04:32.776Z

 

works fine on my small test cluster

Posts: 688
Topics: 1
Kudos: 158
Solutions: 83
Registered: ‎04-22-2014

Re: CM API: Maximum number of requests

You may want to add DEBUG to Service Monitor and try the query again.  *maybe* some clues may come to light, but I hate to say I doubt it.

New Contributor
Posts: 9
Registered: ‎07-06-2018

Re: CM API: Maximum number of requests

@bgooley

 

Your query has "august" mentioned, could be a typo. But what I meant is when I specify "2018-07-09T17:04:32.776Z" in the from clause I still get query results (100 - default) but they still have this warning message at the end of the page.

warnings" : [ "Impala query scan limit reached. Last end time considered is 2018-07-09T17:04:32.776Z

 I was assuming this , If I capture the date in warning message at the end of each page and use it as a date field in the "from clause" the query should return the next result set of query and will have a date warning of some other time.date(future) which I can use again until I reach the current date/time at which point i'll stop the loop.

(Was fetching it via curl)

 

But that doesn't seem to be happening.

 

Other question I had what query string can I use to have to and from both in the same query string. I tried below but it didn't fetch any result:

 

https://hostname:7183/api/v17/clusters/cluster_name/services/impala/impalaQueries?filter=(from=2018-07-09 and to=`date +"%Y-%m-%dT%T"`)

 

 

 

Announcements