[enh] Add timeout limit per request (#1640)

The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.

In addition, the query can start with <[number] to set the timeout limit.

For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds

For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.

In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).

If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.

Related to #1077
Updated version of PR #1413 from @isj-privacore
This commit is contained in:
Alexandre Flament 2019-08-02 13:50:51 +02:00 committed by GitHub
parent 2179079a91
commit 72029d27de
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
10 changed files with 180 additions and 15 deletions

View file

@ -43,6 +43,7 @@ class RawTextQuery(object):
self.query_parts = []
self.engines = []
self.languages = []
self.timeout_limit = None
self.specific = False
# parse query, if tags are set, which
@ -69,6 +70,21 @@ class RawTextQuery(object):
self.query_parts.append(query_part)
continue
# this force the timeout
if query_part[0] == '<':
try:
raw_timeout_limit = int(query_part[1:])
if raw_timeout_limit < 100:
# below 100, the unit is the second ( <3 = 3 seconds timeout )
self.timeout_limit = float(raw_timeout_limit)
else:
# 100 or above, the unit is the millisecond ( <850 = 850 milliseconds timeout )
self.timeout_limit = raw_timeout_limit / 1000.0
parse_next = True
except ValueError:
# error not reported to the user
pass
# this force a language
if query_part[0] == ':':
lang = query_part[1:].lower().replace('_', '-')
@ -161,7 +177,7 @@ class RawTextQuery(object):
class SearchQuery(object):
"""container for all the search parameters (query, language, etc...)"""
def __init__(self, query, engines, categories, lang, safesearch, pageno, time_range):
def __init__(self, query, engines, categories, lang, safesearch, pageno, time_range, timeout_limit=None):
self.query = query.encode('utf-8')
self.engines = engines
self.categories = categories
@ -169,6 +185,7 @@ class SearchQuery(object):
self.safesearch = safesearch
self.pageno = pageno
self.time_range = time_range
self.timeout_limit = timeout_limit
def __str__(self):
return str(self.query) + ";" + str(self.engines)