[mod] botdetection: trusted proxies

Replaces `x_for` functionality with `trusted_proxies`. This allows defining which IP / ranges to trust extracting the client IP address from X-Forwarded-For and X-Real-IP headers.

We don't know if the proxy chain will give us the proper client address, so we rely on reading the headers of the proxy before SearXNG (if there is one, in that case it must be added to trusted_proxies) hoping it has done the proper checks. In case a proxy in the chain does not check the client address correctly, integrity is compromised and this should be fixed by whoever manages the proxy, not us.

I had to move the get_cnf func to another file (config.py) to prevent cyclic imports since we need to read the list inside _helpers.py

Closes https://github.com/searxng/searxng/issues/4907
Closes https://github.com/searxng/searxng/issues/3632
Closes https://github.com/searxng/searxng/issues/3191
Closes https://github.com/searxng/searxng/issues/1237

Related https://github.com/searxng/searxng-docker/issues/386
Related https://github.com/inetol-infrastructure/searxng-container/issues/81

searx/botdetection/config.py

@@ -10,9 +10,10 @@ from __future__ import annotations
 from typing import Any
 
 import copy
-import typing
+import importlib
 import logging
 import pathlib
+import typing
 
 from ..compat import tomllib
 
@@ -20,6 +21,15 @@ __all__ = ['Config', 'UNSET', 'SchemaIssue']
 
 log = logging.getLogger(__name__)
 
+CFG: Config = None  # type: ignore
+
+LIMITER_CFG_SCHEMA = pathlib.Path(importlib.import_module("searx").__file__).parent / "limiter.toml"
+"""Base configuration (schema) of the botdetection."""
+
+CFG_DEPRECATED = {
+    # "dummy.old.foo": "config 'dummy.old.foo' exists only for tests.  Don't use it in your real project config."
+}
+
 
 class FALSE:
     """Class of ``False`` singleton"""
@@ -182,6 +192,17 @@ def toml_load(file_name):
         raise
 
 
+def get_cfg() -> Config:
+    global CFG  # pylint: disable=global-statement
+
+    if CFG is None:
+        from searx import settings_loader  # pylint: disable=import-outside-toplevel
+
+        cfg_file = (settings_loader.get_user_cfg_folder() or pathlib.Path("/etc/searxng")) / "limiter.toml"
+        CFG = Config.from_toml(LIMITER_CFG_SCHEMA, cfg_file, CFG_DEPRECATED)
+    return CFG
+
+
 # working with dictionaries
 
 
@@ -261,7 +282,6 @@ def _validate(
     data_dict: typing.Dict,
     deprecated: typing.Dict[str, str],
 ) -> typing.Tuple[bool, typing.List]:
-
     is_valid = True
 
     for key, data_value in data_dict.items():