searxng/searx
Markus Heiser bdfe1c2a15 [mod] engines: migration of the individual cache solutions to EngineCache
The EngineCache class replaces all previously individual solutions for caches in
the context of the engines.

- demo_offline.py
- duckduckgo.py
- radio_browser.py
- soundcloud.py
- startpage.py
- wolframalpha_api.py
- wolframalpha_noapi.py

Search term to test most of the modified engines::

    !ddg !rb !sc !sp !wa test

    !ddg !rb !sc !sp !wa foo

For introspection of the DB, jump into developer environment and run command to
show cache state::

    $ ./manage pyenv.cmd bash --norc --noprofile
    (py3) python -m searx.enginelib cache state

    cache tables and key/values
    ===========================
    [demo_offline        ] 2025-04-22 11:32:50 count        --> (int) 4
    [startpage           ] 2025-04-22 12:32:30 SC_CODE      --> (str) fSOBnhEMlDfE20
    [duckduckgo          ] 2025-04-22 12:32:31 4dff493e.... --> (str) 4-128634958369380006627592672385352473325
    [duckduckgo          ] 2025-04-22 12:40:06 3e2583e2.... --> (str) 4-263126175288871260472289814259666848451
    [radio_browser       ] 2025-04-23 11:33:08 servers      --> (list) ['https://de2.api.radio-browser.info',  ...]
    [soundcloud          ] 2025-04-29 11:40:06 guest_client_id --> (str) EjkRJG0BLNEZquRiPZYdNtJdyGtTuHdp
    [wolframalpha        ] 2025-04-22 12:40:06 code         --> (str) 5aa79f86205ad26188e0e26e28fb7ae7
    number of tables: 6
    number of key/value pairs: 7

In the "cache tables and key/values" section, the table name (engine name) is at
first position on the second there is the calculated expire date and on the
third and fourth position the key/value is shown.

About duckduckgo: The *vqd coode* of ddg depends on the query term and therefore
the key is a hash value of the query term (to not to store the raw query term).

In the "properties of ENGINES_CACHE" section all properties of the SQLiteAppl /
ExpireCache and their last modification date are shown::

    properties of ENGINES_CACHE
    ===========================
    [last modified: 2025-04-22 11:32:27] DB_SCHEMA           : 1
    [last modified: 2025-04-22 11:32:27] LAST_MAINTENANCE    :
    [last modified: 2025-04-22 11:32:27] crypt_hash          : ca612e3566fdfd7cf7efe2b1c9349f461158d07cb78a3750e5c5be686aa8ebdc
    [last modified: 2025-04-22 11:32:30] CACHE-TABLE--demo_offline: demo_offline
    [last modified: 2025-04-22 11:32:30] CACHE-TABLE--startpage: startpage
    [last modified: 2025-04-22 11:32:31] CACHE-TABLE--duckduckgo: duckduckgo
    [last modified: 2025-04-22 11:33:08] CACHE-TABLE--radio_browser: radio_browser
    [last modified: 2025-04-22 11:40:06] CACHE-TABLE--soundcloud: soundcloud
    [last modified: 2025-04-22 11:40:06] CACHE-TABLE--wolframalpha: wolframalpha

These properties provide information about the state of the ExpireCache and
control the behavior.  For example, the maintenance intervals are controlled by
the last modification date of the LAST_MAINTENANCE property and the hash value
of the password can be used to detect whether the password has been changed (in
this case the DB entries can no longer be decrypted and the entire cache must be
discarded).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-05-03 08:39:12 +02:00
..
answerers [fix] gettext can't work with f-strings (i10n) 2025-01-29 06:04:09 +01:00
botdetection [fix] plugins: bugfix of tor_check and unit_converter 2025-03-15 10:36:33 +01:00
data [data] update searx.data - update_engine_descriptions.py 2025-05-02 16:45:47 +02:00
enginelib [mod] ExpireCache - sqlite based key/value cache with expire time 2025-05-03 08:39:12 +02:00
engines [mod] engines: migration of the individual cache solutions to EngineCache 2025-05-03 08:39:12 +02:00
favicons [fix] ResourceWarning: unclosed database in sqlite3 2025-05-03 08:39:12 +02:00
infopage [refactor] typification of SearXNG (initial) / result items (part 1) 2025-01-28 07:07:08 +01:00
metrics [fix] openmetrics: value is None if there's no data 2024-11-25 16:57:03 +01:00
network [fix] issues when launching a local development server 2025-04-22 14:26:11 +02:00
plugins [refactor] migrate plugins from "module" to class SXNGPlugin 2025-03-29 10:16:43 +01:00
result_types [fix] Result type: remove rstrip "/" form url normalization 2025-04-17 10:24:05 +02:00
search [fix] issues when launching a local development server 2025-04-22 14:26:11 +02:00
static/themes/simple [build] /static 2025-04-06 19:21:07 +02:00
templates/simple [mod] migrate all key-value.html templates to KeyValue type 2025-03-15 10:36:33 +01:00
translations [l10n] update translations from Weblate 2025-05-02 13:29:05 +02:00
__init__.py [fix] issues when launching a local development server 2025-04-22 14:26:11 +02:00
autocomplete.py [feat] add Quark autocomplete support 2025-03-31 14:59:49 +02:00
babel_extract.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
cache.py [mod] ExpireCache - sqlite based key/value cache with expire time 2025-05-03 08:39:12 +02:00
compat.py [mod] Revision of the favicon solution 2024-10-05 08:18:28 +02:00
exceptions.py [fix] engine: duckduckgo - CAPTCHA detection 2024-10-19 14:55:44 +02:00
extended_types.py [refactor] typification of SearXNG (initial) / result items (part 1) 2025-01-28 07:07:08 +01:00
external_bang.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
external_urls.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
flaskfix.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
limiter.py [refactor] typification of SearXNG (initial) / result items (part 1) 2025-01-28 07:07:08 +01:00
limiter.toml [chore] *: fix typos detected by typos-cli 2024-11-24 12:41:57 +01:00
locales.py [refactor] typification of SearXNG (initial) / result items (part 1) 2025-01-28 07:07:08 +01:00
openmetrics.py [fix] openmetrics: value is None if there's no data 2024-11-25 16:57:03 +01:00
preferences.py [refactor] migrate plugins from "module" to class SXNGPlugin 2025-03-29 10:16:43 +01:00
query.py [mod] internal ! and external !! bangs: ignore upper/lower case 2025-04-24 10:40:18 +02:00
redisdb.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
redislib.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
results.py [refactor] typification of SearXNG (MainResult) / result items (part 2) 2025-03-15 10:36:33 +01:00
searxng.msg Revert "[l10n ga] trigger weblate.push.translations (another try)" 2025-01-06 17:12:16 +01:00
settings.yml [feat] engine ChinaSo: support source filter for ChinaSo-News 2025-05-02 14:22:51 +02:00
settings_defaults.py [mod] include SEARXNG_METHOD environment variable 2025-04-23 07:29:26 +02:00
settings_loader.py [mod] revision of the settings_loader 2024-07-14 18:10:06 +02:00
sqlitedb.py [fix] ResourceWarning: unclosed database in sqlite3 2025-05-03 08:39:12 +02:00
sxng_locales.py [data] update searx.data - make data.traits (mullvad leta) 2025-04-23 14:06:32 +02:00
unixthreadname.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
utils.py [fix] hardening against arguments of type None, where str or dict is expected 2025-04-01 11:13:47 +02:00
version.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
webadapter.py [chore] *: fix typos detected by typos-cli 2024-11-24 12:41:57 +01:00
webapp.py [fix] is_werkzeug_reload_active is not realted to python -m 2025-04-24 17:24:51 +02:00
webutils.py [fix] issues when launching a local development server 2025-04-22 14:26:11 +02:00