Icycoide/searxng
1
0
Fork 1
mirror of https://github.com/searxng/searxng.git synced 2025-07-23 05:09:17 +02:00
Code Issues Projects Releases Packages Wiki Activity

[mod] migrate from Redis to Valkey

Browse source 
This patch migrates from `redis==5.2.1` [1] to `valkey==6.1.0` [2].

The migration to valkey is necessary because the company behind Redis has decided
to abandon the open source license. After experiencing a drop in user numbers,
they now want to run it under a dual license again. But this move demonstrates
once again how unreliable the company is and how it treats open source
developers.

To review first, read the docs::

    $ make docs.live

Follow the instructions to remove redis:

- http://0.0.0.0:8000/admin/settings/settings_redis.html

Config and install a local valkey DB:

- http://0.0.0.0:8000/admin/settings/settings_valkey.html

[1] https://pypi.org/project/redis/
[2] https://pypi.org/project/valkey/

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
HLFH 2025-05-16 15:20:09 +02:00 • committed by Markus Heiser
parent fe52290e65
commit 36538e6a20
43 changed files with 468 additions and 724 deletions
Download patch file Download diff file Expand all files Collapse all files

11
docs/admin/arch_public.dot
View file

@ -7,7 +7,8 @@ digraph G {
rp [label="reverse proxy"];
static [label="static files", shape=folder, href="url to configure static files", fillcolor=lightgray];
uwsgi [label="uwsgi", shape=parallelogram href="https://docs.searxng.org/utils/searxng.sh.html"]
redis [label="redis DB", shape=cylinder];
valkey [label="valkey DB", shape=cylinder];
searxng1 [label="SearXNG #1", fontcolor=blue3];
searxng2 [label="SearXNG #2", fontcolor=blue3];
searxng3 [label="SearXNG #3", fontcolor=blue3];
@ -21,10 +22,10 @@ digraph G {
{ rank=same; static rp };
rp -> static [label="optional: reverse proxy serves static files", fillcolor=slategray, fontcolor=slategray];
rp -> uwsgi [label="http:// (tcp) or unix:// (socket)"];
uwsgi -> searxng1 -> redis;
uwsgi -> searxng2 -> redis;
uwsgi -> searxng3 -> redis;
uwsgi -> searxng4 -> redis;
uwsgi -> searxng1 -> valkey;
uwsgi -> searxng2 -> valkey;
uwsgi -> searxng3 -> valkey;
uwsgi -> searxng4 -> valkey;
}
}

12
docs/admin/installation-uwsgi.rst
View file

@ -235,19 +235,19 @@ major release is from Dec. 2013, since the there had been only bugfix releases
**In Tyrant mode, there is no way to get additional groups, and the uWSGI
process misses additional permissions that may be needed.**
For example on Fedora (RHEL): If you try to install a redis DB with socket
For example on Fedora (RHEL): If you try to install a valkey DB with socket
communication and you want to connect to it from the SearXNG uWSGI, you will see a
*Permission denied* in the log of your instance::
ERROR:searx.redisdb: [searxng (993)] can't connect redis DB ...
ERROR:searx.redisdb: Error 13 connecting to unix socket: /usr/local/searxng-redis/run/redis.sock. Permission denied.
ERROR:searx.valkeydb: [searxng (993)] can't connect valkey DB ...
ERROR:searx.valkeydb: Error 13 connecting to unix socket: /usr/local/searxng-valkey/run/valkey.sock. Permission denied.
ERROR:searx.plugins.limiter: init limiter DB failed!!!
Even if your *searxng* user of the uWSGI process is added to additional groups
to give access to the socket from the redis DB::
to give access to the socket from the valkey DB::
$ groups searxng
searxng : searxng searxng-redis
searxng : searxng searxng-valkey
To see the effective groups of the uwsgi process, you have to look at the status
of the process, by example::
@ -257,7 +257,7 @@ of the process, by example::
searxng 186 93 0 12:44 ? 00:00:01 /usr/sbin/uwsgi --ini searxng.ini
Here you can see that the additional "Groups" of PID 186 are unset (missing gid
of ``searxng-redis``)::
of ``searxng-valkey``)::
$ cat /proc/186/task/186/status
...

2
docs/admin/searx.limiter.rst
View file

@ -6,7 +6,7 @@ Limiter
.. sidebar:: info
The limiter requires a :ref:`Redis <settings redis>` database.
The limiter requires a :ref:`Valkey <settings valkey>` database.
.. contents::
:depth: 2

3
docs/admin/settings/index.rst
View file

@ -20,8 +20,7 @@ Settings
settings_server
settings_ui
settings_redis
settings_valkey
settings_outgoing
settings_categories_as_tabs
settings_plugins

44
docs/admin/settings/settings_redis.rst
View file

@ -4,46 +4,34 @@
``redis:``
==========
.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
.. _Valkey: https://valkey.io
A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
will find a description to test your redis connection in SearXNG. When using
sockets, don't forget to check the access rights on the socket::
.. attention::
ls -la /usr/local/searxng-redis/run/redis.sock
srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
SearXNG is switching from the Redis DB to Valkey_. The configuration
description of Valkey_ in SearXNG can be found here: :ref:`settings
<settings valkey>`.
In this example read/write access is given to the *searxng-redis* group. To get
access rights to redis instance (the socket), your SearXNG (or even your
developer) account needs to be added to the *searxng-redis* group.
``url`` : ``$SEARXNG_REDIS_URL``
URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
redis://[[username]:[password]]@localhost:6379/0
rediss://[[username]:[password]]@localhost:6379/0
unix://[[username]:[password]]@/path/to/socket.sock?db=0
If you have built and installed a local Redis DB for SearXNG, it is recommended
to uninstall it now and replace it with the installation of a Valkey_ DB.
.. _Redis Developer Notes:
Redis Developer Notes
=====================
To set up a local redis instance, first set the socket path of the Redis DB
in your YAML setting:
To uninstall SearXNG's local Redis DB you can use:
.. code:: sh
# stop your SearXNG instance
$ ./utils/searxng.sh remove.redis
Remove the Redis DB in your YAML setting:
.. code:: yaml
redis:
url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
Then use the following commands to install the redis instance (:ref:`manage
redis.help`):
.. code:: sh
$ ./manage redis.build
$ sudo -H ./manage redis.install
$ sudo -H ./manage redis.addgrp "${USER}"
# don't forget to logout & login to get member of group
To install Valkey_ read: :ref:`Valkey Developer Notes`

2
docs/admin/settings/settings_server.rst
View file

@ -36,7 +36,7 @@
``limiter`` : ``$SEARXNG_LIMITER``
Rate limit the number of request on the instance, block some bots. The
:ref:`limiter` requires a :ref:`settings redis` database.
:ref:`limiter` requires a :ref:`settings valkey` database.
.. _public_instance:

53
docs/admin/settings/settings_valkey.rst Normal file
View file

@ -0,0 +1,53 @@
.. _settings valkey:
===========
``valkey:``
===========
.. _Valkey:
https://valkey.io
.. _Valkey-Installation:
https://valkey.io/topics/installation/
.. _There are several ways to specify a database number:
https://valkey-py.readthedocs.io/en/stable/connections.html#valkey.Valkey.from_url
A Valkey_ DB can be connected by an URL, in section :ref:`valkey db` you will
find a description to test your valkey connection in SearXNG.
``url`` : ``$SEARXNG_VALKEY_URL``
URL to connect valkey database. `There are several ways to specify a database
number`_::
valkey://[[username]:[password]]@localhost:6379/0
valkeys://[[username]:[password]]@localhost:6379/0
unix://[[username]:[password]]@/path/to/socket.sock?db=0
When using sockets, don't forget to check the access rights on the socket::
ls -la /usr/local/searxng-valkey/run/valkey.sock
srwxrwx--- 1 searxng-valkey searxng-valkey ... /usr/local/searxng-valkey/run/valkey.sock
In this example read/write access is given to the *searxng-valkey* group. To
get access rights to valkey instance (the socket), your SearXNG (or even your
developer) account needs to be added to the *searxng-valkey* group.
.. _Valkey Developer Notes:
Valkey Developer Notes
======================
To set up a local Valkey_ DB, set the URL connector in your YAML setting:
.. code:: yaml
valkey:
url: valkey://localhost:6379/0
To install a local Valkey_ DB from package manager read `Valkey-Installation`_
or use:
.. code:: sh
$ ./utils/searxng.sh install valkey
# restart your SearXNG instance

6
docs/admin/update-searxng.rst
View file

@ -56,7 +56,7 @@ SearXNG is growing rapidly, the services and opportunities are change every now
and then, to name just a few:
- Bot protection has been switched from filtron to SearXNG's :ref:`limiter
<limiter>`, this requires a :ref:`Redis <settings redis>` database.
<limiter>`, this requires a :ref:`Valkey <settings valkey>` database.
- To save bandwidth :ref:`cache busting <static_use_hash>` has been implemented.
To get in use, the ``static-expires`` needs to be set in the :ref:`uwsgi
@ -89,5 +89,5 @@ to see if there are some left overs. In this example there exists a *old*
--------------
ERROR: settings.yml in /etc/searx/ is deprecated, move file to folder /etc/searxng/
...
INFO searx.redisdb : connecting to Redis db=0 path='/usr/local/searxng-redis/run/redis.sock'
INFO searx.redisdb : connected to Redis
INFO searx.valkeydb : connecting to Valkey db=0 path='/usr/local/searxng-valkey/run/valkey.sock'
INFO searx.valkeydb : connected to Valkey

2
docs/conf.py
View file

@ -149,7 +149,7 @@ intersphinx_mapping = {
"jinja": ("https://jinja.palletsprojects.com/en/stable/", None),
"linuxdoc" : ("https://return42.github.io/linuxdoc/", None),
"sphinx" : ("https://www.sphinx-doc.org/en/master/", None),
"redis": ('https://redis.readthedocs.io/en/stable/', None),
"valkey": ('https://valkey-py.readthedocs.io/en/stable/', None),
}
issues_github_path = "searxng/searxng"

23
docs/dev/engines/offline/nosql-engines.rst
View file

@ -7,7 +7,7 @@ NoSQL databases
.. sidebar:: further read
- `NoSQL databases <https://en.wikipedia.org/wiki/NoSQL>`_
- `redis.io <https://redis.io/>`_
- `valkey.io <https://valkey.io/>`_
- `MongoDB <https://www.mongodb.com>`_
.. contents::
@ -22,7 +22,7 @@ NoSQL databases
The following `NoSQL databases`_ are supported:
- :ref:`engine redis_server`
- :ref:`engine valkey_server`
- :ref:`engine mongodb`
All of the engines above are just commented out in the :origin:`settings.yml
@ -45,7 +45,7 @@ section :ref:`private engines`.
Extra Dependencies
==================
For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
For using :ref:`engine valkey_server` or :ref:`engine mongodb` you need to
install additional packages in Python's Virtual Environment of your SearXNG
instance. To switch into the environment (:ref:`searxng-src`) you can use
:ref:`searxng.sh`::
@ -61,20 +61,20 @@ Configure the engines
their structure.
.. _engine redis_server:
.. _engine valkey_server:
Redis Server
------------
Valkey Server
-------------
.. _redis: https://github.com/andymccurdy/redis-py#installation
.. _valkey: https://github.com/andymccurdy/valkey-py#installation
.. sidebar:: info
- ``pip install`` redis_
- redis.io_
- :origin:`redis_server.py <searx/engines/redis_server.py>`
- ``pip install`` valkey_
- valkey.io_
- :origin:`valkey_server.py <searx/engines/valkey_server.py>`
.. automodule:: searx.engines.redis_server
.. automodule:: searx.engines.valkey_server
:members:
@ -94,4 +94,3 @@ MongoDB
.. automodule:: searx.engines.mongodb
:members:

2
docs/dev/lxcdev.rst
View file

@ -140,7 +140,7 @@ proxy :ref:`installation nginx` into the archlinux container run:
.. sidebar:: Fully functional SearXNG suite
From here on you have a fully functional SearXNG suite (including a
:ref:`redis db`).
:ref:`valkey db`).
In such a SearXNG suite admins can maintain and access the debug log of the
services quite easy.

11
docs/dev/makefile.rst
View file

@ -362,17 +362,6 @@ can be used to convenient run common build tasks of the static files.
.. program-output:: bash -c "cd ..; ./manage static.help"
.. _manage redis.help:
``./manage redis.help``
=======================
The ``./manage redis.*`` command line can be used to convenient run common Redis
tasks (:ref:`Redis developer notes`).
.. program-output:: bash -c "cd ..; ./manage redis.help"
.. _manage go.help:
``./manage go.help``

8
docs/src/searx.redisdb.rst
View file

@ -1,8 +0,0 @@
.. _redis db:
========
Redis DB
========
.. automodule:: searx.redisdb
:members:

8
docs/src/searx.redislib.rst
View file

@ -1,8 +0,0 @@
.. _searx.redis:
=============
Redis Library
=============
.. automodule:: searx.redislib
:members:

8
docs/src/searx.valkeydb.rst Normal file
View file

@ -0,0 +1,8 @@
.. _valkey db:
==========
Valkey DB
==========
.. automodule:: searx.valkeydb
:members:

8
docs/src/searx.valkeylib.rst Normal file
View file

@ -0,0 +1,8 @@
.. _searx.valkey:
==============
Valkey Library
==============
.. automodule:: searx.valkeylib
:members: