mirror of
https://github.com/searxng/searxng.git
synced 2025-07-23 05:09:17 +02:00
[mod] migrate from Redis to Valkey
This patch migrates from `redis==5.2.1` [1] to `valkey==6.1.0` [2]. The migration to valkey is necessary because the company behind Redis has decided to abandon the open source license. After experiencing a drop in user numbers, they now want to run it under a dual license again. But this move demonstrates once again how unreliable the company is and how it treats open source developers. To review first, read the docs:: $ make docs.live Follow the instructions to remove redis: - http://0.0.0.0:8000/admin/settings/settings_redis.html Config and install a local valkey DB: - http://0.0.0.0:8000/admin/settings/settings_valkey.html [1] https://pypi.org/project/redis/ [2] https://pypi.org/project/valkey/ Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
parent
fe52290e65
commit
36538e6a20
43 changed files with 468 additions and 724 deletions
|
@ -7,7 +7,8 @@ digraph G {
|
|||
rp [label="reverse proxy"];
|
||||
static [label="static files", shape=folder, href="url to configure static files", fillcolor=lightgray];
|
||||
uwsgi [label="uwsgi", shape=parallelogram href="https://docs.searxng.org/utils/searxng.sh.html"]
|
||||
redis [label="redis DB", shape=cylinder];
|
||||
valkey [label="valkey DB", shape=cylinder];
|
||||
|
||||
searxng1 [label="SearXNG #1", fontcolor=blue3];
|
||||
searxng2 [label="SearXNG #2", fontcolor=blue3];
|
||||
searxng3 [label="SearXNG #3", fontcolor=blue3];
|
||||
|
@ -21,10 +22,10 @@ digraph G {
|
|||
{ rank=same; static rp };
|
||||
rp -> static [label="optional: reverse proxy serves static files", fillcolor=slategray, fontcolor=slategray];
|
||||
rp -> uwsgi [label="http:// (tcp) or unix:// (socket)"];
|
||||
uwsgi -> searxng1 -> redis;
|
||||
uwsgi -> searxng2 -> redis;
|
||||
uwsgi -> searxng3 -> redis;
|
||||
uwsgi -> searxng4 -> redis;
|
||||
uwsgi -> searxng1 -> valkey;
|
||||
uwsgi -> searxng2 -> valkey;
|
||||
uwsgi -> searxng3 -> valkey;
|
||||
uwsgi -> searxng4 -> valkey;
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
@ -235,19 +235,19 @@ major release is from Dec. 2013, since the there had been only bugfix releases
|
|||
**In Tyrant mode, there is no way to get additional groups, and the uWSGI
|
||||
process misses additional permissions that may be needed.**
|
||||
|
||||
For example on Fedora (RHEL): If you try to install a redis DB with socket
|
||||
For example on Fedora (RHEL): If you try to install a valkey DB with socket
|
||||
communication and you want to connect to it from the SearXNG uWSGI, you will see a
|
||||
*Permission denied* in the log of your instance::
|
||||
|
||||
ERROR:searx.redisdb: [searxng (993)] can't connect redis DB ...
|
||||
ERROR:searx.redisdb: Error 13 connecting to unix socket: /usr/local/searxng-redis/run/redis.sock. Permission denied.
|
||||
ERROR:searx.valkeydb: [searxng (993)] can't connect valkey DB ...
|
||||
ERROR:searx.valkeydb: Error 13 connecting to unix socket: /usr/local/searxng-valkey/run/valkey.sock. Permission denied.
|
||||
ERROR:searx.plugins.limiter: init limiter DB failed!!!
|
||||
|
||||
Even if your *searxng* user of the uWSGI process is added to additional groups
|
||||
to give access to the socket from the redis DB::
|
||||
to give access to the socket from the valkey DB::
|
||||
|
||||
$ groups searxng
|
||||
searxng : searxng searxng-redis
|
||||
searxng : searxng searxng-valkey
|
||||
|
||||
To see the effective groups of the uwsgi process, you have to look at the status
|
||||
of the process, by example::
|
||||
|
@ -257,7 +257,7 @@ of the process, by example::
|
|||
searxng 186 93 0 12:44 ? 00:00:01 /usr/sbin/uwsgi --ini searxng.ini
|
||||
|
||||
Here you can see that the additional "Groups" of PID 186 are unset (missing gid
|
||||
of ``searxng-redis``)::
|
||||
of ``searxng-valkey``)::
|
||||
|
||||
$ cat /proc/186/task/186/status
|
||||
...
|
||||
|
|
|
@ -6,7 +6,7 @@ Limiter
|
|||
|
||||
.. sidebar:: info
|
||||
|
||||
The limiter requires a :ref:`Redis <settings redis>` database.
|
||||
The limiter requires a :ref:`Valkey <settings valkey>` database.
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
|
|
|
@ -20,8 +20,7 @@ Settings
|
|||
settings_server
|
||||
settings_ui
|
||||
settings_redis
|
||||
settings_valkey
|
||||
settings_outgoing
|
||||
settings_categories_as_tabs
|
||||
settings_plugins
|
||||
|
||||
|
||||
|
|
|
@ -4,46 +4,34 @@
|
|||
``redis:``
|
||||
==========
|
||||
|
||||
.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
|
||||
.. _Valkey: https://valkey.io
|
||||
|
||||
A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
|
||||
will find a description to test your redis connection in SearXNG. When using
|
||||
sockets, don't forget to check the access rights on the socket::
|
||||
.. attention::
|
||||
|
||||
ls -la /usr/local/searxng-redis/run/redis.sock
|
||||
srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
|
||||
SearXNG is switching from the Redis DB to Valkey_. The configuration
|
||||
description of Valkey_ in SearXNG can be found here: :ref:`settings
|
||||
<settings valkey>`.
|
||||
|
||||
In this example read/write access is given to the *searxng-redis* group. To get
|
||||
access rights to redis instance (the socket), your SearXNG (or even your
|
||||
developer) account needs to be added to the *searxng-redis* group.
|
||||
|
||||
``url`` : ``$SEARXNG_REDIS_URL``
|
||||
URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
|
||||
|
||||
redis://[[username]:[password]]@localhost:6379/0
|
||||
rediss://[[username]:[password]]@localhost:6379/0
|
||||
unix://[[username]:[password]]@/path/to/socket.sock?db=0
|
||||
If you have built and installed a local Redis DB for SearXNG, it is recommended
|
||||
to uninstall it now and replace it with the installation of a Valkey_ DB.
|
||||
|
||||
.. _Redis Developer Notes:
|
||||
|
||||
Redis Developer Notes
|
||||
=====================
|
||||
|
||||
To set up a local redis instance, first set the socket path of the Redis DB
|
||||
in your YAML setting:
|
||||
To uninstall SearXNG's local Redis DB you can use:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
# stop your SearXNG instance
|
||||
$ ./utils/searxng.sh remove.redis
|
||||
|
||||
Remove the Redis DB in your YAML setting:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
redis:
|
||||
url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
|
||||
|
||||
Then use the following commands to install the redis instance (:ref:`manage
|
||||
redis.help`):
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ ./manage redis.build
|
||||
$ sudo -H ./manage redis.install
|
||||
$ sudo -H ./manage redis.addgrp "${USER}"
|
||||
# don't forget to logout & login to get member of group
|
||||
|
||||
To install Valkey_ read: :ref:`Valkey Developer Notes`
|
||||
|
|
|
@ -36,7 +36,7 @@
|
|||
|
||||
``limiter`` : ``$SEARXNG_LIMITER``
|
||||
Rate limit the number of request on the instance, block some bots. The
|
||||
:ref:`limiter` requires a :ref:`settings redis` database.
|
||||
:ref:`limiter` requires a :ref:`settings valkey` database.
|
||||
|
||||
.. _public_instance:
|
||||
|
||||
|
|
53
docs/admin/settings/settings_valkey.rst
Normal file
53
docs/admin/settings/settings_valkey.rst
Normal file
|
@ -0,0 +1,53 @@
|
|||
.. _settings valkey:
|
||||
|
||||
===========
|
||||
``valkey:``
|
||||
===========
|
||||
|
||||
.. _Valkey:
|
||||
https://valkey.io
|
||||
.. _Valkey-Installation:
|
||||
https://valkey.io/topics/installation/
|
||||
.. _There are several ways to specify a database number:
|
||||
https://valkey-py.readthedocs.io/en/stable/connections.html#valkey.Valkey.from_url
|
||||
|
||||
A Valkey_ DB can be connected by an URL, in section :ref:`valkey db` you will
|
||||
find a description to test your valkey connection in SearXNG.
|
||||
|
||||
``url`` : ``$SEARXNG_VALKEY_URL``
|
||||
URL to connect valkey database. `There are several ways to specify a database
|
||||
number`_::
|
||||
|
||||
valkey://[[username]:[password]]@localhost:6379/0
|
||||
valkeys://[[username]:[password]]@localhost:6379/0
|
||||
unix://[[username]:[password]]@/path/to/socket.sock?db=0
|
||||
|
||||
When using sockets, don't forget to check the access rights on the socket::
|
||||
|
||||
ls -la /usr/local/searxng-valkey/run/valkey.sock
|
||||
srwxrwx--- 1 searxng-valkey searxng-valkey ... /usr/local/searxng-valkey/run/valkey.sock
|
||||
|
||||
In this example read/write access is given to the *searxng-valkey* group. To
|
||||
get access rights to valkey instance (the socket), your SearXNG (or even your
|
||||
developer) account needs to be added to the *searxng-valkey* group.
|
||||
|
||||
|
||||
.. _Valkey Developer Notes:
|
||||
|
||||
Valkey Developer Notes
|
||||
======================
|
||||
|
||||
To set up a local Valkey_ DB, set the URL connector in your YAML setting:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
valkey:
|
||||
url: valkey://localhost:6379/0
|
||||
|
||||
To install a local Valkey_ DB from package manager read `Valkey-Installation`_
|
||||
or use:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ ./utils/searxng.sh install valkey
|
||||
# restart your SearXNG instance
|
|
@ -56,7 +56,7 @@ SearXNG is growing rapidly, the services and opportunities are change every now
|
|||
and then, to name just a few:
|
||||
|
||||
- Bot protection has been switched from filtron to SearXNG's :ref:`limiter
|
||||
<limiter>`, this requires a :ref:`Redis <settings redis>` database.
|
||||
<limiter>`, this requires a :ref:`Valkey <settings valkey>` database.
|
||||
|
||||
- To save bandwidth :ref:`cache busting <static_use_hash>` has been implemented.
|
||||
To get in use, the ``static-expires`` needs to be set in the :ref:`uwsgi
|
||||
|
@ -89,5 +89,5 @@ to see if there are some left overs. In this example there exists a *old*
|
|||
--------------
|
||||
ERROR: settings.yml in /etc/searx/ is deprecated, move file to folder /etc/searxng/
|
||||
...
|
||||
INFO searx.redisdb : connecting to Redis db=0 path='/usr/local/searxng-redis/run/redis.sock'
|
||||
INFO searx.redisdb : connected to Redis
|
||||
INFO searx.valkeydb : connecting to Valkey db=0 path='/usr/local/searxng-valkey/run/valkey.sock'
|
||||
INFO searx.valkeydb : connected to Valkey
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue