Implement rules and functions to format shell scripts:
$ make format.shell
or alternatively to format all source code:
$ make format
The formatter `shfmt` reads the rules from the editorconfig[1]
If any EditorConfig files are found, they will be used to apply formatting
options. If any parser or printer flags are given to the tool, no
EditorConfig files will be used.
[1] https://github.com/patrickvane/shfmt?tab=readme-ov-file#description
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In customizing it should be decided which plugin modules should be loaded and
which should not.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch migrates from `redis==5.2.1` [1] to `valkey==6.1.0` [2].
The migration to valkey is necessary because the company behind Redis has decided
to abandon the open source license. After experiencing a drop in user numbers,
they now want to run it under a dual license again. But this move demonstrates
once again how unreliable the company is and how it treats open source
developers.
To review first, read the docs::
$ make docs.live
Follow the instructions to remove redis:
- http://0.0.0.0:8000/admin/settings/settings_redis.html
Config and install a local valkey DB:
- http://0.0.0.0:8000/admin/settings/settings_valkey.html
[1] https://pypi.org/project/redis/
[2] https://pypi.org/project/valkey/
Co-authored-by: HLFH <gaspard@dhautefeuille.eu>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
This is one of various PR to refactor the simple theme internally.
Replace eslint tool with Biome. I have been using this for quite some time, and
it will help us to have more consistent and valid code without extending on
other third party plugins.
Removes unused dependencies.
Tube Archivist [1] is a self-hosted project which archives youtube videos on
your own local server. This engine connects with Tube Archivist's search API to
allow searching from SearXNG into your own hosted videos.
[1] https://www.tubearchivist.com/
Signed-off-by: Robert M. Clabough <robert@claobugh.tech>
Co-authored-by: Bnyro <bnyro@tutanota.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
The types necessary for weather information such as GeoLocation, DateTime,
Temperature,Pressure, WindSpeed, RelativeHumidity, Compass (wind direction) and
symbols for the weather have been implemented.
There are unit conversions and translations for weather property labels.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
That entrypoint is prone to screw things up, especially with permission handling. The new script handles initialization better and fixes some issues like delayed settings update via ENVs and timestamp overwriting, also adjusts what should be copied into the container.
Related https://github.com/searxng/searxng/pull/4721#issuecomment-2850272129
Chapter on the purpose of (git) commits
The commits and their messages are elementary for the traceability of changes
and are unfortunately still too often given too little attention.
It therefore seems necessary to dedicate a chapter to this topic in the context
of development.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
container.yml will run after integration.yml COMPLETES successfully and in master branch.
Style changes, cleanup and improved integration with CI by leveraging the use of
shared cache between all workflows.
* Podman is now supported to build the container images (Docker also received a refactor, merging both build and buildx)
* Container images are being built by Buildah instead of Docker BuildKit.
* Container images are tested before release.
* Splitting "modern" (amd64 & arm64) and "legacy" (armv7) arches on different Dockerfiles allowing future optimizations.
The language settings page in the documentation reference languages.py which was
removed in commit c9cd376
Closes: https://github.com/searxng/searxng/issues/4690
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* filtering ChinaSo-News results by source, option ``chinaso_news_source``
* add ChinaSo engine to the online docs https://docs.searxng.org/dev/engines/online/chinaso.html
* fix SearXNG categories in the settings.yml
* deactivate ChinaSo engines ``inactive: true`` until [1] is fixed
* configure network of the ChinaSo engines
[1] https://github.com/searxng/searxng/issues/4694
Signed-off-by: @BrandonStudio
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Re-writes the Mullvad Leta integration to work with the new breaking changes.
Mullvad Leta is a search engine proxy. Currently Leta only offers text search
results not image, news or any other types of search result. Leta acts as a
proxy to Google and Brave search results.
- Remove docstring comments regarding requiring the use of Mullvad VPN, which is
no longer a hard requirement.
- configured two engines: ``mullvadleta`` (uses google) and
``mullvadleta brave`` (uses brave)
- since leta may not provide up-to-date search results, both search engines are
disabled by default.
.. hint::
Leta caches each search for up to 30 days. For example, if you use search
terms like ``news``, contrary to your intention you'll get very old results!
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Grant Lanham <contact@grantlanham.com>
A local development server can be launched by one of these command lines::
$ flask --app searx.webapp run
$ python -m searx.webapp
The different ways of starting the server should lead to the same result, which
is generally the case. However, if the modules are reloaded after code
changes (reload option), it must be avoided that the application is initialized
twice at startup. We have already discussed this in 2022 [1][2].
Further information on this topic can be found in [3][4][5].
To test a bash in the ./local environment was started and the follwing commands
had been executed::
$ ./manage pyenv.cmd bash --norc --noprofile
(py3) SEARXNG_DEBUG=1 flask --app searx.webapp run --reload
(py3) SEARXNG_DEBUG=1 python -m searx.webapp
Since the generic parts of the docs also initialize the app to generate doc from
it, the build of the docs was also tested::
$ make docs.clean docs.live
[1] https://github.com/searxng/searxng/pull/1656#issuecomment-1214198941
[2] https://github.com/searxng/searxng/pull/1616#issuecomment-1206137468
[3] https://flask.palletsprojects.com/en/stable/api/#flask.Flask.run
[4] https://github.com/pallets/flask/issues/5307#issuecomment-1774646119
[5] https://stackoverflow.com/a/25504196
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
BTW: fix a bug with sys.path: repo-root (not util) needs to added to generate
autodoc from scripts in ./searxng_extra
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch brings two major changes:
- ``Result.filter_urls(..)`` to pass a filter function for URL fields
- The ``enabled_plugins:`` section in SearXNG's settings do no longer exists.
To understand plugin development compile documentation:
$ make docs.clean docs.live
and read http://0.0.0.0:8000/dev/plugins/development.html
There is no longer a distinction between built-in and external plugin, all
plugins are registered via the settings in the ``plugins:`` section.
In SearXNG, plugins can be registered via a fully qualified class name. A
configuration (`PluginCfg`) can be transferred to the plugin, e.g. to activate
it by default / *opt-in* or *opt-out* from user's point of view.
built-in plugins
================
The built-in plugins are all located in the namespace `searx.plugins`.
.. code:: yaml
plugins:
searx.plugins.calculator.SXNGPlugin:
active: true
searx.plugins.hash_plugin.SXNGPlugin:
active: true
searx.plugins.self_info.SXNGPlugin:
active: true
searx.plugins.tracker_url_remover.SXNGPlugin:
active: true
searx.plugins.unit_converter.SXNGPlugin:
active: true
searx.plugins.ahmia_filter.SXNGPlugin:
active: true
searx.plugins.hostnames.SXNGPlugin:
active: true
searx.plugins.oa_doi_rewrite.SXNGPlugin:
active: false
searx.plugins.tor_check.SXNGPlugin:
active: false
external plugins
================
SearXNG supports *external plugins* / there is no need to install one, SearXNG
runs out of the box.
- Only show green hosted results: https://github.com/return42/tgwf-searx-plugins/
To get a developer installation in a SearXNG developer environment:
.. code:: sh
$ git clone git@github.com:return42/tgwf-searx-plugins.git
$ ./manage pyenv.cmd python -m \
pip install -e tgwf-searx-plugins
To register the plugin in SearXNG add ``only_show_green_results.SXNGPlugin`` to
the ``plugins:``:
.. code:: yaml
plugins:
# ...
only_show_green_results.SXNGPlugin:
active: false
Result.filter_urls(..)
======================
The ``Result.filter_urls(..)`` can be used to filter and/or modify URL fields.
In the following example, the filter function ``my_url_filter``:
.. code:: python
def my_url_filter(result, field_name, url_src) -> bool | str:
if "google" in url_src:
return False # remove URL field from result
if "facebook" in url_src:
new_url = url_src.replace("facebook", "fb-dummy")
return new_url # return modified URL
return True # leave URL in field unchanged
is applied to all URL fields in the :py:obj:`Plugin.on_result` hook:
.. code:: python
class MyUrlFilter(Plugin):
...
def on_result(self, request, search, result) -> bool:
result.filter_urls(my_url_filter)
return True
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch adds a new result type: KeyValue
- Python class: searx/result_types/keyvalue.py
- Jinja template: searx/templates/simple/result_templates/keyvalue.html
- CSS (less) client/simple/src/less/result_types/keyvalue.less
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The class ReslutContainer has been revised, it can now handle the typed Result
items of classes:
- MainResult
- LegacyResult (a dict wrapper for backward compatibility)
Due to the now complete typing of theses three clases, instead of the *getitem*
accesses, the fields can now be accessed directly via attributes (which is also
supported by the IDE).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
as long we don't have a solution for #4334 we can't offer startpage as
autocomplete backend.
Related:
- https://github.com/searxng/searxng/issues/4334
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
