[doc] build from commit f32e91e51a

This commit is contained in:
return42 2025-07-28 13:38:00 +00:00
commit 2aed3e5c5c
425 changed files with 96733 additions and 0 deletions

View file

@ -0,0 +1,11 @@
.. _builtin answerers:
==================
Built-in Answerers
==================
.. toctree::
:maxdepth: 1
random
statistics

View file

@ -0,0 +1,7 @@
.. _dev answerers:
====================
Answerer Development
====================
.. automodule:: searx.answerers

View file

@ -0,0 +1,9 @@
=========
Answerers
=========
.. toctree::
:maxdepth: 2
development
builtins

View file

@ -0,0 +1,8 @@
.. _answerer.random:
======
Random
======
.. autoclass:: searx.answerers.random.SXNGAnswerer
:members:

View file

@ -0,0 +1,8 @@
.. _answerer.statistics:
==========
Statistics
==========
.. autoclass:: searx.answerers.statistics.SXNGAnswerer
:members:

View file

@ -0,0 +1,108 @@
.. _create commit:
===============================
Git Commits & Change Management
===============================
.. sidebar:: Create good commits!
- `Conventional Commits`_
- `Structural split of changes`_
- `Git Commit Good Practice`_
A commit and its commit message are among the most important information
available to a developer for bug fixing and further development. A commit is a
change and changes have a context (a change request).
In a SCM system (git), the change history is derived from the commit history. A
commit message is therefore part of the documentation for change management and
thus elementary for the traceability of changes.
**What a commit is not**: *A commit to an SCM system is not used to save files!*
A commit should always have a context and the commit message describes what is
to be changed in that context, just as a function description should describe
what the intention and the goal of the function is, a commit message should
describe what the intention and the goal of that commit is.
The commit messages form the history and are the first and therefore most
important information a developer has when he has to research when and why a
change had to be made and how it was made (what the goal was).
Like any text, a commit message should be written for the reader and not from
the perspective of the author.
When scrolling through the history, the first thing one see is the title of the
commit message. Therefore the title should describe the change as briefly and
precisely as possible ... followed by a blank line and then a somewhat detailed
description of the change.
----
The follwing rules should be in mind, when creating a commit:
- **Commit history should be read like a history book.**
- **Commit messages are for the reader not for the author of the commit.**
- **A commit is the atomic code-modification of a change in change management.**
- **Think about which descriptions from your PR might belong in the commit message.**
- **The maximum line length in a commit message is 80 characters.**
----
Choose meaningful commit messages:
.. code::
[type] optional scope: description
[body]
[optional trailers]
``[type]``:
Commits MUST be prefixed with a type .. ``feat``, ``fix``, ``refactor``,
``mod``, ``upd``, ``doc``, ``l10n``, ``build`` ..
``[body]``
`Information in commit messages`_
``[optional trailers]``:
- `Signed-off-by`_: certify that the committer has the rights to submit the
work under the projects license. That the developer has this right is a
prerequisite for a merge. If the `Signed-off-by`_ is not set in the
commit, the contributor enters his `Developer's Certificate of Origin` at
the latest when creating a PR!
- Closes: Link to the bug report or the bug number (e.g. ``Closes: #10``)
- `Co-authored-by`_: email address of the co-author
- Reported-by: email address (if there is no bug report)
- Suggested-by: email address (if there is no bug report)
----
To give examples at hand, here are a few commits. Follow the links to see the
full commit messages:
:patch:`44d941c93`
``[fix] mojeek web engine: don't add empty fmt argument for web searches``
:patch:`feb15e387`
``[fix] brave.news engine: response is HTML and no longer JSON``
:patch:`bdfe1c2a1`
``[mod] engines: migration of the individual cache solutions to EngineCache``
.. _Conventional Commits:
https://www.conventionalcommits.org/
.. _Structural split of changes:
https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes
.. _Git Commit Good Practice:
https://wiki.openstack.org/wiki/GitCommitMessages
.. _Information in commit messages:
https://wiki.openstack.org/wiki/GitCommitMessages#Information_in_commit_messages
.. _`Developer's Certificate of Origin`:
https://developercertificate.org/
.. _Signed-off-by:
https://git-scm.com/docs/git-commit#Documentation/git-commit.txt-code--signoffcode
.. _Co-authored-by:
https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors

View file

@ -0,0 +1,175 @@
.. _how to contribute:
=================
How to contribute
=================
.. contents::
:depth: 2
:local:
:backlinks: entry
Prime directives: Privacy, Hackability
======================================
SearXNG has two prime directives, **privacy-by-design and hackability** . The
hackability comes in three levels:
- support of search engines
- plugins to alter search behaviour
- hacking SearXNG itself
Note the lack of "world domination" among the directives. SearXNG has no
intention of wide mass-adoption, rounded corners, etc. The prime directive
"privacy" deserves a separate chapter, as it's quite uncommon unfortunately.
Privacy-by-design
-----------------
SearXNG was born out of the need for a **privacy-respecting** search tool which
can be extended easily to maximize both its search and its privacy protecting
capabilities.
Some widely used search engine features may work differently,
may be turned off by default, or may not be implemented at all in SearXNG
**as a consequence of a privacy-by-design approach**.
Following this approach, features reducing the privacy preserving aspects of SearXNG should be
switched off by default or should not be implemented at all. There are plenty of
search engines already providing such features. If a feature reduces
SearXNG's efficacy in protecting a user's privacy, the user must be informed about
the effect of choosing to enable it. Features that protect privacy but differ from the
expectations of the user should also be carefully explained to them.
Also, if you think that something works weird with SearXNG, it might be because
the tool you are using is designed in a way that interferes with SearXNG's privacy aspects.
Submitting a bug report to the vendor of the tool that misbehaves might be a good
feedback for them to reconsider the disrespect to their customers (e.g., ``GET`` vs ``POST``
requests in various browsers).
Remember the other prime directive of SearXNG is to be hackable, so if the above
privacy concerns do not fancy you, simply fork it.
*Happy hacking.*
Code
====
.. _PEP8: https://www.python.org/dev/peps/pep-0008/
.. _Structural split of changes:
https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes
.. sidebar:: Create good commits!
- :ref:`create commit`
In order to submit a patch, please follow the steps below:
- Follow coding conventions.
- PEP8_ standards apply, except the convention of line length
- Maximum line length is 120 characters
- The cardinal rule for creating good commits is to ensure there is only one
*logical change* per commit / read `Structural split of changes`_
- Check if your code breaks existing tests. If so, update the tests or fix your
code.
- If your code can be unit-tested, add unit tests.
- Add yourself to the :origin:`AUTHORS.rst` file.
- Choose meaningful commit messages, see :ref:`create commit`
- Create a pull request.
For more help on getting started with SearXNG development, see :ref:`devquickstart`.
Translation
===========
Translation currently takes place on :ref:`weblate <translation>`.
.. _contrib docs:
Documentation
=============
.. _Sphinx: https://www.sphinx-doc.org
.. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
.. sidebar:: The reST sources
has been moved from ``gh-branch`` into ``master`` (:origin:`docs`).
The documentation is built using Sphinx_. So in order to be able to generate
the required files, you have to install it on your system. Much easier, use
our :ref:`makefile`.
Here is an example which makes a complete rebuild:
.. code:: sh
$ make docs.clean docs.html
...
The HTML pages are in dist/docs.
.. _make docs.live:
Live build
----------
.. _sphinx-autobuild:
https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md
.. sidebar:: docs.clean
It is recommended to assert a complete rebuild before deploying (use
``docs.clean``).
Live build is like WYSIWYG. It's the recommended way to go if you want to edit the documentation.
The Makefile target ``docs.live`` builds the docs, opens
URL in your favorite browser and rebuilds every time a reST file has been
changed (:ref:`make docs.clean`).
.. code:: sh
$ make docs.live
...
The HTML pages are in dist/docs.
... Serving on http://0.0.0.0:8000
... Start watching changes
Live builds are implemented by sphinx-autobuild_. Use environment
``$(SPHINXOPTS)`` to pass arguments to the sphinx-autobuild_ command. You can
pass any argument except for the ``--host`` option (which is always set to ``0.0.0.0``).
E.g., to find and use a free port, use:
.. code:: sh
$ SPHINXOPTS="--port 0" make docs.live
...
... Serving on http://0.0.0.0:50593
...
.. _deploy on github.io:
deploy on github.io
-------------------
To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make
docs.gh-pages`, which builds the documentation and runs all the needed git add,
commit and push:
.. code:: sh
$ make docs.clean docs.gh-pages
.. attention::
If you are working in your own brand, don't forget to adjust your
:ref:`settings brand`.

View file

@ -0,0 +1,14 @@
.. _demo offline engine:
===================
Demo Offline Engine
===================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.demo_offline
:members:

View file

@ -0,0 +1,14 @@
.. _demo online engine:
==================
Demo Online Engine
==================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.demo_online
:members:

View file

@ -0,0 +1,254 @@
.. _engines-dev:
===============
Engine Overview
===============
.. contents::
:depth: 3
:local:
:backlinks: entry
.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
.. sidebar:: Further reading ..
- :ref:`configured engines`
- :ref:`settings engines`
SearXNG is a metasearch-engine_, so it uses different search engines to provide
better results.
Because there is no general search API which could be used for every search
engine, an adapter has to be built between SearXNG and the external search
engines. Adapters are stored under the folder :origin:`searx/engines`.
.. _general engine configuration:
General Engine Configuration
============================
It is required to tell SearXNG the type of results the engine provides. The
arguments can be set in the engine file or in the settings file (normally
``settings.yml``). The arguments in the settings file override the ones in the
engine file.
It does not matter if an option is stored in the engine file or in the settings.
However, the standard way is the following:
.. _engine file:
Engine File
-----------
.. table:: Common options in the engine module
:width: 100%
======================= =========== ========================================================
argument type information
======================= =========== ========================================================
categories list categories, in which the engine is working
paging boolean support multiple pages
time_range_support boolean support search time range
engine_type str - ``online`` :ref:`[ref] <online engines>` by
default, other possibles values are:
- ``offline`` :ref:`[ref] <offline engines>`
- ``online_dictionary`` :ref:`[ref] <online dictionary>`
- ``online_currency`` :ref:`[ref] <online currency>`
- ``online_url_search`` :ref:`[ref] <online url search>`
======================= =========== ========================================================
.. _engine settings:
Engine ``settings.yml``
-----------------------
For a more detailed description, see :ref:`settings engines` in the :ref:`settings.yml`.
.. table:: Common options in the engine setup (``settings.yml``)
:width: 100%
======================= =========== ==================================================
argument type information
======================= =========== ==================================================
name string name of search-engine
engine string name of searxng-engine (file name without ``.py``)
enable_http bool enable HTTP (by default only HTTPS is enabled).
shortcut string shortcut of search-engine
timeout string specific timeout for search-engine
display_error_messages boolean display error messages on the web UI
proxies dict set proxies for a specific engine
(e.g. ``proxies : {http: socks5://proxy:port,
https: socks5://proxy:port}``)
======================= =========== ==================================================
.. _engine overrides:
Overrides
---------
A few of the options have default values in the namespace of the engine's python
module, but are often overwritten by the settings. If ``None`` is assigned to an
option in the engine file, it has to be redefined in the settings, otherwise
SearXNG will not start with that engine (global names with a leading underline can
be ``None``).
Here is an very simple example of the global names in the namespace of engine's
module:
.. code:: python
# engine dependent config
categories = ['general']
paging = True
_non_overwritten_global = 'foo'
.. table:: The naming of overrides is arbitrary / recommended overrides are:
:width: 100%
======================= =========== ===========================================
argument type information
======================= =========== ===========================================
base_url string base-url, can be overwritten to use same
engine on other URL
number_of_results int maximum number of results per request
language string ISO code of language and country like en_US
api_key string api-key if required by engine
======================= =========== ===========================================
.. _engine request:
Making a Request
================
To perform a search an URL have to be specified. In addition to specifying an
URL, arguments can be passed to the query.
.. _engine request arguments:
Passed Arguments (request)
--------------------------
These arguments can be used to construct the search query. Furthermore,
parameters with default value can be redefined for special purposes.
.. _engine request online:
.. table:: If the ``engine_type`` is :py:obj:`online
<searx.search.processors.online.OnlineProcessor.get_params>`
:width: 100%
====================== ============== ========================================================================
argument type default-value, information
====================== ============== ========================================================================
url str ``''``
method str ``'GET'``
headers set ``{}``
data set ``{}``
cookies set ``{}``
verify bool ``True``
headers.User-Agent str a random User-Agent
category str current category, like ``'general'``
safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
pageno int current pagenumber
searxng_locale str SearXNG's locale selected by user. Specific language code like
``'en'``, ``'en-US'``, or ``'all'`` if unspecified.
====================== ============== ========================================================================
.. _engine request online_dictionary:
.. table:: If the ``engine_type`` is :py:obj:`online_dictionary
<searx.search.processors.online_dictionary.OnlineDictionaryProcessor.get_params>`,
in addition to the :ref:`online <engine request online>` arguments:
:width: 100%
====================== ============== ========================================================================
argument type default-value, information
====================== ============== ========================================================================
from_lang str specific language code like ``'en_US'``
to_lang str specific language code like ``'en_US'``
query str the text query without the languages
====================== ============== ========================================================================
.. _engine request online_currency:
.. table:: If the ``engine_type`` is :py:obj:`online_currency
<searx.search.processors.online_currency.OnlineCurrencyProcessor.get_params>`,
in addition to the :ref:`online <engine request online>` arguments:
:width: 100%
====================== ============== ========================================================================
argument type default-value, information
====================== ============== ========================================================================
amount float the amount to convert
from str ISO 4217 code
to str ISO 4217 code
from_name str currency name
to_name str currency name
====================== ============== ========================================================================
.. _engine request online_url_search:
.. table:: If the ``engine_type`` is :py:obj:`online_url_search
<searx.search.processors.online_url_search.OnlineUrlSearchProcessor.get_params>`,
in addition to the :ref:`online <engine request online>` arguments:
:width: 100%
====================== ============== ========================================================================
argument type default-value, information
====================== ============== ========================================================================
search_url dict URLs from the search query:
.. code:: python
{
'http': str,
'ftp': str,
'data:image': str
}
====================== ============== ========================================================================
Specify Request
---------------
The function :py:func:`def request(query, params):
<searx.engines.demo_online.request>` always returns the ``params`` variable, the
following parameters can be used to specify a search request:
.. table::
:width: 100%
=================== =========== ==========================================================================
argument type information
=================== =========== ==========================================================================
url str requested url
method str HTTP request method
headers set HTTP header information
data set HTTP data information
cookies set HTTP cookies
verify bool Performing SSL-Validity check
allow_redirects bool Follow redirects
max_redirects int maximum redirects, hard limit
soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
=================== =========== ==========================================================================
Making a Response
=================
In the ``response`` function of the engine, the HTTP response (``resp``) is
parsed and a list of results is returned.
A engine can append result-items of different media-types and different
result-types to the result list. The list of the result items is render to HTML
by templates. For more details read section:
- :ref:`simple theme templates`
- :ref:`result types`

View file

@ -0,0 +1,16 @@
.. _searx.enginelib:
==============
Engine Library
==============
.. automodule:: searx.enginelib
:members:
.. _searx.enginelib.traits:
Engine traits
=============
.. automodule:: searx.enginelib.traits
:members:

View file

@ -0,0 +1,9 @@
.. _searx.engines loader:
========================
SearXNG's engines loader
========================
.. automodule:: searx.engines
:members:

View file

@ -0,0 +1,116 @@
.. _engine implementations:
======================
Engine Implementations
======================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. toctree::
:caption: Framework Components
:maxdepth: 2
enginelib
engines
engine_overview
ResultList and engines
======================
.. autoclass:: searx.result_types.ResultList
.. autoclass:: searx.result_types.EngineResults
Engine Types
============
The :py:obj:`engine_type <searx.enginelib.Engine.engine_type>` of an engine
determines which :ref:`search processor <searx.search.processors>` is used by
the engine.
In this section a list of the engines that are documented is given, a complete
list of the engines can be found in the source under: :origin:`searx/engines`.
.. _online engines:
Online Engines
--------------
.. sidebar:: info
- :py:obj:`processors.online <searx.search.processors.online>`
.. toctree::
:maxdepth: 1
:glob:
demo/demo_online
xpath
mediawiki
json_engine
.. toctree::
:maxdepth: 1
:glob:
online/*
.. _offline engines:
Offline Engines
---------------
.. sidebar:: info
- :py:obj:`processors.offline <searx.search.processors.offline>`
.. toctree::
:maxdepth: 1
:glob:
offline_concept
demo/demo_offline
offline/*
.. _online url search:
Online URL Search
-----------------
.. sidebar:: info
- :py:obj:`processors.online_url_search <searx.search.processors.online_url_search>`
.. toctree::
:maxdepth: 1
:glob:
online_url_search/*
.. _online currency:
Online Currency
---------------
.. sidebar:: info
- :py:obj:`processors.online_currency <searx.search.processors.online_currency>`
*no engine of this type is documented yet / coming soon*
.. _online dictionary:
Online Dictionary
-----------------
.. sidebar:: info
- :py:obj:`processors.online_dictionary <searx.search.processors.online_dictionary>`
*no engine of this type is documented yet / coming soon*

View file

@ -0,0 +1,13 @@
.. _json_engine engine:
============
JSON Engine
============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.json_engine
:members:

View file

@ -0,0 +1,13 @@
.. _mediawiki engine:
================
MediaWiki Engine
================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.mediawiki
:members:

View file

@ -0,0 +1,23 @@
.. _engine command:
====================
Command Line Engines
====================
.. sidebar:: info
- :origin:`command.py <searx/engines/command.py>`
- :ref:`offline engines`
.. contents::
:depth: 2
:local:
:backlinks: entry
.. sidebar:: info
Initial sponsored by `Search and Discovery Fund
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
.. automodule:: searx.engines.command
:members:

View file

@ -0,0 +1,96 @@
.. _nosql engines:
===============
NoSQL databases
===============
.. sidebar:: further read
- `NoSQL databases <https://en.wikipedia.org/wiki/NoSQL>`_
- `valkey.io <https://valkey.io/>`_
- `MongoDB <https://www.mongodb.com>`_
.. contents::
:depth: 2
:local:
:backlinks: entry
.. sidebar:: info
Initial sponsored by `Search and Discovery Fund
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
The following `NoSQL databases`_ are supported:
- :ref:`engine valkey_server`
- :ref:`engine mongodb`
All of the engines above are just commented out in the :origin:`settings.yml
<searx/settings.yml>`, as you have to set various options and install
dependencies before using them.
By default, the engines use the ``key-value`` template for displaying results /
see :origin:`simple <searx/templates/simple/result_templates/key-value.html>`
theme. If you are not satisfied with the original result layout, you can use
your own template, set ``result_template`` attribute to ``{template_name}`` and
place the templates at::
searx/templates/{theme_name}/result_templates/{template_name}
Furthermore, if you do not wish to expose these engines on a public instance, you
can still add them and limit the access by setting ``tokens`` as described in
section :ref:`private engines`.
Extra Dependencies
==================
For using :ref:`engine valkey_server` or :ref:`engine mongodb` you need to
install additional packages in Python's Virtual Environment of your SearXNG
instance. To switch into the environment (:ref:`searxng-src`) you can use
:ref:`searxng.sh`::
$ sudo utils/searxng.sh instance cmd bash
(searxng-pyenv)$ pip install ...
Configure the engines
=====================
`NoSQL databases`_ are used for storing arbitrary data without first defining
their structure.
.. _engine valkey_server:
Valkey Server
-------------
.. _valkey: https://github.com/andymccurdy/valkey-py#installation
.. sidebar:: info
- ``pip install`` valkey_
- valkey.io_
- :origin:`valkey_server.py <searx/engines/valkey_server.py>`
.. automodule:: searx.engines.valkey_server
:members:
.. _engine mongodb:
MongoDB
-------
.. _pymongo: https://github.com/mongodb/mongo-python-driver#installation
.. sidebar:: info
- ``pip install`` pymongo_
- MongoDB_
- :origin:`mongodb.py <searx/engines/mongodb.py>`
.. automodule:: searx.engines.mongodb
:members:

View file

@ -0,0 +1,62 @@
=================
Local Search APIs
=================
.. sidebar:: further read
- `Comparison to alternatives
<https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
.. contents::
:depth: 1
:local:
:backlinks: entry
.. sidebar:: info
Initial sponsored by `Search and Discovery Fund
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
Administrators might find themselves wanting to integrate locally running search
engines. The following ones are supported for now:
* `Elasticsearch`_
* `Meilisearch`_
* `Solr`_
Each search engine is powerful, capable of full-text search. All of the engines
above are added to ``settings.yml`` just commented out, as you have to
``base_url`` for all them.
Please note that if you are not using HTTPS to access these engines, you have to
enable HTTP requests by setting ``enable_http`` to ``True``.
Furthermore, if you do not want to expose these engines on a public instance,
you can still add them and limit the access by setting ``tokens`` as described
in section :ref:`private engines`.
.. _engine meilisearch:
MeiliSearch
===========
.. automodule:: searx.engines.meilisearch
:members:
.. _engine elasticsearch:
Elasticsearch
=============
.. automodule:: searx.engines.elasticsearch
:members:
.. _engine solr:
Solr
====
.. automodule:: searx.engines.solr
:members:

View file

@ -0,0 +1,134 @@
.. _sql engines:
===========
SQL Engines
===========
.. sidebar:: further read
- `SQLite <https://www.sqlite.org/index.html>`_
- `PostgreSQL <https://www.postgresql.org>`_
- `MySQL <https://www.mysql.com>`_
.. contents::
:depth: 2
:local:
:backlinks: entry
.. sidebar:: info
Initial sponsored by `Search and Discovery Fund
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
With the *SQL engines* you can bind SQL databases into SearXNG. The following
Relational Database Management System (RDBMS) are supported:
- :ref:`engine sqlite`
- :ref:`engine postgresql`
- :ref:`engine mysql_server` & :ref:`engine mariadb_server`
All of the engines above are just commented out in the :origin:`settings.yml
<searx/settings.yml>`, as you have to set the required attributes for the
engines, e.g. ``database:`` ...
.. code:: yaml
- name: ...
engine: {sqlite|postgresql|mysql_server}
database: ...
result_template: {template_name}
query_str: ...
By default, the engines use the ``key-value`` template for displaying results /
see :origin:`simple <searx/templates/simple/result_templates/key-value.html>`
theme. If you are not satisfied with the original result layout, you can use
your own template, set ``result_template`` attribute to ``{template_name}`` and
place the templates at::
searx/templates/{theme_name}/result_templates/{template_name}
If you do not wish to expose these engines on a public instance, you can still
add them and limit the access by setting ``tokens`` as described in section
:ref:`private engines`.
Extra Dependencies
==================
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
install additional packages in Python's Virtual Environment of your SearXNG
instance. To switch into the environment (:ref:`searxng-src`) you can use
:ref:`searxng.sh`::
$ sudo utils/searxng.sh instance cmd bash
(searxng-pyenv)$ pip install ...
Configure the engines
=====================
The configuration of the new database engines are similar. You must put a valid
SQL-SELECT query in ``query_str``. At the moment you can only bind at most one
parameter in your query. By setting the attribute ``limit`` you can define how
many results you want from the SQL server. Basically, it is the same as the
``LIMIT`` keyword in SQL.
Please, do not include ``LIMIT`` or ``OFFSET`` in your SQL query as the engines
rely on these keywords during paging. If you want to configure the number of
returned results use the option ``limit``.
.. _engine sqlite:
SQLite
------
.. sidebar:: info
- :origin:`sqlite.py <searx/engines/sqlite.py>`
.. automodule:: searx.engines.sqlite
:members:
.. _engine postgresql:
PostgreSQL
----------
.. _psycopg2: https://www.psycopg.org/install
.. sidebar:: info
- :origin:`postgresql.py <searx/engines/postgresql.py>`
- ``pip install`` `psycopg2-binary <psycopg2>`_
.. automodule:: searx.engines.postgresql
:members:
.. _engine mysql_server:
MySQL
-----
.. sidebar:: info
- :origin:`mysql_server.py <searx/engines/mysql_server.py>`
- ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
.. automodule:: searx.engines.mysql_server
:members:
.. _engine mariadb_server:
MariaDB
--------
.. sidebar:: info
- :origin:`mariadb_server.py <searx/engines/mariadb_server.py>`
- ``pip install`` :pypi:`mariadb <mariadb>`
.. automodule:: searx.engines.mariadb_server
:members:

View file

@ -0,0 +1,69 @@
===============
Offline Concept
===============
.. sidebar:: offline engines
- :ref:`demo offline engine`
- :ref:`engine command`
- :ref:`sql engines`
- :ref:`nosql engines`
- :py:obj:`searx.search.processors.offline`
To extend the functionality of SearXNG, offline engines are going to be
introduced. An offline engine is an engine which does not need Internet
connection to perform a search and does not use HTTP to communicate.
Offline engines can be configured, by adding those to the `engines` list of
:origin:`settings.yml <searx/settings.yml>`. An example skeleton for offline
engines can be found in :ref:`demo offline engine` (:origin:`demo_offline.py
<searx/engines/demo_offline.py>`).
Programming Interface
=====================
:py:func:`init(engine_settings=None) <searx.engines.demo_offline.init>`
All offline engines can have their own init function to setup the engine before
accepting requests. The function gets the settings from settings.yml as a
parameter. This function can be omitted, if there is no need to setup anything
in advance.
:py:func:`search(query, params) <searx.engines.demo_offline.search>`
Each offline engine has a function named ``search``. This function is
responsible to perform a search and return the results in a presentable
format. (Where *presentable* means presentable by the selected result
template.)
The return value is a list of results retrieved by the engine.
Engine representation in ``/config``
If an engine is offline, the attribute ``offline`` is set to ``True``.
.. _offline requirements:
Extra Dependencies
==================
If an offline engine depends on an external tool, SearXNG does not install it by
default. When an administrator configures such engine and starts the instance,
the process returns an error with the list of missing dependencies. Also,
required dependencies will be added to the comment/description of the engine, so
admins can install packages in advance.
If there is a need to install additional packages in *Python's Virtual
Environment* of your SearXNG instance you need to switch into the environment
(:ref:`searxng-src`) first, for this you can use :ref:`searxng.sh`::
$ sudo utils/searxng.sh instance cmd bash
(searxng-pyenv)$ pip install ...
Private engines (Security)
==========================
To limit the access to offline engines, if an instance is available publicly,
administrators can set token(s) for each of the :ref:`private engines`. If a
query contains a valid token, then SearXNG performs the requested private
search. If not, requests from an offline engines return errors.

View file

@ -0,0 +1,13 @@
.. _adobe stock engine:
===========
Adobe Stock
===========
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.adobe_stock
:members:

View file

@ -0,0 +1,13 @@
.. _alpinelinux engine:
=====================
Alpine Linux Packages
=====================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.alpinelinux
:members:

View file

@ -0,0 +1,13 @@
.. _annas_archive engine:
==============
Anna's Archive
==============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.annas_archive
:members:

View file

@ -0,0 +1,14 @@
.. _archlinux engine:
==========
Arch Linux
==========
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.archlinux
:members:

View file

@ -0,0 +1,43 @@
.. _bing engines:
============
Bing Engines
============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. _bing web engine:
Bing WEB
========
.. automodule:: searx.engines.bing
:members:
.. _bing images engine:
Bing Images
===========
.. automodule:: searx.engines.bing_images
:members:
.. _bing videos engine:
Bing Videos
===========
.. automodule:: searx.engines.bing_videos
:members:
.. _bing news engine:
Bing News
=========
.. automodule:: searx.engines.bing_news
:members:

View file

@ -0,0 +1,13 @@
.. _bpb engine:
===
Bpb
===
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.bpb
:members:

View file

@ -0,0 +1,13 @@
.. _brave engine:
=============
Brave Engines
=============
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.brave
:members:

View file

@ -0,0 +1,14 @@
.. _bt4g engine:
====
BT4G
====
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.bt4g
:members:

View file

@ -0,0 +1,8 @@
.. _chinaso engine:
=======
ChinaSo
=======
.. automodule:: searx.engines.chinaso
:members:

View file

@ -0,0 +1,13 @@
.. _core engine:
====
CORE
====
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.core
:members:

View file

@ -0,0 +1,13 @@
.. _dailymotion engine:
===========
Dailymotion
===========
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.dailymotion
:members:

View file

@ -0,0 +1,8 @@
.. _discourse engine:
================
Discourse Forums
================
.. automodule:: searx.engines.discourse
:members:

View file

@ -0,0 +1,22 @@
.. _duckduckgo engines:
==================
DuckDuckGo Engines
==================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.duckduckgo
:members:
.. automodule:: searx.engines.duckduckgo_extra
:members:
.. automodule:: searx.engines.duckduckgo_definitions
:members:
.. automodule:: searx.engines.duckduckgo_weather
:members:

View file

@ -0,0 +1,8 @@
.. _gitea geizhals:
========
Geizhals
========
.. automodule:: searx.engines.geizhals
:members:

View file

@ -0,0 +1,8 @@
.. _gitea engine:
=====
Gitea
=====
.. automodule:: searx.engines.gitea
:members:

View file

@ -0,0 +1,8 @@
.. _gitlab engine:
======
GitLab
======
.. automodule:: searx.engines.gitlab
:members:

View file

@ -0,0 +1,76 @@
.. _google engines:
==============
Google Engines
==============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. _google API:
Google API
==========
.. _Query Parameter Definitions:
https://developers.google.com/custom-search/docs/xml_results#WebSearch_Query_Parameter_Definitions
SearXNG's implementation of the Google API is mainly done in
:py:obj:`get_google_info <searx.engines.google.get_google_info>`.
For detailed description of the *REST-full* API see: `Query Parameter
Definitions`_. The linked API documentation can sometimes be helpful during
reverse engineering. However, we cannot use it in the freely accessible WEB
services; not all parameters can be applied and some engines are more *special*
than other (e.g. :ref:`google news engine`).
.. _google web engine:
Google WEB
==========
.. automodule:: searx.engines.google
:members:
.. _google autocomplete:
Google Autocomplete
====================
.. autofunction:: searx.autocomplete.google_complete
.. _google images engine:
Google Images
=============
.. automodule:: searx.engines.google_images
:members:
.. _google videos engine:
Google Videos
=============
.. automodule:: searx.engines.google_videos
:members:
.. _google news engine:
Google News
===========
.. automodule:: searx.engines.google_news
:members:
.. _google scholar engine:
Google Scholar
==============
.. automodule:: searx.engines.google_scholar
:members:

View file

@ -0,0 +1,8 @@
.. _huggingface engine:
============
Hugging Face
============
.. automodule:: searx.engines.huggingface
:members:

View file

@ -0,0 +1,13 @@
.. _lemmy engine:
=====
Lemmy
=====
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.lemmy
:members:

View file

@ -0,0 +1,13 @@
.. _loc engine:
===================
Library of Congress
===================
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.loc
:members:

View file

@ -0,0 +1,13 @@
.. _mastodon engine:
========
Mastodon
========
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.mastodon
:members:

View file

@ -0,0 +1,13 @@
.. _moviepilot engine:
==========
Moviepilot
==========
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.moviepilot
:members:

View file

@ -0,0 +1,13 @@
.. _mrs engine:
=========================
Matrix Rooms Search (MRS)
=========================
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.mrs
:members:

View file

@ -0,0 +1,8 @@
.. _voidlinux mullvad_leta:
============
Mullvad-Leta
============
.. automodule:: searx.engines.mullvad_leta
:members:

View file

@ -0,0 +1,27 @@
.. _Mwmbl engine:
============
Mwmbl Engine
============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. _mwmbl web engine:
Mwmbl WEB
=========
.. automodule:: searx.engines.mwmbl
:members:
.. _mwmbl autocomplete:
Mwmbl Autocomplete
==================
.. autofunction:: searx.autocomplete.mwmbl

View file

@ -0,0 +1,13 @@
.. _odysee engine:
======
Odysee
======
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.odysee
:members:

View file

@ -0,0 +1,27 @@
.. _peertube engines:
================
Peertube Engines
================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. _peertube video engine:
Peertube Video
==============
.. automodule:: searx.engines.peertube
:members:
.. _sepiasearch engine:
SepiaSearch
===========
.. automodule:: searx.engines.sepiasearch
:members:

View file

@ -0,0 +1,13 @@
.. _piped engine:
=====
Piped
=====
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.piped
:members:

View file

@ -0,0 +1,13 @@
.. _engine presearch:
================
Presearch Engine
================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.presearch
:members:

View file

@ -0,0 +1,13 @@
.. _qwant engine:
=====
Qwant
=====
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.qwant
:members:

View file

@ -0,0 +1,13 @@
.. _RadioBrowser engine:
============
RadioBrowser
============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.radio_browser
:members:

View file

@ -0,0 +1,13 @@
.. _engine recoll:
=============
Recoll Engine
=============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.recoll
:members:

View file

@ -0,0 +1,8 @@
.. _reuters engine:
=======
Reuters
=======
.. automodule:: searx.engines.reuters
:members:

View file

@ -0,0 +1,13 @@
.. _seekr engine:
=============
Seekr Engines
=============
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.seekr
:members:

View file

@ -0,0 +1,13 @@
.. _soundcloud engine:
==========
Soundcloud
==========
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.soundcloud
:members:

View file

@ -0,0 +1,13 @@
.. _startpage engines:
=================
Startpage Engines
=================
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.startpage
:members:

View file

@ -0,0 +1,13 @@
.. _tagesschau engine:
==============
Tagesschau API
==============
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.tagesschau
:members:

View file

@ -0,0 +1,13 @@
.. _torznab engine:
==============
Torznab WebAPI
==============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.torznab
:members:

View file

@ -0,0 +1,8 @@
.. _tubearchivist engine:
==============
Tube Archivist
==============
.. automodule:: searx.engines.tubearchivist
:members:

View file

@ -0,0 +1,13 @@
.. _voidlinux engine:
==========================
Void Linux binary packages
==========================
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.voidlinux
:members:

View file

@ -0,0 +1,13 @@
.. _wallhaven engine:
=========
Wallhaven
=========
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.wallhaven
:members:

View file

@ -0,0 +1,27 @@
.. _wikimedia engines:
=========
Wikimedia
=========
.. contents::
:depth: 2
:local:
:backlinks: entry
.. _wikipedia engine:
Wikipedia
=========
.. automodule:: searx.engines.wikipedia
:members:
.. _wikidata engine:
Wikidata
=========
.. automodule:: searx.engines.wikidata
:members:

View file

@ -0,0 +1,13 @@
.. _yacy engine:
====
Yacy
====
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.yacy
:members:

View file

@ -0,0 +1,13 @@
.. _yahoo engine:
============
Yahoo Engine
============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.yahoo
:members:

View file

@ -0,0 +1,13 @@
.. _zlibrary engine:
=========
Z-Library
=========
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.zlibrary
:members:

View file

@ -0,0 +1,14 @@
.. _tineye engine:
======
Tineye
======
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.tineye
:members:

View file

@ -0,0 +1,14 @@
.. _xpath engine:
============
XPath Engine
============
.. contents::
:depth: 2
:local:
:backlinks: entry
.. automodule:: searx.engines.xpath
:members:

View file

@ -0,0 +1,7 @@
.. _extended_types.:
==============
Extended Types
==============
.. automodule:: searx.extended_types

View file

@ -0,0 +1,22 @@
=======================
Developer documentation
=======================
.. toctree::
:maxdepth: 2
quickstart
commits
rtm_asdf
contribution_guide
extended_types
engines/index
result_types/index
templates
search_api
plugins/index
answerers/index
translation
makefile
reST
searxng_extra/index

View file

@ -0,0 +1,373 @@
.. _makefile:
=======================
Makefile & ``./manage``
=======================
.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
All relevant build and development tasks are implemented in the
:origin:`./manage <manage>` script and for CI or IDE integration a small
:origin:`Makefile` wrapper is available. If you are not familiar with
Makefiles, we recommend to read gnu-make_ introduction.
.. sidebar:: build environment
Before looking deeper at the targets, first read about :ref:`make
install`.
To install developer requirements follow :ref:`buildhosts`.
.. contents::
:depth: 2
:local:
:backlinks: entry
The usage is simple, just type ``make {target-name}`` to *build* a target.
Calling the ``help`` target gives a first overview (``make help``):
.. tabs::
.. group-tab:: ``make``
.. program-output:: bash -c "cd ..; make --no-print-directory help"
.. group-tab:: ``./manage``
The Makefile targets are implemented for comfort, if you can do without
tab-completion and need to have a more granular control, use
:origin:`manage` without the Makefile wrappers.
.. code:: sh
$ ./manage help
.. _make install:
Python environment (``make install``)
=====================================
.. sidebar:: activate environment
``source ./local/py3/bin/activate``
We do no longer need to build up the virtualenv manually. Jump into your git
working tree and release a ``make install`` to get a virtualenv with a
*developer install* of SearXNG (:origin:`setup.py`). ::
$ cd ~/searxng-clone
$ make install
PYENV [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV [install] pip install --use-pep517 --no-build-isolation -e 'searx[test]'
...
Successfully installed searxng-2023.7.19+a446dea1b
If you release ``make install`` multiple times the installation will only
rebuild if the sha256 sum of the *requirement files* fails. With other words:
the check fails if you edit the requirements listed in
:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
$ make install
PYENV OK
PYENV [virtualenv] requirements.sha256 failed
[virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
[virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
...
PYENV [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV [install] pip install --use-pep517 --no-build-isolation -e 'searx[test]'
...
Successfully installed searxng-2023.7.19+a446dea1b
.. sidebar:: drop environment
To get rid of the existing environment before re-build use :ref:`clean target
<make clean>` first.
If you think, something goes wrong with your ./local environment or you change
the :origin:`setup.py` file, you have to call :ref:`make clean`.
.. _make node.env:
Node.js environment (``make node.env``)
=======================================
.. _Node.js: https://nodejs.org/
.. _nvm: https://github.com/nvm-sh
.. _npm: https://www.npmjs.com/
.. jinja:: searx
Node.js_ version {{version.node}} or higher is required to build the themes.
If the requirement is not met, the build chain uses nvm_ (Node Version
Manager) to install latest LTS of Node.js_ locally: there is no need to
install nvm_ or npm_ on your system.
To install NVM_ and Node.js_ in once you can use :ref:`make nvm.nodejs`.
.. _make nvm:
NVM ``make nvm.install nvm.status``
-----------------------------------
Use ``make nvm.status`` to get the current status of your Node.js_ and nvm_
setup.
.. tabs::
.. group-tab:: nvm.install
.. code:: sh
$ LANG=C make nvm.install
INFO: install (update) NVM at ./searxng/.nvm
INFO: clone: https://github.com/nvm-sh/nvm.git
|| Cloning into './searxng/.nvm'...
INFO: checkout v0.39.4
|| HEAD is now at 8fbf8ab v0.39.4
.. group-tab:: nvm.status (ubu2004)
Here is the output you will typically get on a Ubuntu 20.04 system which
serves only a `no longer active <https://nodejs.org/en/about/releases/>`_
Release `Node.js v10.19.0 <https://packages.ubuntu.com/focal/nodejs>`_.
.. code:: sh
$ make nvm.status
INFO: Node.js is installed at /usr/bin/node
INFO: Node.js is version v10.19.0
WARN: minimal Node.js version is 16.13.0
INFO: npm is installed at /usr/bin/npm
INFO: npm is version 6.14.4
WARN: NVM is not installed
.. _make nvm.nodejs:
``make nvm.nodejs``
-------------------
Install latest Node.js_ LTS locally (uses nvm_)::
$ make nvm.nodejs
INFO: install (update) NVM at /share/searxng/.nvm
INFO: clone: https://github.com/nvm-sh/nvm.git
...
Downloading and installing node v16.13.0...
...
INFO: Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node
INFO: Node.js is version v16.13.0
INFO: npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm
INFO: npm is version 8.1.0
INFO: NVM is installed at searxng/.nvm
.. _make run:
``make run``
============
To get up a running a developer instance simply call ``make run``. This enables
*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
instance and opens the URL in your favorite WEB browser (:man:`xdg-open`)::
$ make run
Changes to theme's HTML templates (jinja2) are instant. Changes to the CSS & JS
sources of the theme need to be rebuild. You can do that by running::
$ make themes.all
..
ToDo: vite server is not implemented yet / will be done in a follow up PR
Alternatively to ``themes.all`` you can run *live builds* of the theme you are
modify (:ref:`make themes`)::
$ LIVE_THEME=simple make run
.. _make format:
``make format``
======================
.. _Black code style:
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html
.. _shfmt: https://github.com/mvdan/sh?tab=readme-ov-file#shfmt
.. _EditorConfig: https://github.com/patrickvane/shfmt?tab=readme-ov-file#description
- Format Python source code using `Black code style`_. See ``$BLACK_OPTIONS``
and ``$BLACK_TARGETS`` in :origin:`Makefile`.
- Format Shell scripts using shfmt_. The formatter ``shfmt`` reads the rules
from the EditorConfig_ files.
.. _make clean:
``make clean``
==============
Drops all intermediate files, all builds, but keep sources untouched. Before
calling ``make clean`` stop all processes using the :ref:`make install` or
:ref:`make node.env`. ::
$ make clean
CLEAN pyenv
PYENV [virtualenv] drop local/py3
CLEAN docs -- build/docs dist/docs
CLEAN themes -- locally installed npm dependencies
...
CLEAN test stuff
CLEAN common files
.. _make docs:
``make docs``
=============
Target ``docs`` builds the documentation:
.. code:: bash
$ make docs
HTML ./docs --> file://
DOCS build build/docs/includes
...
The HTML pages are in dist/docs.
.. _make docs.clean:
``make docs.clean docs.live``
----------------------------------
We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
Documentation <contrib docs>` section. If you want to edit the documentation
read our :ref:`make docs.live` section. If you are working in your own brand,
adjust your :ref:`settings brand`.
.. _make docs.gh-pages:
``make docs.gh-pages``
----------------------
To deploy on github.io first adjust your :ref:`settings brand`. For any
further read :ref:`deploy on github.io`.
.. _make test:
``make test``
=============
Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
and ``test.robot``. You can run tests selective, e.g.::
$ make test.pep8 test.unit test.shell
TEST test.pep8 OK
...
TEST test.unit OK
...
TEST test.shell OK
.. _make test.shell:
``make test.shell``
-------------------
:ref:`sh lint` / if you have changed some bash scripting run this test before
commit.
.. _make test.pylint:
``make test.pylint``
--------------------
.. _Pylint: https://www.pylint.org/
Pylint_ is known as one of the best source-code, bug and quality checker for the
Python programming language. The pylint profile used in the SearXNG project is
found in project's root folder :origin:`.pylintrc`.
.. _make search.checker:
``make search.checker.{engine name}``
=====================================
To check all engines::
make search.checker
To check a engine with whitespace in the name like *google news* replace space
by underline::
make search.checker.google_news
To see HTTP requests and more use SEARXNG_DEBUG::
make SEARXNG_DEBUG=1 search.checker.google_news
.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
To filter out HTTP redirects (3xx_)::
make SEARXNG_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
...
Engine google news Checking
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
.. _make themes:
``make themes.*``
=================
.. sidebar:: further read
- :ref:`devquickstart`
The :origin:`Makefile` targets ``make theme.*`` cover common tasks to build the
theme(s). The ``./manage themes.*`` command line can be used to convenient run
common theme build tasks.
.. program-output:: bash -c "cd ..; ./manage themes.help"
To get live builds while modifying CSS & JS use (:ref:`make run`):
.. code:: sh
$ LIVE_THEME=simple make run
.. _make static.build:
``make static.build.*``
=======================
.. sidebar:: further read
- :ref:`devquickstart`
The :origin:`Makefile` targets ``static.build.*`` cover common tasks to build (a
commit of) the static files. The ``./manage static.build..*`` command line
can be used to convenient run common build tasks of the static files.
.. program-output:: bash -c "cd ..; ./manage static.help"
.. _manage go.help:
``./manage go.help``
====================
The ``./manage go.*`` command line can be used to convenient run common `go
(wiki)`_ tasks.
.. _go (wiki): https://en.wikipedia.org/wiki/Go_(programming_language)
.. program-output:: bash -c "cd ..; ./manage go.help"

View file

@ -0,0 +1,15 @@
.. _builtin plugins:
================
Built-in Plugins
================
.. toctree::
:maxdepth: 1
calculator
hash_plugin
hostnames
self_info
tor_check
unit_converter

View file

@ -0,0 +1,8 @@
.. _plugins.calculator:
==========
Calculator
==========
.. automodule:: searx.plugins.calculator
:members:

View file

@ -0,0 +1,7 @@
.. _dev plugin:
==================
Plugin Development
==================
.. automodule:: searx.plugins

View file

@ -0,0 +1,8 @@
.. _hash_plugin plugin:
===========
Hash Values
===========
.. autoclass:: searx.plugins.hash_plugin.SXNGPlugin
:members:

View file

@ -0,0 +1,8 @@
.. _hostnames plugin:
=========
Hostnames
=========
.. automodule:: searx.plugins.hostnames
:members:

View file

@ -0,0 +1,9 @@
=======
Plugins
=======
.. toctree::
:maxdepth: 2
development
builtins

View file

@ -0,0 +1,8 @@
.. _self_info plugin:
=========
Self-Info
=========
.. autoclass:: searx.plugins.self_info.SXNGPlugin
:members:

View file

@ -0,0 +1,8 @@
.. _tor check plugin:
=========
Tor check
=========
.. automodule:: searx.plugins.tor_check
:members:

View file

@ -0,0 +1,8 @@
.. _unit converter plugin:
==============
Unit Converter
==============
.. automodule:: searx.plugins.unit_converter
:members:

View file

@ -0,0 +1,82 @@
.. _devquickstart:
======================
Development Quickstart
======================
.. _npm: https://www.npmjs.com/
.. _Node.js: https://nodejs.org/
.. _Biome: https://biomejs.dev/
.. _Stylelint: https://stylelint.io/
.. sidebar:: further read
- :ref:`makefile`
- :ref:`buildhosts`
SearXNG loves developers; Developers do not need to worry about tool chains, the
usual developer tasks can be comfortably executed via :ref:`make <makefile>`.
Don't hesitate, just clone SearXNG's sources and start hacking right now ..
.. code:: bash
git clone https://github.com/searxng/searxng.git searxng
Here is how a minimal workflow looks like:
1. *start* hacking
2. *run* your code: :ref:`make run`
3. *format & test* your code: :ref:`make format` and :ref:`make test`
If you think at some point something fails, go back to *start*. Otherwise,
choose a meaningful commit message and we are happy to receive your pull
request. To not end in *wild west* we have some directives, please pay attention
to our ":ref:`how to contribute`" guideline.
.. sidebar:: further read
- :ref:`make nvm`
- :ref:`make themes`
If you implement themes, you will need to setup a :ref:`Node.js environment
<make node.env>`. Before you call *make run* (2.), you need to compile the
modified styles and JavaScript: ``make node.clean themes.all``. If Biome_
or Stylelint_ reports issues, try ``make themes.fix``.
Alternatively you can also compile selective the theme you have modified,
e.g. the *simple* theme.
.. code:: bash
make themes.simple
.. tip::
To get live builds while modifying CSS & JS use: ``LIVE_THEME=simple make run``
.. sidebar:: further read
- :ref:`make static.build`
If you finished your *tests* you can start to commit your changes. To separate
the modified source code from the build products first run:
.. code:: bash
make static.build.restore
This will restore the old build products and only your changes of the code
remain in the working tree which can now be added & committed. When all sources
are committed, you can commit the build products simply by:
.. code:: bash
make static.build.commit
Committing the build products should be the last step, just before you send us
your PR. There is also a make target to rewind this last build commit:
.. code:: bash
make static.build.drop

1438
_sources/dev/reST.rst.txt Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,10 @@
.. _result_types.answer:
==============
Answer Results
==============
The :ref:`area answer results` is an area in which additional information can
be displayed.
.. automodule:: searx.result_types.answer

View file

@ -0,0 +1,5 @@
======
Result
======
.. automodule:: searx.result_types._base

View file

@ -0,0 +1,34 @@
.. _result_types.corrections:
==================
Correction Results
==================
.. hint::
There is still no typing for these result items. The templates can be used as
orientation until the final typing is complete.
The :ref:`area corrections results` shows the user alternative search terms.
A result of this type is a very simple dictionary with only one key/value pair
.. code:: python
{"correction" : "lorem ipsum .."}
From this simple dict another dict is build up:
.. code:: python
# use RawTextQuery to get the corrections URLs with the same bang
{"url" : "!bang lorem ipsum ..", "title": "lorem ipsum .." }
and used in the template :origin:`corrections.html
<searx/templates/simple/elements/corrections.html>`:
title : :py:class:`str`
Corrected search term.
url : :py:class:`str`
Not really an URL, its the value to insert in a HTML form for a SearXNG query.

View file

@ -0,0 +1,105 @@
.. _result types:
============
Result Types
============
To understand the typification of the results, let's take a brief look at the
structure of SearXNG .. At its core, SearXNG is nothing more than an aggregator
that aggregates the results from various sources, renders them via templates and
displays them to the user.
The **sources** can be:
1. :ref:`engines <engine implementations>`
2. :ref:`plugins <dev plugin>`
3. :ref:`answerers <dev answerers>`
The sources provide the results, which are displayed in different **areas**
depending on the type of result. The areas are:
.. _area main results:
:ref:`area main results <main search results>`
It is the main area in which -- as is typical for search engines -- the
results that a search engine has found for the search term are displayed.
.. _area answer results:
:ref:`area answers <result_types.answer>`
This area displays short answers that could be found for the search term.
.. _area info box:
:ref:`area info box <result_types.infobox>`
An area in which additional information can be displayed, e.g. excerpts from
wikipedia or other sources such as maps.
.. _area suggestions results:
:ref:`area suggestions <result_types.suggestion>`
Suggestions for alternative search terms can be found in this area. These can
be clicked on and a search is carried out with these search terms.
.. _area corrections results:
:ref:`area corrections <result_types.corrections>`
Results in this area are like the suggestion of alternative search terms,
which usually result from spelling corrections
At this point it is important to note that all **sources** can contribute
results to all of the areas mentioned above.
In most cases, however, the :ref:`engines <engine implementations>` will fill
the *main results* and the :ref:`answerers <dev answerers>` will generally
provide the contributions for the *answer* area. Not necessary to mention here
but for a better understanding: the plugins can also filter out or change
results from the main results area (e.g. the URL of the link).
The result items are organized in the :py:obj:`results.ResultContainer` and
after all sources have delivered their results, this container is passed to the
templating to build a HTML output. The output is usually HTML, but it is also
possible to output the result lists as JSON or RSS feed. Thats quite all we need
to know before we dive into typification of result items.
.. hint::
Typification of result items: we are at the very first beginng!
The first thing we have to realize is that there is no typification of the
result items so far, we have to build it up first .. and that is quite a big
task, which we will only be able to accomplish gradually.
The foundation for the typeless results was laid back in 2013 in the very first
commit :commit:`ae9fb1d7d`, and the principle has not changed since then. At
the time, the approach was perfectly adequate, but we have since evolved and the
demands on SearXNG increase with every feature request.
**Motivation:** in the meantime, it has become very difficult to develop new
features that require structural changes and it is especially hard for newcomers
to find their way in this typeless world. As long as the results are only
simple key/value dictionaries, it is not even possible for the IDEs to support
the application developer in his work.
**Planning:** The procedure for subsequent typing will have to be based on the
circumstances ..
.. attention::
As long as there is no type defined for a kind of result the HTML template
specify what the properties of a type are.
In this sense, you will either find a type definition here in the
documentation or, if this does not yet exist, a description of the HTML
template.
.. toctree::
:maxdepth: 2
base_result
main_result
answer
correction
suggestion
infobox

View file

@ -0,0 +1,59 @@
.. _result_types.infobox:
===============
Infobox Results
===============
.. hint::
There is still no typing for these result items. The templates can be used as
orientation until the final typing is complete.
The :ref:`area info box` is an area where addtional infos shown to the user.
Fields used in the :origin:`infobox.html
<searx/templates/simple/elements/infobox.html>`:
img_src: :py:class:`str`
URL of a image or thumbnail that is displayed in the infobox.
infobox: :py:class:`str`
Title of the info box.
content: :py:class:`str`
Text of the info box.
The infobox has additional subsections for *attributes*, *urls* and
*relatedTopics*:
attributes: :py:class:`List <list>`\ [\ :py:class:`dict`\ ]
A list of attributes. An *attribute* is a dictionary with keys:
- label :py:class:`str`: (mandatory)
- value :py:class:`str`: (mandatory)
- image :py:class:`List <list>`\ [\ :py:class:`dict`\ ] (optional)
A list of images. An *image* is a dictionary with keys:
- src :py:class:`str`: URL of an image/thumbnail (mandatory)
- alt :py:class:`str`: alternative text for the image (mandatory)
urls: :py:class:`List <list>`\ [\ :py:class:`dict`\ ]
A list of links. An *link* is a dictionary with keys:
- url :py:class:`str`: URL of the link (mandatory)
- title :py:class:`str`: Title of the link (mandatory)
relatedTopics: :py:class:`List <list>`\ [\ :py:class:`dict`\ ]
A list of topics. An *topic* is a dictionary with keys:
- name: :py:class:`str`: (mandatory)
- suggestions: :py:class:`List <list>`\ [\ :py:class:`dict`\ ] (optional)
A list of suggestions. A *suggestion* is simple dictionary with just one
key/value pair:
- suggestion: :py:class:`str`: suggested search term (mandatory)

View file

@ -0,0 +1,7 @@
.. _result_types.keyvalue:
=================
Key-Value Results
=================
.. automodule:: searx.result_types.keyvalue

View file

@ -0,0 +1,4 @@
.. _result_types.mainresult:
.. autoclass:: searx.result_types._base.MainResult
:members:

View file

@ -0,0 +1,32 @@
.. _main search results:
===================
Main Search Results
===================
In the :ref:`area main results` the results that a search engine has found for
the search term are displayed.
There is still no typing for all items in the :ref:`main result list`. The
following types have been implemented so far ..
.. toctree::
:maxdepth: 2
main/mainresult
main/keyvalue
The :ref:`LegacyResult <LegacyResult>` is used internally for the results that
have not yet been typed. The templates can be used as orientation until the
final typing is complete.
- :ref:`template default` / :py:obj:`Result`
- :ref:`template images`
- :ref:`template videos`
- :ref:`template torrent`
- :ref:`template map`
- :ref:`template paper`
- :ref:`template packages`
- :ref:`template code`
- :ref:`template files`
- :ref:`template products`

View file

@ -0,0 +1,38 @@
.. _result_types.suggestion:
==================
Suggestion Results
==================
.. hint::
There is still no typing for these result items. The templates can be used as
orientation until the final typing is complete.
The :ref:`area suggestions results` shows the user alternative search terms.
A result of this type is a very simple dictionary with only one key/value pair
.. code:: python
{"suggestion" : "lorem ipsum .."}
From this simple dict another dict is build up:
.. code:: python
{"url" : "!bang lorem ipsum ..", "title": "lorem ipsum" }
and used in the template :origin:`suggestions.html
<searx/templates/simple/elements/suggestions.html>`:
.. code:: python
# use RawTextQuery to get the suggestion URLs with the same bang
{"url" : "!bang lorem ipsum ..", "title": "lorem ipsum" }
title : :py:class:`str`
Suggested search term
url : :py:class:`str`
Not really an URL, its the value to insert in a HTML form for a SearXNG query.

View file

@ -0,0 +1,121 @@
==================
Runtime Management
==================
The runtimes are managed with asdf and are activated in this project via the
`.tool-versions <.tool-versions>`_. If you have not yet installed asdf_, then
chapter :ref:`introduce asdf` may be of help to you.
.. contents::
:depth: 2
:local:
:backlinks: entry
Get started
===========
If you have asdf installed you can install the runtimes of this project by:
.. code:: bash
$ cd /path/to/searxng
$ asdf install # will install runtimes listed in .tool-versions
...
Manage Versions
===============
If you want to perform a ``test`` with special runtime versions of nodejs,
python or shellcheck, you can patch the ``.tool-versions``:
.. code:: diff
--- a/.tool-versions
+++ b/.tool-versions
@@ -1,2 +1,2 @@
-python 3.12.0
-shellcheck 0.9.0
+python 3.11.6
+shellcheck 0.8.0
To install use ``asdf install`` again. If the runtime tools have changed, any
existing (nodejs and python) environments should be cleaned up with a ``make
clean``.
.. code:: bash
$ asdf install
...
$ make clean test
.. _introduce asdf:
Introduce asdf
==============
To `download asdf`_ and `install asdf`_:
.. code:: bash
$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch <version>
$ echo '. "$HOME/.asdf/asdf.sh"' >> ~/.bashrc
$ echo '. "$HOME/.asdf/completions/asdf.bash"' >> ~/.bashrc
Start a new shell and try to `install plugins`_:
.. code:: bash
$ asdf plugin-list-all | grep -E '(golang|python|nodejs|shellcheck).git'
golang https://github.com/asdf-community/asdf-golang.git
nodejs https://github.com/asdf-vm/asdf-nodejs.git
python https://github.com/danhper/asdf-python.git
shellcheck https://github.com/luizm/asdf-shellcheck.git
$ asdf plugin add golang https://github.com/asdf-community/asdf-golang.git
$ asdf plugin add nodejs https://github.com/asdf-vm/asdf-nodejs.git
$ asdf plugin add python https://github.com/danhper/asdf-python.git
$ asdf plugin add shellcheck https://github.com/luizm/asdf-shellcheck.git
Each plugin has dependencies, to compile runtimes visit the URLs from above and
look out for the dependencies you need to install on your OS, on Debian for the
runtimes listed above you will need:
.. code:: bash
$ sudo apt update
$ sudo apt install \
dirmngr gpg curl gawk coreutils build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
With dependencies installed you can install/compile runtimes:
.. code:: bash
$ asdf install golang latest
$ asdf install nodejs latest
$ asdf install python latest
$ asdf install shellcheck latest
Python will be compiled and will take a while.
In the repository the version is defined in `.tool-versions`_. Outside the
repository, its recommended that the runtime should use the versions of the OS
(`Fallback to System Version`_) / if not already done register the system
versions global:
.. code:: bash
$ cd /
$ asdf global golang system
$ asdf global nodejs system
$ asdf global python system
$ asdf global shellcheck system
.. _asdf: https://asdf-vm.com/
.. _download asdf: https://asdf-vm.com/guide/getting-started.html#_2-download-asdf
.. _install asdf: https://asdf-vm.com/guide/getting-started.html#_3-install-asdf
.. _install plugins: https://asdf-vm.com/guide/getting-started.html#install-the-plugin
.. _Fallback to System Version: https://asdf-vm.com/manage/versions.html#fallback-to-system-version

View file

@ -0,0 +1,124 @@
.. _search API:
==========
Search API
==========
The search supports both ``GET`` and ``POST``.
Furthermore, two endpoints ``/`` and ``/search`` are available for querying.
``GET /``
``GET /search``
Parameters
==========
.. sidebar:: Further reading ..
- :ref:`engines-dev`
- :ref:`settings.yml`
- :ref:`configured engines`
``q`` : required
The search query. This string is passed to external search services. Thus,
SearXNG supports syntax of each search service. For example, ``site:github.com
SearXNG`` is a valid query for Google. However, if simply the query above is
passed to any search engine which does not filter its results based on this
syntax, you might not get the results you wanted.
See more at :ref:`search-syntax`
``categories`` : optional
Comma separated list, specifies the active search categories (see
:ref:`configured engines`)
``engines`` : optional
Comma separated list, specifies the active search engines (see
:ref:`configured engines`).
``language`` : default from :ref:`settings search`
Code of the language.
``pageno`` : default ``1``
Search page number.
``time_range`` : optional
[ ``day``, ``month``, ``year`` ]
Time range of search for engines which support it. See if an engine supports
time range search in the preferences page of an instance.
``format`` : optional
[ ``json``, ``csv``, ``rss`` ]
Output format of results. Format needs to be activated in :ref:`settings
search`.
``results_on_new_tab`` : default ``0``
[ ``0``, ``1`` ]
Open search results on new tab.
``image_proxy`` : default from :ref:`settings server`
[ ``True``, ``False`` ]
Proxy image results through SearXNG.
``autocomplete`` : default from :ref:`settings search`
[ ``google``, ``dbpedia``, ``duckduckgo``, ``mwmbl``, ``startpage``,
``wikipedia``, ``stract``, ``swisscows``, ``qwant`` ]
Service which completes words as you type.
``safesearch`` : default from :ref:`settings search`
[ ``0``, ``1``, ``2`` ]
Filter search results of engines which support safe search. See if an engine
supports safe search in the preferences page of an instance.
``theme`` : default ``simple``
[ ``simple`` ]
Theme of instance.
Please note, available themes depend on an instance. It is possible that an
instance administrator deleted, created or renamed themes on their instance.
See the available options in the preferences page of the instance.
``enabled_plugins`` : optional
List of enabled plugins.
:default:
``Hash_plugin``, ``Self_Information``,
``Tracker_URL_remover``, ``Ahmia_blacklist``
:values:
.. enabled by default
``Hash_plugin``, ``Self_Information``,
``Tracker_URL_remover``, ``Ahmia_blacklist``,
.. disabled by default
``Hostnames_plugin``, ``Open_Access_DOI_rewrite``,
``Vim-like_hotkeys``, ``Tor_check_plugin``
``disabled_plugins``: optional
List of disabled plugins.
:default:
``Hostnames_plugin``, ``Open_Access_DOI_rewrite``,
``Vim-like_hotkeys``, ``Tor_check_plugin``
:values:
see values from ``enabled_plugins``
``enabled_engines`` : optional : *all* :origin:`engines <searx/engines>`
List of enabled engines.
``disabled_engines`` : optional : *all* :origin:`engines <searx/engines>`
List of disabled engines.

View file

@ -0,0 +1,14 @@
.. _searxng_extra:
=============================
Tooling box ``searxng_extra``
=============================
In the folder :origin:`searxng_extra/` we maintain some tools useful for CI and
developers.
.. toctree::
:maxdepth: 2
update
standalone_searx.py

View file

@ -0,0 +1,9 @@
.. _standalone_searx.py:
=====================================
``searxng_extra/standalone_searx.py``
=====================================
.. automodule:: searxng_extra.standalone_searx
:members:

View file

@ -0,0 +1,99 @@
=========================
``searxng_extra/update/``
=========================
:origin:`[source] <searxng_extra/update/__init__.py>`
Scripts to update static data in :origin:`searx/data/`
.. _update_ahmia_blacklist.py:
``update_ahmia_blacklist.py``
=============================
:origin:`[source] <searxng_extra/update/update_ahmia_blacklist.py>`
.. automodule:: searxng_extra.update.update_ahmia_blacklist
:members:
``update_currencies.py``
========================
:origin:`[source] <searxng_extra/update/update_currencies.py>`
.. automodule:: searxng_extra.update.update_currencies
:members:
``update_engine_descriptions.py``
=================================
:origin:`[source] <searxng_extra/update/update_engine_descriptions.py>`
.. automodule:: searxng_extra.update.update_engine_descriptions
:members:
``update_external_bangs.py``
============================
:origin:`[source] <searxng_extra/update/update_external_bangs.py>`
.. automodule:: searxng_extra.update.update_external_bangs
:members:
``update_firefox_version.py``
=============================
:origin:`[source] <searxng_extra/update/update_firefox_version.py>`
.. automodule:: searxng_extra.update.update_firefox_version
:members:
``update_engine_traits.py``
===========================
:origin:`[source] <searxng_extra/update/update_engine_traits.py>`
.. automodule:: searxng_extra.update.update_engine_traits
:members:
.. _update_osm_keys_tags.py:
``update_osm_keys_tags.py``
===========================
:origin:`[source] <searxng_extra/update/update_osm_keys_tags.py>`
.. automodule:: searxng_extra.update.update_osm_keys_tags
:members:
``update_pygments.py``
======================
:origin:`[source] <searxng_extra/update/update_pygments.py>`
.. automodule:: searxng_extra.update.update_pygments
:members:
.. _update_locales.py:
``update_locales.py``
=====================
:origin:`[source] <searxng_extra/update/update_locales.py>`
.. automodule:: searxng_extra.update.update_locales
:members:
``update_wikidata_units.py``
============================
:origin:`[source] <searxng_extra/update/update_wikidata_units.py>`
.. automodule:: searxng_extra.update.update_wikidata_units
:members:

View file

@ -0,0 +1,577 @@
.. _simple theme templates:
======================
Simple Theme Templates
======================
The simple template is complex, it consists of many different elements and also
uses macros and include statements. The following is a rough overview that we
would like to give the developer at hand, details must still be taken from the
:origin:`sources <searx/templates/simple/>`.
A :ref:`result item <result types>` can be of different media types. The media
type of a result is defined by the :py:obj:`result_type.Result.template`. To
set another media-type as :ref:`template default`, the field ``template``
in the result item must be set to the desired type.
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
.. _result template macros:
Result template macros
======================
.. _macro result_header:
``result_header``
-----------------
Execpt ``image.html`` and some others this macro is used in nearly all result
types in the :ref:`main result list`.
Fields used in the template :origin:`macro result_header
<searx/templates/simple/macros.html>`:
url : :py:class:`str`
Link URL of the result item.
title : :py:class:`str`
Link title of the result item.
img_src, thumbnail : :py:class:`str`
URL of a image or thumbnail that is displayed in the result item.
.. _macro result_sub_header:
``result_sub_header``
---------------------
Execpt ``image.html`` and some others this macro is used in nearly all result
types in the :ref:`main result list`.
Fields used in the template :origin:`macro result_sub_header
<searx/templates/simple/macros.html>`:
publishedDate : :py:obj:`datetime.datetime`
The date on which the object was published.
length: :py:obj:`time.struct_time`
Playing duration in seconds.
views: :py:class:`str`
View count in humanized number format.
author : :py:class:`str`
Author of the title.
metadata : :py:class:`str`
Miscellaneous metadata.
.. _engine_data:
``engine_data_form``
--------------------
The ``engine_data_form`` macro is used in :origin:`results,html
<searx/templates/simple/results.html>` in a HTML ``<form/>`` element. The
intention of this macro is to pass data of a engine from one :py:obj:`response
<searx.engines.demo_online.response>` to the :py:obj:`searx.search.SearchQuery`
of the next :py:obj:`request <searx.engines.demo_online.request>`.
To pass data, engine's response handler can append result items of typ
``engine_data``. This is by example used to pass a token from the response to
the next request:
.. code:: python
def response(resp):
...
results.append({
'engine_data': token,
'key': 'next_page_token',
})
...
return results
def request(query, params):
page_token = params['engine_data'].get('next_page_token')
.. _main result list:
Main Result List
================
The **media types** of the **main result type** are the template files in
the :origin:`result_templates <searx/templates/simple/result_templates>`.
.. _template default:
``default.html``
----------------
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`default.html
<searx/templates/simple/result_templates/default.html>`:
content : :py:class:`str`
General text of the result item.
iframe_src : :py:class:`str`
URL of an embedded ``<iframe>`` / the frame is collapsible.
audio_src : uri,
URL of an embedded ``<audio controls>``.
.. _template images:
``images.html``
---------------
The images are displayed as small thumbnails in the main results list.
title : :py:class:`str`
Title of the image.
thumbnail_src : :py:class:`str`
URL of a preview of the image.
resolution :py:class:`str`
The resolution of the image (e.g. ``1920 x 1080`` pixel)
Image labels
~~~~~~~~~~~~
Clicking on the preview opens a gallery view in which all further metadata for
the image is displayed. Addition fields used in the :origin:`images.html
<searx/templates/simple/result_templates/images.html>`:
img_src : :py:class:`str`
URL of the full size image.
content: :py:class:`str`
Description of the image.
author: :py:class:`str`
Name of the author of the image.
img_format : :py:class:`str`
The format of the image (e.g. ``png``).
source : :py:class:`str`
Source of the image.
filesize: :py:class:`str`
Size of bytes in :py:obj:`human readable <searx.humanize_bytes>` notation
(e.g. ``MB`` for 1024 \* 1024 Bytes filesize).
url : :py:class:`str`
URL of the page from where the images comes from (source).
.. _template videos:
``videos.html``
---------------
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`videos.html
<searx/templates/simple/result_templates/videos.html>`:
iframe_src : :py:class:`str`
URL of an embedded ``<iframe>`` / the frame is collapsible.
The videos are displayed as small thumbnails in the main results list, there
is an additional button to collaps/open the embeded video.
content : :py:class:`str`
Description of the code fragment.
.. _template torrent:
``torrent.html``
----------------
.. _magnet link: https://en.wikipedia.org/wiki/Magnet_URI_scheme
.. _torrent file: https://en.wikipedia.org/wiki/Torrent_file
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`torrent.html
<searx/templates/simple/result_templates/torrent.html>`:
magnetlink:
URL of the `magnet link`_.
torrentfile
URL of the `torrent file`_.
seed : ``int``
Number of seeders.
leech : ``int``
Number of leecher
filesize : ``int``
Size in Bytes (rendered to human readable unit of measurement).
files : ``int``
Number of files.
.. _template map:
``map.html``
------------
.. _GeoJSON: https://en.wikipedia.org/wiki/GeoJSON
.. _Leaflet: https://github.com/Leaflet/Leaflet
.. _bbox: https://wiki.openstreetmap.org/wiki/Bounding_Box
.. _HTMLElement.dataset: https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/dataset
.. _Nominatim: https://nominatim.org/release-docs/latest/
.. _Lookup: https://nominatim.org/release-docs/latest/api/Lookup/
.. _place_id is not a persistent id:
https://nominatim.org/release-docs/latest/api/Output/#place_id-is-not-a-persistent-id
.. _perma_id: https://wiki.openstreetmap.org/wiki/Permanent_ID
.. _country code: https://wiki.openstreetmap.org/wiki/Country_code
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`map.html
<searx/templates/simple/result_templates/map.html>`:
content : :py:class:`str`
Description of the item.
address_label : :py:class:`str`
Label of the address / default ``_('address')``.
geojson : GeoJSON_
Geometries mapped to HTMLElement.dataset_ (``data-map-geojson``) and used by
Leaflet_.
boundingbox : ``[ min-lon, min-lat, max-lon, max-lat]``
A bbox_ area defined by min longitude , min latitude , max longitude and max
latitude. The bounding box is mapped to HTMLElement.dataset_
(``data-map-boundingbox``) and is used by Leaflet_.
longitude, latitude : :py:class:`str`
Geographical coordinates, mapped to HTMLElement.dataset_ (``data-map-lon``,
``data-map-lat``) and is used by Leaflet_.
address : ``{...}``
A dicticonary with the address data:
.. code:: python
address = {
'name' : str, # name of object
'road' : str, # street name of object
'house_number' : str, # house number of object
'postcode' : str, # postcode of object
'country' : str, # country of object
'country_code' : str,
'locality' : str,
}
country_code : :py:class:`str`
`Country code`_ of the object.
locality : :py:class:`str`
The name of the city, town, township, village, borough, etc. in which this
object is located.
links : ``[link1, link2, ...]``
A list of links with labels:
.. code:: python
links.append({
'label' : str,
'url' : str,
'url_label' : str, # set by some engines but unused (oscar)
})
data : ``[data1, data2, ...]``
A list of additional data, shown in two columns and containing a label and
value.
.. code:: python
data.append({
'label' : str,
'value' : str,
'key' : str, # set by some engines but unused
})
type : :py:class:`str` # set by some engines but unused (oscar)
Tag label from :ref:`OSM_KEYS_TAGS['tags'] <update_osm_keys_tags.py>`.
type_icon : :py:class:`str` # set by some engines but unused (oscar)
Type's icon.
osm : ``{...}``
OSM-type and OSM-ID, can be used to Lookup_ OSM data (Nominatim_). There is
also a discussion about "`place_id is not a persistent id`_" and the
perma_id_.
.. code:: python
osm = {
'type': str,
'id': str,
}
type : :py:class:`str`
Type of osm-object (if OSM-Result).
id :
ID of osm-object (if OSM-Result).
.. hint::
The ``osm`` property is set by engine ``openstreetmap.py``, but it is not
used in the ``map.html`` template yet.
.. _template paper:
``paper.html``
--------------
.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
Displays result fields from:
- :ref:`macro result_header`
Additional fields used in the :origin:`paper.html
<searx/templates/simple/result_templates/paper.html>`:
content : :py:class:`str`
An abstract or excerpt from the document.
comments : :py:class:`str`
Free text display in italic below the content.
tags : :py:class:`List <list>`\ [\ :py:class:`str`\ ]
Free tag list.
type : :py:class:`str`
Short description of medium type, e.g. *book*, *pdf* or *html* ...
authors : :py:class:`List <list>`\ [\ :py:class:`str`\ ]
List of authors of the work (authors with a "s" suffix, the "author" is in the
:ref:`macro result_sub_header`).
editor : :py:class:`str`
Editor of the book/paper.
publisher : :py:class:`str`
Name of the publisher.
journal : :py:class:`str`
Name of the journal or magazine the article was published in.
volume : :py:class:`str`
Volume number.
pages : :py:class:`str`
Page range where the article is.
number : :py:class:`str`
Number of the report or the issue number for a journal article.
doi : :py:class:`str`
DOI number (like ``10.1038/d41586-018-07848-2``).
issn : :py:class:`List <list>`\ [\ :py:class:`str`\ ]
ISSN number like ``1476-4687``
isbn : :py:class:`List <list>`\ [\ :py:class:`str`\ ]
ISBN number like ``9780201896831``
pdf_url : :py:class:`str`
URL to the full article, the PDF version
html_url : :py:class:`str`
URL to full article, HTML version
.. _template packages:
``packages``
------------
Displays result fields from:
- :ref:`macro result_header`
Additional fields used in the :origin:`packages.html
<searx/templates/simple/result_templates/packages.html>`:
package_name : :py:class:`str`
The name of the package.
version : :py:class:`str`
The current version of the package.
maintainer : :py:class:`str`
The maintainer or author of the project.
publishedDate : :py:class:`datetime <datetime.datetime>`
Date of latest update or release.
tags : :py:class:`List <list>`\ [\ :py:class:`str`\ ]
Free tag list.
popularity : :py:class:`str`
The popularity of the package, e.g. rating or download count.
license_name : :py:class:`str`
The name of the license.
license_url : :py:class:`str`
The web location of a license copy.
homepage : :py:class:`str`
The url of the project's homepage.
source_code_url: :py:class:`str`
The location of the project's source code.
links : :py:class:`dict`
Additional links in the form of ``{'link_name': 'http://example.com'}``
.. _template code:
``code.html``
-------------
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`code.html
<searx/templates/simple/result_templates/code.html>`:
content : :py:class:`str`
Description of the code fragment.
codelines : ``[line1, line2, ...]``
Lines of the code fragment.
code_language : :py:class:`str`
Name of the code language, the value is passed to
:py:obj:`pygments.lexers.get_lexer_by_name`.
repository : :py:class:`str`
URL of the repository of the code fragment.
.. _template files:
``files.html``
--------------
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`code.html
<searx/templates/simple/result_templates/files.html>`:
filename, size, time: :py:class:`str`
Filename, Filesize and Date of the file.
mtype : ``audio`` | ``video`` | :py:class:`str`
Mimetype type of the file.
subtype : :py:class:`str`
Mimetype / subtype of the file.
abstract : :py:class:`str`
Abstract of the file.
author : :py:class:`str`
Name of the author of the file
embedded : :py:class:`str`
URL of an embedded media type (``audio`` or ``video``) / is collapsible.
.. _template products:
``products.html``
-----------------
Displays result fields from:
- :ref:`macro result_header` and
- :ref:`macro result_sub_header`
Additional fields used in the :origin:`products.html
<searx/templates/simple/result_templates/products.html>`:
content : :py:class:`str`
Description of the product.
price : :py:class:`str`
The price must include the currency.
shipping : :py:class:`str`
Shipping details.
source_country : :py:class:`str`
Place from which the shipment is made.
.. _template answer results:
Answer results
==============
See :ref:`result_types.answer`
Suggestion results
==================
See :ref:`result_types.suggestion`
Correction results
==================
See :ref:`result_types.corrections`
Infobox results
===============
See :ref:`result_types.infobox`

View file

@ -0,0 +1,81 @@
.. _translation:
===========
Translation
===========
.. _translate.codeberg.org: https://translate.codeberg.org/projects/searxng/
.. _Weblate: https://docs.weblate.org
.. _translations branch: https://github.com/searxng/searxng/tree/translations
.. _orphan branch: https://git-scm.com/docs/git-checkout#Documentation/git-checkout.txt---orphanltnewbranchgt
.. _Weblate repository: https://translate.codeberg.org/projects/searxng/searxng/#repository
.. _wlc: https://docs.weblate.org/en/latest/wlc.html
.. |translated| image:: https://translate.codeberg.org/widgets/searxng/-/searxng/svg-badge.svg
:target: https://translate.codeberg.org/projects/searxng/
.. sidebar:: |translated|
- :ref:`searx.babel_extract`
- Weblate_
- SearXNG `translations branch`_
- SearXNG `Weblate repository`_
- Weblate Client: wlc_
- Babel Command-Line: `pybabel <http://babel.pocoo.org/en/latest/cmdline.html>`_
- `weblate workflow <https://docs.weblate.org/en/latest/workflows.html>`_
Translation takes place on translate.codeberg.org_.
Translations which has been added by translators on the translate.codeberg.org_ UI are
committed to Weblate's counterpart of the SearXNG *origin* repository which is
located at ``https://translate.codeberg.org/git/searxng/searxng``.
There is no need to clone this repository, :ref:`SearXNG Weblate workflow` take
care of the synchronization with the *origin*. To avoid merging commits from
the counterpart directly on the ``master`` branch of *SearXNG origin*, a *pull
request* (PR) is created by this workflow.
Weblate monitors the `translations branch`_, not the ``master`` branch. This
branch is an `orphan branch`_, decoupled from the master branch (we already know
orphan branches from the ``gh-pages``). The `translations branch`_ contains
only the
- ``translation/messages.pot`` and the
- ``translation/*/messages.po`` files, nothing else.
.. _SearXNG Weblate workflow:
.. figure:: translation.svg
SearXNG's PR workflow to be in sync with Weblate
Sync from *origin* to *weblate*: using ``make weblate.push.translations``
For each commit on the ``master`` branch of SearXNG *origin* the GitHub job
:origin:`babel / Update translations branch
<.github/workflows/integration.yml>` checks for updated translations.
Sync from *weblate* to *origin*: using ``make weblate.translations.commit``
Every Friday, the GitHub workflow :origin:`babel / create PR for additions from
weblate <.github/workflows/translations-update.yml>` creates a PR with the
updated translation files:
- ``translation/messages.pot``,
- ``translation/*/messages.po`` and
- ``translation/*/messages.mo``
wlc
===
.. _wlc configuration: https://docs.weblate.org/en/latest/wlc.html#wlc-config
.. _API key: https://translate.codeberg.org/accounts/profile/#api
All weblate integration is done by GitHub workflows, but if you want to use wlc_,
copy this content into `wlc configuration`_ in your HOME ``~/.config/weblate``
.. code-block:: ini
[keys]
https://translate.codeberg.org/api/ = APIKEY
Replace ``APIKEY`` by your `API key`_.