mirror of
https://github.com/searxng/searxng.git
synced 2025-08-02 01:52:21 +02:00
[doc] build from commit ff2e0ea278
This commit is contained in:
inetol
commit
2d9f6712e2
429 changed files with 98580 additions and 0 deletions
254
_sources/dev/engines/engine_overview.rst.txt
Normal file
254
_sources/dev/engines/engine_overview.rst.txt
Normal file
|
|
@ -0,0 +1,254 @@
|
|||
.. _engines-dev:
|
||||
|
||||
===============
|
||||
Engine Overview
|
||||
===============
|
||||
|
||||
.. contents::
|
||||
:depth: 3
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`configured engines`
|
||||
- :ref:`settings engines`
|
||||
|
||||
SearXNG is a metasearch-engine_, so it uses different search engines to provide
|
||||
better results.
|
||||
|
||||
Because there is no general search API which could be used for every search
|
||||
engine, an adapter has to be built between SearXNG and the external search
|
||||
engines. Adapters are stored under the folder :origin:`searx/engines`.
|
||||
|
||||
.. _general engine configuration:
|
||||
|
||||
General Engine Configuration
|
||||
============================
|
||||
|
||||
It is required to tell SearXNG the type of results the engine provides. The
|
||||
arguments can be set in the engine file or in the settings file (normally
|
||||
``settings.yml``). The arguments in the settings file override the ones in the
|
||||
engine file.
|
||||
|
||||
It does not matter if an option is stored in the engine file or in the settings.
|
||||
However, the standard way is the following:
|
||||
|
||||
.. _engine file:
|
||||
|
||||
Engine File
|
||||
-----------
|
||||
|
||||
.. table:: Common options in the engine module
|
||||
:width: 100%
|
||||
|
||||
======================= =========== ========================================================
|
||||
argument type information
|
||||
======================= =========== ========================================================
|
||||
categories list categories, in which the engine is working
|
||||
paging boolean support multiple pages
|
||||
time_range_support boolean support search time range
|
||||
engine_type str - ``online`` :ref:`[ref] <online engines>` by
|
||||
default, other possibles values are:
|
||||
- ``offline`` :ref:`[ref] <offline engines>`
|
||||
- ``online_dictionary`` :ref:`[ref] <online dictionary>`
|
||||
- ``online_currency`` :ref:`[ref] <online currency>`
|
||||
- ``online_url_search`` :ref:`[ref] <online url search>`
|
||||
======================= =========== ========================================================
|
||||
|
||||
.. _engine settings:
|
||||
|
||||
Engine ``settings.yml``
|
||||
-----------------------
|
||||
|
||||
For a more detailed description, see :ref:`settings engines` in the :ref:`settings.yml`.
|
||||
|
||||
.. table:: Common options in the engine setup (``settings.yml``)
|
||||
:width: 100%
|
||||
|
||||
======================= =========== ==================================================
|
||||
argument type information
|
||||
======================= =========== ==================================================
|
||||
name string name of search-engine
|
||||
engine string name of searxng-engine (file name without ``.py``)
|
||||
enable_http bool enable HTTP (by default only HTTPS is enabled).
|
||||
shortcut string shortcut of search-engine
|
||||
timeout string specific timeout for search-engine
|
||||
display_error_messages boolean display error messages on the web UI
|
||||
proxies dict set proxies for a specific engine
|
||||
(e.g. ``proxies : {http: socks5://proxy:port,
|
||||
https: socks5://proxy:port}``)
|
||||
======================= =========== ==================================================
|
||||
|
||||
.. _engine overrides:
|
||||
|
||||
Overrides
|
||||
---------
|
||||
|
||||
A few of the options have default values in the namespace of the engine's python
|
||||
module, but are often overwritten by the settings. If ``None`` is assigned to an
|
||||
option in the engine file, it has to be redefined in the settings, otherwise
|
||||
SearXNG will not start with that engine (global names with a leading underline can
|
||||
be ``None``).
|
||||
|
||||
Here is an very simple example of the global names in the namespace of engine's
|
||||
module:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# engine dependent config
|
||||
categories = ['general']
|
||||
paging = True
|
||||
_non_overwritten_global = 'foo'
|
||||
|
||||
|
||||
.. table:: The naming of overrides is arbitrary / recommended overrides are:
|
||||
:width: 100%
|
||||
|
||||
======================= =========== ===========================================
|
||||
argument type information
|
||||
======================= =========== ===========================================
|
||||
base_url string base-url, can be overwritten to use same
|
||||
engine on other URL
|
||||
number_of_results int maximum number of results per request
|
||||
language string ISO code of language and country like en_US
|
||||
api_key string api-key if required by engine
|
||||
======================= =========== ===========================================
|
||||
|
||||
.. _engine request:
|
||||
|
||||
Making a Request
|
||||
================
|
||||
|
||||
To perform a search an URL have to be specified. In addition to specifying an
|
||||
URL, arguments can be passed to the query.
|
||||
|
||||
.. _engine request arguments:
|
||||
|
||||
Passed Arguments (request)
|
||||
--------------------------
|
||||
|
||||
These arguments can be used to construct the search query. Furthermore,
|
||||
parameters with default value can be redefined for special purposes.
|
||||
|
||||
.. _engine request online:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online
|
||||
<searx.search.processors.online.OnlineProcessor.get_params>`
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
url str ``''``
|
||||
method str ``'GET'``
|
||||
headers set ``{}``
|
||||
data set ``{}``
|
||||
cookies set ``{}``
|
||||
verify bool ``True``
|
||||
headers.User-Agent str a random User-Agent
|
||||
category str current category, like ``'general'``
|
||||
safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
|
||||
time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
|
||||
pageno int current pagenumber
|
||||
searxng_locale str SearXNG's locale selected by user. Specific language code like
|
||||
``'en'``, ``'en-US'``, or ``'all'`` if unspecified.
|
||||
====================== ============== ========================================================================
|
||||
|
||||
|
||||
.. _engine request online_dictionary:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online_dictionary
|
||||
<searx.search.processors.online_dictionary.OnlineDictionaryProcessor.get_params>`,
|
||||
in addition to the :ref:`online <engine request online>` arguments:
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
from_lang str specific language code like ``'en_US'``
|
||||
to_lang str specific language code like ``'en_US'``
|
||||
query str the text query without the languages
|
||||
====================== ============== ========================================================================
|
||||
|
||||
.. _engine request online_currency:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online_currency
|
||||
<searx.search.processors.online_currency.OnlineCurrencyProcessor.get_params>`,
|
||||
in addition to the :ref:`online <engine request online>` arguments:
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
amount float the amount to convert
|
||||
from str ISO 4217 code
|
||||
to str ISO 4217 code
|
||||
from_name str currency name
|
||||
to_name str currency name
|
||||
====================== ============== ========================================================================
|
||||
|
||||
.. _engine request online_url_search:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online_url_search
|
||||
<searx.search.processors.online_url_search.OnlineUrlSearchProcessor.get_params>`,
|
||||
in addition to the :ref:`online <engine request online>` arguments:
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
search_url dict URLs from the search query:
|
||||
|
||||
.. code:: python
|
||||
|
||||
{
|
||||
'http': str,
|
||||
'ftp': str,
|
||||
'data:image': str
|
||||
}
|
||||
====================== ============== ========================================================================
|
||||
|
||||
Specify Request
|
||||
---------------
|
||||
|
||||
The function :py:func:`def request(query, params):
|
||||
<searx.engines.demo_online.request>` always returns the ``params`` variable, the
|
||||
following parameters can be used to specify a search request:
|
||||
|
||||
.. table::
|
||||
:width: 100%
|
||||
|
||||
=================== =========== ==========================================================================
|
||||
argument type information
|
||||
=================== =========== ==========================================================================
|
||||
url str requested url
|
||||
method str HTTP request method
|
||||
headers set HTTP header information
|
||||
data set HTTP data information
|
||||
cookies set HTTP cookies
|
||||
verify bool Performing SSL-Validity check
|
||||
allow_redirects bool Follow redirects
|
||||
max_redirects int maximum redirects, hard limit
|
||||
soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
|
||||
raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
|
||||
=================== =========== ==========================================================================
|
||||
|
||||
|
||||
Making a Response
|
||||
=================
|
||||
|
||||
In the ``response`` function of the engine, the HTTP response (``resp``) is
|
||||
parsed and a list of results is returned.
|
||||
|
||||
A engine can append result-items of different media-types and different
|
||||
result-types to the result list. The list of the result items is render to HTML
|
||||
by templates. For more details read section:
|
||||
|
||||
- :ref:`simple theme templates`
|
||||
- :ref:`result types`
|
||||
|
||||
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue