master #2

Merged
kevadesu merged 46 commits from Icycoide/searxng:master into master 2025-05-31 13:31:47 +02:00
Owner

What does this PR do?

Why is this change important?

How to test this PR locally?

Author's checklist

## What does this PR do? <!-- MANDATORY --> <!-- explain the changes in your PR, algorithms, design, architecture --> ## Why is this change important? <!-- MANDATORY --> <!-- explain the motivation behind your PR --> ## How to test this PR locally? <!-- commands to run the tests or instructions to test the changes --> ## Author's checklist <!-- additional notes for reviewers --> ## Related issues <!-- Closes #234 -->
kevadesu added 46 commits 2025-05-31 13:31:36 +02:00
Wolfi OS images are specifically designed for container use. Using a specially designed base image for containers not only reduces maintenance burdens, but improves overall experience for developers (fewer packages we have to track) and end users (smaller images).

Discussion here: https://github.com/searxng/searxng/issues/4753
Bumps the minor group with 1 update: [typer-slim](https://github.com/fastapi/typer).


Updates `typer-slim` from 0.15.3 to 0.15.4
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.15.3...0.15.4)

---
updated-dependencies:
- dependency-name: typer-slim
  dependency-version: 0.15.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Instead of using Wolfi base images from cgr.dev and making that mess on the Dockerfile, why don't we build the base images ourselves from Wolfi repos with apko? The intention of this is to simplify the main Dockerfile and avoid having to patch the base image every time, it also simplifies some steps like image ownership management and provides extremely fast builds.
Reported-by: @unifox https://github.com/searxng/searxng/issues/2722#issuecomment-2884993248

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Bumps [vite-plugin-static-copy](https://github.com/sapphi-red/vite-plugin-static-copy) from 2.3.1 to 3.0.0.
- [Release notes](https://github.com/sapphi-red/vite-plugin-static-copy/releases)
- [Changelog](https://github.com/sapphi-red/vite-plugin-static-copy/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sapphi-red/vite-plugin-static-copy/compare/vite-plugin-static-copy@2.3.1...vite-plugin-static-copy@3.0.0)
The wolfi-base metapackage includes busybox, ca-certificates-bundle and the package manager. The change is to make the use of base-builder image more flexible.
That entrypoint is prone to screw things up, especially with permission handling. The new script handles initialization better and fixes some issues like delayed settings update via ENVs and timestamp overwriting, also adjusts what should be copied into the container.

Related https://github.com/searxng/searxng/pull/4721#issuecomment-2850272129
The global variable CACHE is not initialized when DDG images or DDG videos
import the get_vqd() function (please remember: the engine modules are imported
using the importlib method and not via the `import` keyword).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Icons category makes sense because it allows to quickly search for free SVG
icons to use for websites / other designs with a quick `!icons` query

Icons don't seem to fit into the normal images category that well because icons
are quite a special type of images
- uxwing provides attribution-free icons to use for design projects
- svgrepo was my go-to before, but it's ratelimiting a lot recently
- apparently the API now requires a `X-Pinterest-PWS-Handler` in order to
  properly function (extracted from their web UI)

- the other `X-Pinterest` headers here are added in case they become mandatory
  too

Closes: https://github.com/searxng/searxng/issues/4812
Temporarily remove the -e flag from set to prevent entrypoint.sh from stopping execution if any command returns a non-zero status. This doesn't solve anything but relaxes the script checks.

Related https://github.com/searxng/searxng/issues/4818
- z-library https://github.com/searxng/searxng/issues/3610
- library of congress: https://github.com/searxng/searxng/issues/4810
- qwant: https://github.com/searxng/searxng/issues/3929

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Refactor Naver engine (Web, News, Images, Videos, Autocomplete)

- ref: https://search.naver.com/
- lang: `ko`
- Wikidata: https://www.wikidata.org/wiki/Q485639

Co-authored-by: Bnyro <bnyro@tutanota.com>
The previously used library is unmaintained for 6 years now [1] and the solution
had know issues [2][3]

[1] https://github.com/searxng/searxng/pull/4284#discussion_r1954493434
[2] https://github.com/searxng/searxng/pull/4318#issuecomment-2731576657
[3] https://github.com/privau/searxng/issues/56
In the previous implementation, all databases were loaded into memory when
importing the searx.data package, regardless of whether they were ever needed.

Regardless of this, it is an antipattern to load entire databases into memory
when importing a package or module; databases should be loaded when needed.

Lazy loading is a first step toward improving memory usage and also improves
performance when setting up the runtime environment.  Building on this,
subsequent PRs will be able to further optimize memory behavior, e.g., by using
a real database application such as the one already available via

    searx.cache.ExpireCache

Related:

- https://github.com/searxng/searxng/discussions/1892
- https://github.com/searxng/searxng/pull/3458
- https://github.com/searxng/searxng/pull/4650

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
There is currently no known z-library, and all known URLs are dead [1]. To avoid
interrupting automated updates, a connection error to a z-library is treated as
a *known error*, and the old properties of the z-library are retained.

[1] https://github.com/searxng/searxng/issues/3610

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Changes:
- Add trailing slash to base URL to prevent potential redirects
- Remove advanced search syntax filtering (no longer guarantees a CAPTCHA)
- Correct pagination offset calculation: Page 2 now starts at offset 10,
  subsequent pages use 10 + (n-2)*15 formula instead of the previous
  broken 20 + (n-2)*50 calculation that caused CAPTCHAs
- Restructure request parameter building to better match a real request
- "kt" cookie is no longer an empty string if the language/region is "all"
- Group related parameter assignments together
- Add header logging to debugging output

Related:

- https://github.com/searxng/searxng/issues/4824
Sometimes (e.g. when ddg does not have a result item) there is no content and
the engine will fail with an IndexError:

  * Error: IndexError
  * Percentage: 10
  * Parameters: `()`
  * File name: `searx/engines/duckduckgo.py:375`
  * Function: `response`
  * Code: `item["content"] = extract_text(eval_xpath(div_result, './/a[contains(@class, "result__snippet")]')[0])`

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The previous implementation for determining the description of an engine did not
take into account that the UI languages ​​can also have a region tag and/or a
script tag:

    el-GR:      Ελληνικά, Ελλάδα (Greek, Greece)
    fa-IR:      فارسی, ایران (Persian, Iran)
    nb-NO:      Norsk bokmål, Norge (Norwegian bokmål, Norway)
    nl-BE:      Nederlands, België (Dutch, Belgium)
    pt-BR:      Português, Brasil (Portuguese, Brazil)
    zh-HK:      中文, 中國香港特別行政區 (Chinese, Hong Kong SAR China)
    zh-Hans-CN: 中文, 中国 (Chinese, China)
    zh-Hant-TW: 中文, 台灣 (Chinese, Taiwan)

Closes: https://github.com/searxng/searxng/issues/4842

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Avoid a confusing warning:

    DeprecationWarning: Setting per-request cookies=<...> is being deprecated

Code based on httpx unit test [1]

[1] 6a99f6f2b3/tests/client/test_cookies.py (L123-L137)

Closes: https://github.com/searxng/searxng/issues/4833
BTW fix issue reported in [1]

[1] https://github.com/searxng/searxng/pull/4814#issuecomment-2896948787

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Bnyro <bnyro@tutanota.com>
51e0a154c - 2025-05-22 - Priit Jõerüüt <jrtcdbrg@noreply.codeberg.org>
256b8cb8e - 2025-05-16 - wetinee <wetinee@noreply.codeberg.org>
ac12b76a8 - 2025-05-16 - wetinee <wetinee@noreply.codeberg.org>

Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
To reduce the memory footprint, this patch no longer loads the JSON data
completely into memory.  Instead, there is an SQL database based on
`ExpireCacheSQLite`.

The class CurrenciesDB is a simple DB application that encapsulates the
DB (queries and initialization) and provides convenient methods like
`name_to_iso4217` and `iso4217_to_name`.

Related:

- https://github.com/searxng/searxng/discussions/1892
- https://github.com/searxng/searxng/pull/3458#issuecomment-2900807671
- https://github.com/searxng/searxng/pull/4650

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Excludes some workflows/jobs unneeded on forks.

Closes https://github.com/searxng/searxng/issues/4847
This has been a regression introduced with the removal of
the unmaintained autocomplete.js library.

We should only focus the search bar on the main search page at `/`
and not at the results page located at `/search`.

I'm not sure if there's a better way to figure out if
we're on the results page than checking if the id of the
main element is `#main_results`, checking the path
obviously isn't a better solution because it can differ
depending on the instance / reverse proxy / ....

- related to 32823ecb69
- closes https://github.com/searxng/searxng/issues/4846
Why?
- presearch requires the response cookies of the first request to be sent within the second request
- otherwise we miss auth information and the engine doesn't work

Related:
- https://github.com/searxng/searxng/pull/4858
- closes https://github.com/searxng/searxng/issues/4854

Co-authored-by: Aadniz <8147434+Aadniz@users.noreply.github.com>
Fix C&P typo from  https://github.com/searxng/searxng/pull/4836

Closes: https://github.com/searxng/searxng/issues/4861
Bumps the minor group in /client/simple with 4 updates: [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js), [eslint](https://github.com/eslint/eslint), [sharp](https://github.com/lovell/sharp) and [webpack](https://github.com/webpack/webpack).


Updates `@eslint/js` from 9.26.0 to 9.27.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/commits/v9.27.0/packages/js)

Updates `eslint` from 9.26.0 to 9.27.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v9.26.0...v9.27.0)

Updates `sharp` from 0.34.1 to 0.34.2
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.34.1...v0.34.2)

Updates `webpack` from 5.99.8 to 5.99.9
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.99.8...v5.99.9)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.27.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: eslint
  dependency-version: 9.27.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: sharp
  dependency-version: 0.34.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: webpack
  dependency-version: 5.99.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
0f55f2438 - 2025-05-29 - sandijs <sandijs@noreply.codeberg.org>
109f7dcb8 - 2025-05-26 - ngf <ngf@noreply.codeberg.org>
d2ba9b3d0 - 2025-05-26 - artnay <artnay@noreply.codeberg.org>
33c6f23bf - 2025-05-26 - realkendrick_fr <realkendrick_fr@noreply.codeberg.org>
Bumps the minor group in /client/simple with 2 updates: [globals](https://github.com/sindresorhus/globals) and [stylelint](https://github.com/stylelint/stylelint).


Updates `globals` from 16.1.0 to 16.2.0
- [Release notes](https://github.com/sindresorhus/globals/releases)
- [Commits](https://github.com/sindresorhus/globals/compare/v16.1.0...v16.2.0)

Updates `stylelint` from 16.19.1 to 16.20.0
- [Release notes](https://github.com/stylelint/stylelint/releases)
- [Changelog](https://github.com/stylelint/stylelint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/stylelint/stylelint/compare/16.19.1...16.20.0)

---
updated-dependencies:
- dependency-name: globals
  dependency-version: 16.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: stylelint
  dependency-version: 16.20.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps the minor group with 2 updates: [selenium](https://github.com/SeleniumHQ/Selenium) and [typer-slim](https://github.com/fastapi/typer).


Updates `selenium` from 4.32.0 to 4.33.0
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.32.0...selenium-4.33.0)

Updates `typer-slim` from 0.15.4 to 0.16.0
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.15.4...0.16.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-version: 4.33.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: typer-slim
  dependency-version: 0.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
kevadesu merged commit 7182190057 into master 2025-05-31 13:31:47 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: kevadesu/costmikuSearch#2
No description provided.