kevadesu/costmikuSearch

Fork 0

forked from Icycoide/searxng

master #2

Merged

kevadesu merged 46 commits from Icycoide/searxng:master into master

2025-05-31 13:31:47 +02:00

kevadesu commented

2025-05-31 13:31:34 +02:00

Owner

What does this PR do?

Why is this change important?

How to test this PR locally?

Author's checklist

## What does this PR do?   ## Why is this change important?   ## How to test this PR locally?  ## Author's checklist  ## Related issues

kevadesu added 46 commits 2025-05-31 13:31:36 +02:00

[enh] container: use Wolfi OS as base image 03083f07da

Wolfi OS images are specifically designed for container use. Using a specially designed base image for containers not only reduces maintenance burdens, but improves overall experience for developers (fewer packages we have to track) and end users (smaller images).

Discussion here: https://github.com/searxng/searxng/issues/4753

[upd] pypi: Bump typer-slim from 0.15.3 to 0.15.4 in the minor group (#4789 ) b8682ffc69

Bumps the minor group with 1 update: [typer-slim](https://github.com/fastapi/typer).


Updates `typer-slim` from 0.15.3 to 0.15.4
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.15.3...0.15.4)

---
updated-dependencies:
- dependency-name: typer-slim
  dependency-version: 0.15.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[l10n] update translations from Weblate (#4788 ) 1b08324f26

[mod] container: build custom base images (#4799 ) 86373e7c87

Instead of using Wolfi base images from cgr.dev and making that mess on the Dockerfile, why don't we build the base images ourselves from Wolfi repos with apko? The intention of this is to simplify the main Dockerfile and avoid having to patch the base image every time, it also simplifies some steps like image ownership management and provides extremely fast builds.

[mod] engine invidious: commented out / no public API available nowadays (#4800 ) b8b857d24c

Reported-by: @unifox https://github.com/searxng/searxng/issues/2722#issuecomment-2884993248

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[upd] web-client (simple): Bump vite-plugin-static-copy (#4791 ) 5dff826937

Bumps [vite-plugin-static-copy](https://github.com/sapphi-red/vite-plugin-static-copy) from 2.3.1 to 3.0.0.
- [Release notes](https://github.com/sapphi-red/vite-plugin-static-copy/releases)
- [Changelog](https://github.com/sapphi-red/vite-plugin-static-copy/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sapphi-red/vite-plugin-static-copy/compare/vite-plugin-static-copy@2.3.1...vite-plugin-static-copy@3.0.0)

[fix] container: base-builder should have essentials a195f52412

The wolfi-base metapackage includes busybox, ca-certificates-bundle and the package manager. The change is to make the use of base-builder image more flexible.

[mod] container: refactor entrypoint script 749de829d5

That entrypoint is prone to screw things up, especially with permission handling. The new script handles initialization better and fixes some issues like delayed settings update via ENVs and timestamp overwriting, also adjusts what should be copied into the container.

Related https://github.com/searxng/searxng/pull/4721#issuecomment-2850272129

[fix] duckduckgo engines: issue when get_vqd() is used by ddg-images and ddg-videos (#4809 ) ca67f1dffe

The global variable CACHE is not initialized when DDG images or DDG videos
import the get_vqd() function (please remember: the engine modules are imported
using the importlib method and not via the `import` keyword).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[mod] upgrade to httpx 0.28.1 (#4674 ) 7a3742ae56

[fix] yahoo: url and title xpath 6ec554cb5b

[chore] fix some docstring typos (#4815 ) 7420706a50

[refactor] icon engines: add new icon category (#4817 ) 8bff73c9b6

Icons category makes sense because it allows to quickly search for free SVG
icons to use for websites / other designs with a quick `!icons` query

Icons don't seem to fit into the normal images category that well because icons
are quite a special type of images

[feat] engines: add uxwing engine for icons (#4819 ) 88973f5431

- uxwing provides attribution-free icons to use for design projects
- svgrepo was my go-to before, but it's ratelimiting a lot recently

[fix] pinterest: engine broken due to API changes (#4816 ) 502017b901

- apparently the API now requires a `X-Pinterest-PWS-Handler` in order to
  properly function (extracted from their web UI)

- the other `X-Pinterest` headers here are added in case they become mandatory
  too

Closes: https://github.com/searxng/searxng/issues/4812

[mod] container: remove -e flag 9ffe23ecf3

Temporarily remove the -e flag from set to prevent entrypoint.sh from stopping execution if any command returns a non-zero status. This doesn't solve anything but relaxes the script checks.

Related https://github.com/searxng/searxng/issues/4818

[fix] engines: disable those with known issues (#4813 ) 365b9426f1

- z-library https://github.com/searxng/searxng/issues/3610
- library of congress: https://github.com/searxng/searxng/issues/4810
- qwant: https://github.com/searxng/searxng/issues/3929

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[feat] engines: add Naver engine (#4573 ) 156d1eb8c8

Refactor Naver engine (Web, News, Images, Videos, Autocomplete)

- ref: https://search.naver.com/
- lang: `ko`
- Wikidata: https://www.wikidata.org/wiki/Q485639

Co-authored-by: Bnyro <bnyro@tutanota.com>

[refactor] search.js: use custom auto completion implementation 32823ecb69

The previously used library is unmaintained for 6 years now [1] and the solution
had know issues [2][3]

[1] https://github.com/searxng/searxng/pull/4284#discussion_r1954493434
[2] https://github.com/searxng/searxng/pull/4318#issuecomment-2731576657
[3] https://github.com/privau/searxng/issues/56

[build] /static 861f9c4be5

[mod] searx.data: lazy load of the data objects (databases) (#4834 ) d29cf64ce4

In the previous implementation, all databases were loaded into memory when
importing the searx.data package, regardless of whether they were ever needed.

Regardless of this, it is an antipattern to load entire databases into memory
when importing a package or module; databases should be loaded when needed.

Lazy loading is a first step toward improving memory usage and also improves
performance when setting up the runtime environment.  Building on this,
subsequent PRs will be able to further optimize memory behavior, e.g., by using
a real database application such as the one already available via

    searx.cache.ExpireCache

Related:

- https://github.com/searxng/searxng/discussions/1892
- https://github.com/searxng/searxng/pull/3458
- https://github.com/searxng/searxng/pull/4650

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[fix] searx.data: fetch-traits - z-library (httpx.ConnectError) (#4835 ) 98badc9cd0

There is currently no known z-library, and all known URLs are dead [1]. To avoid
interrupting automated updates, a connection error to a z-library is treated as
a *known error*, and the old properties of the z-library are retained.

[1] https://github.com/searxng/searxng/issues/3610

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[refactor] duckduckgo engine: improve request logic and code structure (#4837 ) 4fa7de8033

Changes:
- Add trailing slash to base URL to prevent potential redirects
- Remove advanced search syntax filtering (no longer guarantees a CAPTCHA)
- Correct pagination offset calculation: Page 2 now starts at offset 10,
  subsequent pages use 10 + (n-2)*15 formula instead of the previous
  broken 20 + (n-2)*50 calculation that caused CAPTCHAs
- Restructure request parameter building to better match a real request
- "kt" cookie is no longer an empty string if the language/region is "all"
- Group related parameter assignments together
- Add header logging to debugging output

Related:

- https://github.com/searxng/searxng/issues/4824

[fix] ddg engine: IndexError exception is raised on empty contend (#4843 ) 1ef5c03962

Sometimes (e.g. when ddg does not have a result item) there is no content and
the engine will fail with an IndexError:

  * Error: IndexError
  * Percentage: 10
  * Parameters: `()`
  * File name: `searx/engines/duckduckgo.py:375`
  * Function: `response`
  * Code: `item["content"] = extract_text(eval_xpath(div_result, './/a[contains(@class, "result__snippet")]')[0])`

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[fix] preferences: description not localized for all UI languages (#4844 ) 230215c250

The previous implementation for determining the description of an engine did not
take into account that the UI languages can also have a region tag and/or a
script tag:

    el-GR:      Ελληνικά, Ελλάδα (Greek, Greece)
    fa-IR:      فارسی, ایران (Persian, Iran)
    nb-NO:      Norsk bokmål, Norge (Norwegian bokmål, Norway)
    nl-BE:      Nederlands, België (Dutch, Belgium)
    pt-BR:      Português, Brasil (Portuguese, Brazil)
    zh-HK:      中文, 中國香港特別行政區 (Chinese, Hong Kong SAR China)
    zh-Hans-CN: 中文, 中国 (Chinese, China)
    zh-Hant-TW: 中文, 台灣 (Chinese, Taiwan)

Closes: https://github.com/searxng/searxng/issues/4842

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[fix] searx.network: don't trigger DeprecationWarning (#4845 ) 9ed9a9aa53

Avoid a confusing warning:

    DeprecationWarning: Setting per-request cookies=<...> is being deprecated

Code based on httpx unit test [1]

[1] 6a99f6f2b3/tests/client/test_cookies.py (L123-L137)

Closes: https://github.com/searxng/searxng/issues/4833

[mod] engines: Yahoo in different languages (#4826 ) 064eb50473

BTW fix issue reported in [1]

[1] https://github.com/searxng/searxng/pull/4814#issuecomment-2896948787

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Bnyro <bnyro@tutanota.com>

[l10n] update translations from Weblate (#4840 ) e46187e3ce

51e0a154c - 2025-05-22 - Priit Jõerüüt <jrtcdbrg@noreply.codeberg.org>
256b8cb8e - 2025-05-16 - wetinee <wetinee@noreply.codeberg.org>
ac12b76a8 - 2025-05-16 - wetinee <wetinee@noreply.codeberg.org>

Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>

[mod] data: implement a simple currencies (SQL) database (#4836 ) 848c8d0544

To reduce the memory footprint, this patch no longer loads the JSON data
completely into memory.  Instead, there is an SQL database based on
`ExpireCacheSQLite`.

The class CurrenciesDB is a simple DB application that encapsulates the
DB (queries and initialization) and provides convenient methods like
`name_to_iso4217` and `iso4217_to_name`.

Related:

- https://github.com/searxng/searxng/discussions/1892
- https://github.com/searxng/searxng/pull/3458#issuecomment-2900807671
- https://github.com/searxng/searxng/pull/4650

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

[mod] ci: exclude some workflows from forks (#4849 ) 14b8a999f3

Excludes some workflows/jobs unneeded on forks.

Closes https://github.com/searxng/searxng/issues/4847

[fix] search: autocomplete focus on results page 9dfdd30da0

This has been a regression introduced with the removal of
the unmaintained autocomplete.js library.

We should only focus the search bar on the main search page at `/`
and not at the results page located at `/search`.

I'm not sure if there's a better way to figure out if
we're on the results page than checking if the id of the
main element is `#main_results`, checking the path
obviously isn't a better solution because it can differ
depending on the instance / reverse proxy / ....

- related to 32823ecb69
- closes https://github.com/searxng/searxng/issues/4846

[build] /static 7a5a499795

[chore] il post: disable by default 20b40351b9

[fix] presearch: reuse response cookies from token extraction 2288f07d62

Why?
- presearch requires the response cookies of the first request to be sent within the second request
- otherwise we miss auth information and the engine doesn't work

Related:
- https://github.com/searxng/searxng/pull/4858
- closes https://github.com/searxng/searxng/issues/4854

Co-authored-by: Aadniz <8147434+Aadniz@users.noreply.github.com>

[data] update searx.data - update_engine_descriptions.py 70a387a48f

[data] update searx.data - update_engine_traits.py 9d8592afd7

[data] update searx.data - update_currencies.py b2727e1be9

[data] update searx.data - update_wikidata_units.py dbe6a6f4fa

[data] update searx.data - update_firefox_version.py 23396f5a1d

[data] update searx.data - update_ahmia_blacklist.py 7448a18efa

[fix] plugins: currency processor bug 913dfbde3c

Fix C&P typo from  https://github.com/searxng/searxng/pull/4836

Closes: https://github.com/searxng/searxng/issues/4861

[upd] web-client (simple): Bump the minor group 0b4c47a123

Bumps the minor group in /client/simple with 4 updates: [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js), [eslint](https://github.com/eslint/eslint), [sharp](https://github.com/lovell/sharp) and [webpack](https://github.com/webpack/webpack).


Updates `@eslint/js` from 9.26.0 to 9.27.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/commits/v9.27.0/packages/js)

Updates `eslint` from 9.26.0 to 9.27.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v9.26.0...v9.27.0)

Updates `sharp` from 0.34.1 to 0.34.2
- [Release notes](https://github.com/lovell/sharp/releases)
- [Commits](https://github.com/lovell/sharp/compare/v0.34.1...v0.34.2)

Updates `webpack` from 5.99.8 to 5.99.9
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.99.8...v5.99.9)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.27.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: eslint
  dependency-version: 9.27.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: sharp
  dependency-version: 0.34.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: webpack
  dependency-version: 5.99.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>

[l10n] update translations from Weblate 44471bef35

0f55f2438 - 2025-05-29 - sandijs <sandijs@noreply.codeberg.org>
109f7dcb8 - 2025-05-26 - ngf <ngf@noreply.codeberg.org>
d2ba9b3d0 - 2025-05-26 - artnay <artnay@noreply.codeberg.org>
33c6f23bf - 2025-05-26 - realkendrick_fr <realkendrick_fr@noreply.codeberg.org>

[upd] web-client (simple): Bump the minor group b991688fcb

Bumps the minor group in /client/simple with 2 updates: [globals](https://github.com/sindresorhus/globals) and [stylelint](https://github.com/stylelint/stylelint).


Updates `globals` from 16.1.0 to 16.2.0
- [Release notes](https://github.com/sindresorhus/globals/releases)
- [Commits](https://github.com/sindresorhus/globals/compare/v16.1.0...v16.2.0)

Updates `stylelint` from 16.19.1 to 16.20.0
- [Release notes](https://github.com/stylelint/stylelint/releases)
- [Changelog](https://github.com/stylelint/stylelint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/stylelint/stylelint/compare/16.19.1...16.20.0)

---
updated-dependencies:
- dependency-name: globals
  dependency-version: 16.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: stylelint
  dependency-version: 16.20.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>

[upd] pypi: Bump the minor group with 2 updates 1b424b8d54

Bumps the minor group with 2 updates: [selenium](https://github.com/SeleniumHQ/Selenium) and [typer-slim](https://github.com/fastapi/typer).


Updates `selenium` from 4.32.0 to 4.33.0
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.32.0...selenium-4.33.0)

Updates `typer-slim` from 0.15.4 to 0.16.0
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.15.4...0.16.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-version: 4.33.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: typer-slim
  dependency-version: 0.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>