kreativmonkey/homelab-app

Fork 0

Update getmeili/meilisearch Docker tag to v1.39.0 #14

Merged

kreativmonkey merged 1 commit from renovate/getmeili-meilisearch-1.x into main

2026-03-24 10:37:09 +01:00

Renovate commented

2026-03-03 07:21:12 +01:00

Collaborator

This PR contains the following updates:

Package	Update	Change	Pending
getmeili/meilisearch	minor	`v1.36.0` → `v1.39.0`	`v1.40.0`

Release Notes

meilisearch/meilisearch (getmeili/meilisearch)

What's Changed

✨ Enhancement

🔬 Cross-index document hydration

Add a new ForeignKeys setting allowing to hydrate documents with documents coming from other indexes.

📓 Note: This implementation doesn't support a remote sharding environment

`foreignKeys` experimental feature

A new experimental feature, foreignKeys, has been added to the /experimental-feature route that must be set to true to activate the hydration.

curl -X PATCH 'http://127.0.0.1:7700/experimental-features' \
  -H 'Content-Type: application/json' \
  --data-binary '{"foreignKeys": true}'

`foreignKeys` index setting

A new index setting, foreignKeys, has been added to the /indexes/{index_uid}/settings:

// new setting, an array of foreign keys that allows multiple foreign relationships between indexes
"foreignKeys":  [
	{
		// the path in the JSON document containing foreign document ids
		"fieldName": "actors",
		
		// the UID of the foreign index containing the documents to fetch during hydration
        "foreignIndexUid": "actors"
	}
]

Example

With this new feature, a document shaped as follows:

{
  "id": 1,
  "title": "Forrest Gump",
  // Document IDs in foreign index
  "actors": [
    1
  ]
}

Will be hydrated as follows in a search response:

{
  "id": 1,
  "title": "Forrest Gump",
  "actors": [
	{
	  "id": 1,
	  "name": "Tom",
	  "familyName": "Hanks",
	  "birthDate": "1956-07-09"
	}
  ]
}

By @ManyTheFish in #6047

Disable proxy response buffering on Server-Sent Events (SSE)

Add X-Accel-Buffering: no on POST /chats/{workspace_uid}/chat/completions when the streaming mode is activated.
By @YoEight in #6228

🪲 Bug fixes

Fix a memory leak in the indexation pipeline by @Kerollmops in #6212

We fixed an important memory leak caused by an invalid use of bumpalo. If you've seen Meilisearch using more and more memory over time, this issue is no longer visible in the latest version. If you want to read more about the bumpalo-trap we felt in, here is a good summary. It looks like this leak was introduced in v1.12, so approximately a year ago...
Avoid losing tasks by applying deltas on the wtxn view by @Kerollmops in #6222

We addressed a small performance regression introduced in v1.38.1 that affected sending updates to the engine during task deletion. We've restored the task deletion performance to match v1.38.0 levels and ensured no race conditions occur while still allowing updates during deletion.

🔩 Miscellaneous

Ignore flaky test on windows by @dureuill in #6226
Fix small documentation mistakes by @ManyTheFish in #6205
Improve messaging when field is missing in the settings indexing pipeline by @Kerollmops in #6213

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.39.0

Meilisearch v1.38.2 fixes a regression introduced in Meilisearch v1.38.0 where Meilisearch could sometimes stop processing tasks after an automatic task queue cleanup performed while more tasks where being added.

We recommend that all users in v1.38.0 and v1.38.1 migrate to v1.38.2.

Note that the dumpless upgrade process can take up to one minute before the HTTP server becomes available when migrating from a previous v1.38 version.

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.1...v1.38.2

`v1.38.1`: 🐍

Compare Source

Meilisearch v1.38.1 fixes a regression we have seen in task processing, which caused the engine to stop processing tasks.

🪲 Bug fixes

Revert, optimize, and fix task deletion. By @Kerollmops in #6219

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.38.1

`v1.38.0`: 🐍

Compare Source

Meilisearch v1.38 drastically improves indexing performance for embeddings, cleans up the task queue of orphan tasks and batches, ensures they are correctly deleted now, fixes issues users may encounter with OpenAI or VoyageAI when generating embeddings, and improves the tooling for generating routes in the documentation.

✨ Enhancement

Embeddings: Use the latest version of hannoy with improved performances by @Kerollmops in #6184
We drastically reduced embedding indexing time by eliminating full database scans. It is more efficient to add a couple of embeddings to a large database now.
Tasks: Optimize and fix task deletion by @Mubelotix and @kerollmops in #5828

🪲 Bug fixes

Embedders: Fix the intermittent connection reset by peer errors when embedding documents or search queries using a remote embedder. By @dureuill in #6191

🔩 Miscellaneous

Enforce that routes appear in the reference documentation by @dureuill and @manythefish in #6200
The reference API is now generated from the source code, but it used to miss some actix-declared routes. It is now mandatory to attach a route when using the newly-provided routes::routes and routes::path macros, which ensures the route is added to the reference. Please refer to the documentation of the individual macros and the many examples throughout the codebase for more details.
Chore: remove verbose debug log on list documents by @StephaneRob in #6198
Add guide to CONTRIBUTING.md to test doc locally by @curquiza in #6180
Docs: capitalize LangChain and MCP in README by @24f1000637 in #6183
Docs: improve documentation of the parameters of the search route by @curquiza in #6194

New Contributors

@StephaneRob made their first contribution in #6198
@24f1000637 made their first contribution in #6183

`v1.37.0`

Compare Source

[!IMPORTANT]
This release contains breaking changes for users of the network experimental feature.

Meilisearch v1.37 introduces replicated sharding, removes the vectorStoreSetting experimental feature, stabilizes our new vector store for best performance, adds a security fix and miscellaneous improvements.

✨ Improvements

§ Replicated sharding

[!NOTE]
Replicated sharding requires Meilisearch Enterprise Edition (EE).

Users of Meilisearch Cloud, please contact support if you need replicated sharding.

Users of the Community Edition, please contact the sales if you want to use replicated sharding in production.

§ Breaking changes

network objects sent to the PATCH /network route must now contain at least one shard object containing at least one remote when leader is not null.

Existing databases will be migrated automatically when upgraded with --experimental-dumpless-upgrade when leader is not null, such that for each remote:

A shard with the same name as the remote is created
This shard has exactly one remote in its remotes list: the remote with the same name as the shard.

This change will not cause any document to be resharded.

To be able to upgrade without resharding, the migration uses the same name for remotes and for shards. However, in new configurations, we recommend using different names for shards and remotes.

Example of migration

For instance, the following network object:

{
  "leader": "ms-00",
  "self": "ms-01",
  "remotes": {
    "ms-00": { /* .. */ },
    "ms-01": { /* .. */ }
  }
}

is converted to:

{
  "leader": "ms-00",
  "self": "ms-01",
  "remotes": {
    "ms-00": { /* .. */ },
    "ms-01": { /* .. */ }
  },
  "shards": {  // ✨ NEW
    "ms-00": {  // shard named like the remote
      "remotes": ["ms-00"] // is owned by the remote
    },
    "ms-01": {
      "remotes": ["ms-01"]
    }
  }
}

Addition of `network.shards`

The network object for routes PATCH /network and GET /network now contains the new field shards, which is an object whose values are shard objects, and keys the name of each shard.

Each shard object contains a single field remotes, which is an array of strings, each string representing the name of an existing remote.

Convenience fields

The shard objects in PATCH /network contain the additional fields addRemotes and removeRemotes meant for convenience:

pass an array of remote names to shard.addRemotes to add these remotes to the list of remotes of a shard.
pass an array of remote names to shard.removeRemotes to remove these remotes from the list of remotes of a shard.
if present and non-null, shard.remotes will completely override the existing list of remotes for a shard.
if several of these options are present and non-null, then the order of application is shard.remotes, then shard.addRemotes, then shard.removeShards.

Adding a new shard with some remotes

// PATCH /network
{
  // assuming that remotes `ms-0`, `ms-1`, `ms-2` where sent in a previous call to PATCH /network
  "shards": {
    "s-a": { // new shard
      "remotes": ["ms-0", "ms-1"]
    }
  }
}

Remotes ms-0 and ms-1 own the new shard s-a.

Fully overriding the list of remotes owning a shard

// PATCH /network
{
  // assuming remotes `ms-0`, `ms-1`, `ms-2`
  // assuming shard `s-a`, owned by `ms-0` and `ms-1`
  "shards": {
    "s-a": {
      "remotes": ["ms-2"]
    }
  }
}

ms-2 is now the sole owner of s-a, replacing ms-0 and ms-1.

Adding a remote without overriding the list of remotes owning a shard

// PATCH /network
{
  // assuming remotes `ms-0`, `ms-1`, `ms-2`
  // assuming shard `s-a`, owned by `ms-2`
  "shards": {
    "s-a": {
      "addRemotes": ["ms-0"]
    }
  }
}

ms-0 and ms-2 are now the owners of s-a.

Removing a remote without overriding the list of remotes owning a shard

// PATCH /network
{
  // assuming remotes `ms-0`, `ms-1`, `ms-2`
  // assuming shard `s-a`, owned by `ms-0` and `ms-2`
  "shards": {
    "s-a": {
      "removeRemotes": ["ms-2"]
    }
  }
}

ms-0 is now the sole owner of s-a.

Entirely removing a shard from the list of shards

Set the shard to null:

// PATCH /network
{
  "shards": {
    "s-a": null
  }
}

Or set its remotes list to the empty list:

// PATCH /network
{
  "shards": {
    "s-a": {
      "remotes": []
    }
  }
}

`network.shards` validity

When network.leader is not null, each shard object in network.shards must:

Only contain remotes that exist in the list of remotes.
Contain at least one remote.

Additionally, network.shards must contain at least one shard.

Failure to meet any of these conditions will cause the PATCH /network route to respond with 400 invalid_network_shards.

Change in sharding logic

Documents are now sharded according to the list of shards declared in the network rather than the list of remotes. All remotes owning a shard will process the documents that belong to this shard, allowing for replication.

Example of replication

The following configuration defines 3 remotes 0, 1 and 2, and 3 shards A, B, C, such that each remote owns two shards, achieving replication (losing one remote does not lose any document).

{
  "leader": "0",
  "self": "0",
  "remotes": {
    "0": { /* .. */ },
    "1": { /* .. */ },
    "2": { /* .. */ }
  },
  "shards": {
    "A": {
      "remotes": ["0", "1"]
    },
    "B": {
      "remotes": ["1", "2"]
    },
    "C": {
      "remotes": ["2", "0"]
    }
  }
}

Full replication is supported by having all remotes own all the shards.
Unbalanced replication is supported by having some remotes own more shards than other remotes.
"Watcher" remotes are supported by having remotes that own no shards. Watcher remotes are not very useful in this release, and might be upgraded in a future release, so that they keep all documents without indexing them, allowing to "respawn" shards for other remotes.

`useNetwork` takes `network.shards` into account

When useNetwork: true is passed to a search query, it is expanded to multiple queries such that each shard declared in network.shards appears exactly once, associated with a remote that owns that shard.

This ensures that there is no missing or duplicate documents in the results.

`_shard` filters

When the network experimental feature is enabled, then it becomes possible to filter documents depending on the shard they belong to.

Given s-a and s-b the names of two shards declared in network.shards, then:

_shard = "s-a" in a filter parameter to the search or documents fetch will return the documents that belong to s-a.
_shard != "s-a" will return the documents that do not belong to s-a
_shard IN ["s-a", "s-b"] will return the documents that belong to s-a or to s-b.

You can use these new filters in manual remote federated search to create a partitioning over all shards in the network.

[!IMPORTANT]
To avoid duplicate or missing documents in results, for manually crafted remote federated search requests, all shards should appear in exactly one query.

[!TIP]
Search requests built with useNetwork: true already build a correct partitioning over shards. They should be preferred to manually crafted remote federated search requests in replicated sharding scenarios.

Update instructions

When updating your Meilisearch network using dumpless upgrade, please observe the following guidelines:

Do not call the PATCH /network route until all remotes of the network are finished updating
If using the search routes with useNetwork: true, call them on un-updated remotes. Calling it on already updated remotes will cause un-updated remotes to fail the search as they don't know about the _shard filters.

By @dureuill in #6128

§ Remove `vectorStoreSetting` experimental feature

The new HNSW vector store (hannoy) has been stabilized and is now the only supported vector store in Meilisearch.

As a result, updating to v1.37.0 will migrate all remaining legacy vector store indexes (using arroy) to hannoy, and the vectorStoreSetting experimental feature is no longer available.

By @Kerollmops in #6176

Improve indexing performance for embeddings

We removed a computationally expensive step from vector indexing.

On a DB with 20M documents, this removes 300s per indexing batch of 1100s.

By @Kerollmops in #6175

§ 🔒 Security

Bump mini-dashboard (local web interface) which
- now stores API key in RAM instead of in the localStorage
- bumps dependencies with potential security vulnerabilities

By @Strift and @curquiza in #6186 and #6172

§ 🔩 Miscellaneous

Mark Cargo.lock as not linguist-generated by @Kerollmops in #6181

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.36.0...v1.37.0

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

This PR contains the following updates: | Package | Update | Change | Pending | |---|---|---|---| | [getmeili/meilisearch](https://github.com/meilisearch/meilisearch) | minor | `v1.36.0` → `v1.39.0` | `v1.40.0` | --- ### Release Notes <details> <summary>meilisearch/meilisearch (getmeili/meilisearch)</summary> ### [`v1.39.0`](https://github.com/meilisearch/meilisearch/releases/tag/v1.39.0): 🐳 [Compare Source](https://github.com/meilisearch/meilisearch/compare/v1.38.2...v1.39.0) ### What's Changed #### **✨ Enhancement** ##### 🔬 Cross-index document hydration Add a new `ForeignKeys` setting allowing to hydrate documents with documents coming from other indexes. > 📓 Note: This implementation doesn't support a remote sharding environment ##### `foreignKeys` experimental feature A new experimental feature, `foreignKeys`, has been added to the `/experimental-feature` route that must be set to true to activate the hydration. ``` curl -X PATCH 'http://127.0.0.1:7700/experimental-features' \ -H 'Content-Type: application/json' \ --data-binary '{"foreignKeys": true}' ``` ##### `foreignKeys` index setting A new index setting, `foreignKeys`, has been added to the `/indexes/{index_uid}/settings`: ```jsonc // new setting, an array of foreign keys that allows multiple foreign relationships between indexes "foreignKeys": [ { // the path in the JSON document containing foreign document ids "fieldName": "actors", // the UID of the foreign index containing the documents to fetch during hydration "foreignIndexUid": "actors" } ] ``` <details> <summary> ##### Example </summary> With this new feature, a document shaped as follows: ```jsonc { "id": 1, "title": "Forrest Gump", // Document IDs in foreign index "actors": [ 1 ] } ``` Will be hydrated as follows in a search response: ```jsonc { "id": 1, "title": "Forrest Gump", "actors": [ { "id": 1, "name": "Tom", "familyName": "Hanks", "birthDate": "1956-07-09" } ] } ``` </details> By [@ManyTheFish](https://github.com/ManyTheFish) in [#6047](https://github.com/meilisearch/meilisearch/issues/6047) ##### Disable proxy response buffering on Server-Sent Events (SSE) Add `X-Accel-Buffering: no` on `POST /chats/{workspace_uid}/chat/completions` when the streaming mode is activated. By **[@YoEight](https://github.com/YoEight)** in [#6228](https://github.com/meilisearch/meilisearch/pull/6228) #### 🪲 Bug fixes - Fix a memory leak in the indexation pipeline by **[@Kerollmops](https://github.com/Kerollmops)** in [#6212](https://github.com/meilisearch/meilisearch/pull/6212) We fixed an important memory leak caused by an invalid use of bumpalo. If you've seen Meilisearch using more and more memory over time, this issue is no longer visible in the latest version. If you want to read more about the bumpalo-trap we felt in, [here is a good summary](https://docs.rs/bumpalo/latest/bumpalo/#deallocation-en-masse-but-no-drop). It looks like this leak was introduced in v1.12, so approximately a year ago... - Avoid losing tasks by applying deltas on the wtxn view by **[@Kerollmops](https://github.com/Kerollmops)** in [#6222](https://github.com/meilisearch/meilisearch/pull/6222) We addressed a small performance regression introduced in v1.38.1 that affected sending updates to the engine during task deletion. We've restored the task deletion performance to match v1.38.0 levels and ensured no race conditions occur while still allowing updates during deletion. #### **🔩 Miscellaneous** - Ignore flaky test on windows by **[@dureuill](https://github.com/dureuill)** in [#6226](https://github.com/meilisearch/meilisearch/pull/6226) - Fix small documentation mistakes by **[@ManyTheFish](https://github.com/ManyTheFish)** in [#6205](https://github.com/meilisearch/meilisearch/pull/6205) - Improve messaging when field is missing in the settings indexing pipeline by **[@Kerollmops](https://github.com/Kerollmops)** in [#6213](https://github.com/meilisearch/meilisearch/pull/6213) **Full Changelog**: <https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.39.0> ### [`v1.38.2`](https://github.com/meilisearch/meilisearch/releases/tag/v1.38.2): 🐍 [Compare Source](https://github.com/meilisearch/meilisearch/compare/v1.38.1...v1.38.2) Meilisearch v1.38.2 fixes a regression introduced in Meilisearch v1.38.0 where Meilisearch could sometimes stop processing tasks after an automatic task queue cleanup performed while more tasks where being added. We recommend that all users in v1.38.0 and v1.38.1 migrate to v1.38.2. Note that the dumpless upgrade process can take up to one minute before the HTTP server becomes available when migrating from a previous v1.38 version. **Full Changelog**: <https://github.com/meilisearch/meilisearch/compare/v1.38.1...v1.38.2> ### [`v1.38.1`](https://github.com/meilisearch/meilisearch/releases/tag/v1.38.1): 🐍 [Compare Source](https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.38.1) Meilisearch v1.38.1 fixes a regression we have seen in task processing, which caused the engine to stop processing tasks. ##### 🪲 Bug fixes - Revert, optimize, and fix task deletion. By [@Kerollmops](https://github.com/Kerollmops) in [#6219](https://github.com/meilisearch/meilisearch/pull/6219) **Full Changelog**: <https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.38.1> ### [`v1.38.0`](https://github.com/meilisearch/meilisearch/releases/tag/v1.38.0): 🐍 [Compare Source](https://github.com/meilisearch/meilisearch/compare/v1.37.0...v1.38.0) Meilisearch v1.38 drastically improves indexing performance for embeddings, cleans up the task queue of orphan tasks and batches, ensures they are correctly deleted now, fixes issues users may encounter with OpenAI or VoyageAI when generating embeddings, and improves the tooling for generating routes in the documentation. ##### **✨ Enhancement** - Embeddings: Use the latest version of hannoy with improved performances by [@Kerollmops](https://github.com/Kerollmops) in [#6184](https://github.com/meilisearch/meilisearch/pull/6184) We drastically reduced embedding indexing time by eliminating full database scans. It is more efficient to add a couple of embeddings to a large database now. - Tasks: Optimize and fix task deletion by [@Mubelotix](https://github.com/Mubelotix) and [@kerollmops](https://github.com/kerollmops) in [#5828](https://github.com/meilisearch/meilisearch/pull/5828) ##### 🪲 Bug fixes - Embedders: Fix the intermittent `connection reset by peer` errors when embedding documents or search queries using a remote embedder. By [@dureuill](https://github.com/dureuill) in [#6191](https://github.com/meilisearch/meilisearch/pull/6191) ##### 🔩 Miscellaneous - Enforce that routes appear in the reference documentation by [@dureuill](https://github.com/dureuill) and [@manythefish](https://github.com/manythefish) in [#6200](https://github.com/meilisearch/meilisearch/pull/6200) [The reference API](https://www.meilisearch.com/docs/reference/api/) is now generated from the source code, but it used to miss some actix-declared routes. It is now mandatory to attach a route when using the newly-provided `routes::routes` and `routes::path` macros, which ensures the route is added to the reference. Please refer to the documentation of the individual macros and the many examples throughout the codebase for more details. - Chore: remove verbose debug log on list documents by [@StephaneRob](https://github.com/StephaneRob) in [#6198](https://github.com/meilisearch/meilisearch/pull/6198) - Add guide to [CONTRIBUTING.md](http://contributing.md/) to test doc locally by [@curquiza](https://github.com/curquiza) in [#6180](https://github.com/meilisearch/meilisearch/pull/6180) - Docs: capitalize LangChain and MCP in README by [@24f1000637](https://github.com/24f1000637) in [#6183](https://github.com/meilisearch/meilisearch/pull/6183) - Docs: improve documentation of the parameters of the search route by [@curquiza](https://github.com/curquiza) in [#6194](https://github.com/meilisearch/meilisearch/pull/6194) ##### New Contributors - [@StephaneRob](https://github.com/StephaneRob) made their first contribution in [#6198](https://github.com/meilisearch/meilisearch/pull/6198) - [@24f1000637](https://github.com/24f1000637) made their first contribution in [#6183](https://github.com/meilisearch/meilisearch/pull/6183) ### [`v1.37.0`](https://github.com/meilisearch/meilisearch/releases/tag/v1.37.0) [Compare Source](https://github.com/meilisearch/meilisearch/compare/v1.36.0...v1.37.0) > \[!IMPORTANT]\ > This release contains [**breaking changes**](#v137-replicated-sharding-breaking-changes) for users of the `network` experimental feature. Meilisearch v1.37 introduces [replicated sharding](#v137-replicated-sharding), removes the [`vectorStoreSetting` experimental feature](#v137-vector-store), stabilizes our new vector store for best performance, adds a [security fix](#v137-security) and [miscellaneous improvements](#v137-misc). ### ✨ Improvements #### <a id="v137-replicated-sharding" href="#v137-replicated-sharding">§</a> Replicated sharding > \[!NOTE] > Replicated sharding requires Meilisearch Enterprise Edition (EE). > > - Users of Meilisearch Cloud, please contact support if you need replicated sharding. > - Users of the Community Edition, please contact the sales if you want to use replicated sharding in production. ##### <a id="v137-replicated-sharding-breaking-changes" href="#v137-replicated-sharding-breaking-changes">§</a> Breaking changes - `network` objects sent to the `PATCH /network` route must now contain at least one `shard` object containing at least one remote when `leader` is not `null`. Existing databases will be migrated automatically when upgraded with `--experimental-dumpless-upgrade` when `leader` is not `null`, such that for each remote: 1. A shard with the same name as the remote is created 2. This shard has exactly one remote in its `remotes` list: the remote with the same name as the shard. This change will not cause any document to be resharded. To be able to upgrade without resharding, the migration uses the same name for remotes and for shards. However, in new configurations, we recommend using different names for shards and remotes. <details> <summary>Example of migration</summary> For instance, the following network object: ```jsonc { "leader": "ms-00", "self": "ms-01", "remotes": { "ms-00": { /* .. */ }, "ms-01": { /* .. */ } } } ``` is converted to: ```jsonc { "leader": "ms-00", "self": "ms-01", "remotes": { "ms-00": { /* .. */ }, "ms-01": { /* .. */ } }, "shards": { // ✨ NEW "ms-00": { // shard named like the remote "remotes": ["ms-00"] // is owned by the remote }, "ms-01": { "remotes": ["ms-01"] } } } ``` </details> ##### Addition of `network.shards` The `network` object for routes `PATCH /network` and `GET /network` now contains the new field `shards`, which is an object whose values are `shard` objects, and keys the name of each shard. Each `shard` object contains a single field `remotes`, which is an array of strings, each string representing the name of an existing remote. ##### Convenience fields The `shard` objects in `PATCH /network` contain the additional fields `addRemotes` and `removeRemotes` meant for convenience: - pass an array of remote names to `shard.addRemotes` to add these remotes to the list of remotes of a shard. - pass an array of remote names to `shard.removeRemotes` to remove these remotes from the list of remotes of a shard. - if present and non-`null`, `shard.remotes` will completely override the existing list of remotes for a shard. - if several of these options are present and non-`null`, then the order of application is `shard.remotes`, then `shard.addRemotes`, then `shard.removeShards`. <details> <summary>Adding a new shard with some remotes</summary> ```jsonc // PATCH /network { // assuming that remotes `ms-0`, `ms-1`, `ms-2` where sent in a previous call to PATCH /network "shards": { "s-a": { // new shard "remotes": ["ms-0", "ms-1"] } } } ``` Remotes `ms-0` and `ms-1` own the new shard `s-a`. </details> <details> <summary>Fully overriding the list of remotes owning a shard</summary> ```jsonc // PATCH /network { // assuming remotes `ms-0`, `ms-1`, `ms-2` // assuming shard `s-a`, owned by `ms-0` and `ms-1` "shards": { "s-a": { "remotes": ["ms-2"] } } } ``` `ms-2` is now the sole owner of `s-a`, replacing `ms-0` and `ms-1`. </details> <details> <summary>Adding a remote without overriding the list of remotes owning a shard</summary> ```jsonc // PATCH /network { // assuming remotes `ms-0`, `ms-1`, `ms-2` // assuming shard `s-a`, owned by `ms-2` "shards": { "s-a": { "addRemotes": ["ms-0"] } } } ``` `ms-0` and `ms-2` are now the owners of `s-a`. </details> <details> <summary>Removing a remote without overriding the list of remotes owning a shard</summary> ```jsonc // PATCH /network { // assuming remotes `ms-0`, `ms-1`, `ms-2` // assuming shard `s-a`, owned by `ms-0` and `ms-2` "shards": { "s-a": { "removeRemotes": ["ms-2"] } } } ``` `ms-0` is now the sole owner of `s-a`. </details> <details> <summary>Entirely removing a shard from the list of shards</summary> Set the shard to `null`: ```jsonc // PATCH /network { "shards": { "s-a": null } } ``` Or set its `remotes` list to the empty list: ```jsonc // PATCH /network { "shards": { "s-a": { "remotes": [] } } } ``` </details> ##### `network.shards` validity When `network.leader` is not `null`, each `shard` object in `network.shards` must: 1. Only contain `remotes` that exist in the list of `remotes`. 2. Contain at least one remote. Additionally, `network.shards` must contain at least one shard. Failure to meet any of these conditions will cause the `PATCH /network` route to respond with `400 invalid_network_shards`. ##### Change in sharding logic Documents are now sharded according to the list of shards declared in the network rather than the list of remotes. All remotes owning a shard will process the documents that belong to this shard, allowing for replication. <details> <summary>Example of replication</summary> The following configuration defines 3 remotes `0`, `1` and `2`, and 3 shards `A`, `B`, `C`, such that each remote owns two shards, achieving replication (losing one remote does not lose any document). ```jsonc { "leader": "0", "self": "0", "remotes": { "0": { /* .. */ }, "1": { /* .. */ }, "2": { /* .. */ } }, "shards": { "A": { "remotes": ["0", "1"] }, "B": { "remotes": ["1", "2"] }, "C": { "remotes": ["2", "0"] } } } ``` </details> - Full replication is supported by having all remotes own all the shards. - Unbalanced replication is supported by having some remotes own more shards than other remotes. - "Watcher" remotes are supported by having remotes that own no shards. Watcher remotes are not very useful in this release, and might be upgraded in a future release, so that they keep all documents without indexing them, allowing to "respawn" shards for other remotes. ##### `useNetwork` takes `network.shards` into account When `useNetwork: true` is passed to a search query, it is expanded to multiple queries such that each shard declared in `network.shards` appears exactly once, associated with a remote that owns that shard. This ensures that there is no missing or duplicate documents in the results. ##### `_shard` filters When the `network` experimental feature is enabled, then it becomes possible to filter documents depending on the shard they belong to. Given `s-a` and `s-b` the names of two shards declared in `network.shards`, then: - `_shard = "s-a"` in a `filter` parameter to the search or documents fetch will return the documents that belong to `s-a`. - `_shard != "s-a"` will return the documents that do **not** belong to `s-a` - `_shard IN ["s-a", "s-b"]` will return the documents that belong to `s-a` or to `s-b`. You can use these new filters in manual remote federated search to create a partitioning over all shards in the network. > \[!IMPORTANT] > To avoid duplicate or missing documents in results, for manually crafted remote federated search requests, all shards should appear in **exactly one** query. > \[!TIP] > Search requests built with `useNetwork: true` already build a correct partitioning over shards. They should be preferred to manually crafted remote federated search requests in replicated sharding scenarios. ##### Update instructions When updating your Meilisearch network using dumpless upgrade, please observe the following guidelines: 1. Do not call the `PATCH /network` route until all remotes of the network are finished updating 2. If using the search routes with `useNetwork: true`, call them on un-updated remotes. Calling it on already updated remotes will cause un-updated remotes to fail the search as they don't know about the `_shard` filters. By [@dureuill](https://github.com/dureuill) in [#6128](https://github.com/meilisearch/meilisearch/pull/6128) #### <a id="v137-vector-store" href="#v137-vector-store">§</a> Remove `vectorStoreSetting` experimental feature The new HNSW vector store ([hannoy](https://github.com/nnethercott/hannoy)) has been stabilized and is now the only supported vector store in Meilisearch. As a result, updating to v1.37.0 will migrate all remaining legacy vector store indexes (using arroy) to hannoy, and the [`vectorStoreSetting` experimental feature](https://github.com/orgs/meilisearch/discussions/860) is no longer available. By [@Kerollmops](https://github.com/Kerollmops) in [#6176](https://github.com/meilisearch/meilisearch/pull/6176) #### Improve indexing performance for embeddings We removed a computationally expensive step from vector indexing. On a DB with 20M documents, this removes 300s per indexing batch of 1100s. By [@Kerollmops](https://github.com/Kerollmops) in [#6175](https://github.com/meilisearch/meilisearch/pull/6175) #### <a id="v137-security" href="#v137-security">§</a> 🔒 Security - Bump mini-dashboard (local web interface) which - now stores API key in RAM instead of in the `localStorage` - bumps dependencies with potential security vulnerabilities By [@Strift](https://github.com/Strift) and [@curquiza](https://github.com/curquiza) in [#6186](https://github.com/meilisearch/meilisearch/pull/6186) and [#6172](https://github.com/meilisearch/meilisearch/pull/6172) #### <a id="v137-misc" href="#v137-misc">§</a> 🔩 Miscellaneous - Mark Cargo.lock as not linguist-generated by [@Kerollmops](https://github.com/Kerollmops) in [#6181](https://github.com/meilisearch/meilisearch/pull/6181) **Full Changelog**: <https://github.com/meilisearch/meilisearch/compare/v1.36.0...v1.37.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Renovate added 1 commit

2026-03-03 07:21:13 +01:00