Normal usage needs no adaptation and gains capabilities or correctness fixes.
This is the multi-page printable view of this section. .
Compatibility
- 1: Component Versions
- 2: Password Permissions
- 3: Access-Tiering Removal
- 4: Migrate from MinIO to Silo
- 5: Silo vs. MinIO Server Compatibility
- 6: MCLI Client Compatibility Notes
- 7: Native Package Migration
- 8: Console Compatibility Notes
- 9: Feature Notes
Most applications using standard S3 APIs can switch from MinIO to SILO without code changes. Common S3 APIs, SigV4, SDK integration and object storage layouts carry over. Deployment, authorization and replicated state require the checks below: SILO is a conditional drop-in replacement.
Mostly presentation, UI interaction and response information; usually no application changes.
When a condition applies, verify or adjust deployment, policies, tools or upgrade procedures.
The current release baseline is 20261026. This index covers source and companion-component changes since the fork and identifies when each repair first shipped. The component matrix records the current stack and fixes absent from older versions.
20261026 release changes
The Server 20261026 adds fixes for LDAP long-DN login, built-in policy Reset, live bucket inventory, versioned lifecycle expiration, multipart replication, and multi-pool deletion. Existing read-quorum and healing rules remain unchanged; see the component matrix for lease-loss, partial-metadata, and pre-existing inconsistency boundaries.
20261018 release
Server 20261026 retains the fixes delivered in 20261018. Upgrade checks cover Object Lock, IAM Admin authorization, ordinary LIST recovery, retirement of strict multipart listing and OIDC/JWK key limits; see Server compatibility. The strict-mode prerequisites below apply only to the optional mode in Server 20260916.
🟢 Compatible improvements: 12 categories
Normal usage needs no migration work for these capabilities. Dependencies on old bugs, mixed versions or damaged historical state are covered by the orange entries.
| Category and change | When it applies | Impact and action | Details |
|---|---|---|---|
| G01 · Full Web administration Restores full Web management of buckets, objects, users, policies and the system. |
Managing a deployment in a browser. | More capable administration; ordinary S3 applications need no changes. | Console features |
| G02 · Chinese UI and text preview Adds English/Chinese UI and bounded, safe text preview. |
Switching language or previewing text objects. | Changes the Console experience, not stored object content. | Languages, text preview |
| G03 · Per-bucket CORS The existing CORS APIs now persist and enforce per-bucket cross-origin rules. |
Browser cross-origin access with bucket CORS configured. | Bucket rules override global CORS; object access still requires authorization. | CORS behavior |
| G04 · Native health checks Adds silo healthcheck for container and operational probes. |
Choosing to use the new probe command. | Existing HTTP health endpoints remain available. | Command and scope |
| G05 · Client checksum audit Adds mcli checksum verify to read objects and verify checksums. |
Explicitly running an integrity audit. | Read-only checks and reports; no object modification or repair. | Audit command |
| G06 · Upload and copy checksums Valid uploads, multipart completion and copies compute, retain and return checksums more accurately. |
Using the affected upload or copy paths. | Better SDK interoperability; invalid requests are covered by O04. | Multipart uploads, S3 fixes |
| G07 · Encryption and compression fixes Corrects data and logical-size handling across SSE-C, KMS, compression and copying. |
Using these encryption, compression or copy combinations. | Fixes new requests; existing objects are unchanged. | SSE-C and old objects, federated copies |
| G08 · Concurrent bucket configuration A shared metadata lock prevents concurrent bucket updates from overwriting each other. |
Concurrent policy, lifecycle, encryption or other bucket updates. | More reliable administration with the same calling pattern. | Shared configuration lock |
| G09 · Tag and Object Lock replication Replication preserves and orders tag, retention and lock state more accurately. |
Replicating or healing tags and Object Lock metadata. | Reduces lost state and stale-event overwrites; see O07 for mixed versions. | Tag ordering, lock ordering |
| G10 · Pool migration and healing Rebalance, decommission and recovery handle tags, versions and delete markers more accurately. |
Moving pools, healing or recovering replication. | Better metadata preservation and retries; historical anomalies still need inspection. | Multi-pool consistency, replication recovery |
| G11 · Notifications and streaming Fixes NATS/AMQP setting recognition and flushing of notification and Select streams. |
Using those notification targets or streaming APIs. | More accurate configuration and event output; see O03 for old database settings. | Notification settings, stream flushing |
| G12 · Stability and resource cleanup Fixes concurrent collection, buffer ownership and connection cleanup. |
Concurrent I/O, metric collection or request cancellation. | Prevents related crashes and resource leaks. | Read buffers, runtime and metrics |
🔵 Minor differences: 4 categories
Ordinary applications usually need no changes. Tools that hard-code product strings, metric meanings or response fields should still be checked.
| Category and change | When it applies | Impact and action | Details |
|---|---|---|---|
| B01 · Product identity Banners, logs, HTTP product identity and UI labels use SILO. |
Reading output or matching product names. | Presentation changes; adapt name-matching scripts. Installation changes are in O01. | Server identity, client output |
| B02 · Paginated Console listings Object listings use cursors, with sorting, filtering and selection limited to the current page. |
Browsing, filtering or selecting objects in Console. | UI habits change; direct S3 application access does not. | Pagination boundaries |
| B03 · Richer monitoring and diagnostics Adds diagnostic metrics and corrects quota, replication counters and status meanings. |
Using affected metrics, dashboards or alerts. | Metric namespaces remain; review the affected displays and alert rules. | Metric changes, Console metrics |
| B04 · Additional response fields Some copy, multipart and Console responses include checksum or session fields. |
Reading those API responses. | Standard clients usually handle them; parsers rejecting unknown fields belong in O05. | Multipart responses, copy responses, Console API |
🟠 Conditional checks: 8 categories
If the condition applies, check the entry.
| Category and change | When it applies | Impact and action | Details |
|---|---|---|---|
| O01 · Installation and startup Artifacts become silo / mcli; verify services, images, directories and permissions. |
Migrating MinIO packages, containers, systemd or Helm. | Administrators perform a deployment migration check; application code usually stays the same. | Containers, packages and accounts, client configuration |
| O02 · Custom authorization policies Password changes, permanent version deletion and some bucket operations use corrected authorization rules. |
Custom Allow/Deny rules, object ARNs granting bucket operations, or source-IP conditions. | Effective permissions may change; verify policies and proxy trust. | Passwords, version deletion, bucket authorization, proxy trust |
| O03 · Older auth and notification settings Legacy authentication, database notifications and some TLS/environment-file usage need adjustment or validation. |
OIDC HMAC tokens, discrete legacy database settings, old proxies/TLS or unusual environment files. | Login, notifications or startup may fail; migrate the relevant configuration. | OIDC algorithms, database connections, TLS, environment files |
| O04 · Programs relying on old bugs Validation, conditions, errors, exit codes and listing limits change on affected paths. |
Expecting invalid requests to succeed, If-Match to be ignored, or more than 1,000 multipart uploads per page when opting into Server 20260916 strict mode. |
Check error handling, retries, pagination and final batch exit codes. | Checksum errors, conditional deletion, multipart listing, CLI exit codes |
| O05 · Custom clients and administration tools Private APIs, signed-header coverage, Console automation and Go package paths have changed. |
Using ReadMultiple, custom signing, strict parsing, Go embedding or customized Console clients. |
Verify custom integrations and builds separately; ordinary S3 SDKs do not use the private storage API. | Private APIs, signing, Console API, Go modules |
| O06 · Multi-pool operations and retired strict multipart mode Multi-pool writes/deletes verify metadata; the optional 20260916 strict listing mode is retired. |
Multi-pool conditional writes or deletion, or upgrading a deployment that enabled strict listing. | Failures can return 503 while GET works. Server 20261018 and 20261026 ignore the strict switch with a warning; durable listing and strict cancellation are unavailable. | Multi-pool conditions, current multipart contract |
| O07 · Replication, mixed versions and rollback IAM revocations and bucket-configuration deletion state require coordinated upgrades and recovery. |
Multiple sites, shared IAM, mixed versions or downgrades. | Preserve a complete recovery point and verify all peers before rolling back. | IAM upgrades, configuration convergence, rollback scope |
| O08 · Self-update and upstream online services In-place updates, SUBNET and upstream hosted-support integrations are disabled. |
Using admin update, mcli update or MinIO online support workflows. |
Upgrade through packages, images or orchestration; ordinary S3 access is unaffected by this change. | Server updates, client updates, SUBNET |
Comparison scope and release boundaries
The server comparison starts at upstream source 27742d469462, dated 2025-12-03 (after the final upstream release tag on 2025-10-15). See the Console and client pages for their separate baselines. This index covers APIs, capabilities, observable behavior and operations; one fix can affect several scenarios.
Upstream MinIO/MC compatibility is best effort. The supported, release-tested combination is SILO + SILO Console + mcli + silo-pkg. Retained protocols, environment variables and storage layouts do not guarantee arbitrary upstream mixed-version operation or downgrades. The detailed server audit retains historical baselines and individual findings; the component matrix lists published component versions across the stack.
The experimental access-frequency pool tiering that was added and later withdrawn is outside these 24 categories: it was not an upstream feature at the fork and never shipped in a public Server release: 20260903 predates it, and it was removed before 20260916. Ordinary lifecycle expiration, remote tiering, rebalance and pool decommission remain available.
For migration, select the relevant O01–O08 conditions, then follow the migration guide.
1 - Component Versions
SILO 20261026 is the current Server release, with embedded Console 2.5.1, mcli 20261026 and silo-pkg 3.15.2. The latest standalone client is mcli 20261109. KES 20261026 is a separate, optional external key service. The maintained PGSTY stack is supported; upstream MinIO/MC compatibility is best effort.
Standalone Console
The latest published standalone Console is 2.7.0,
from source 4e91ae506e706ccdecb512b1d8b5ab4d9871d505. It adds ten UI languages, Arabic
RTL, container CA/temporary-directory fixes, and usage-worker, plain-HTTP and
upload fixes, using Go 1.27.3, silo-pkg v3.16.0 and the official mcli 20261109
release (v0.0.0-20261109021353-e1d4cc5abfe0).
2.6.0 (source
f8ab146695fd1df74e4b84128a286d552921bf00) added usage scans,
retained-version totals and bucket-administration fixes.
Server 20261026 continues to embed Console 2.5.1 until a
separate Server release adopts newer source.
mcli 20261109
The standalone client release uses tag
RELEASE.2026-11-09T00-00-00Z, package 20261109000000.0.0, and source
e1d4cc5abfe00b674eb36347399e7398990be467.
It is built with Go 1.27.3, silo-pkg v3.16.0 and upstream minio-go
v7.4.2-0.20261025165926-320af050c445. Existing Server and Console artifacts
retain their embedded client versions; adopting the new client requires a rebuild.
Shared package v3.16.0
silo-pkg v3.16.0 is a separate source-only release
requiring Go 1.27.3. It updates JOSE, networking, CPU detection and YAML
dependencies while preserving public APIs and policy semantics. Applications
must select github.com/pgsty/silo-pkg/v3 v3.16.0 and rebuild to adopt it.
The historical application artifacts below still contain their original
dependency versions; publishing this library does not update those artifacts.
Released 20261026 components
| Component | Released version | Fixed source / module |
|---|---|---|
| Server | RELEASE.2026-10-26T00-00-00Z |
c3ad836aed0d8a7d23c2e44a5eaa3009d08525e3 |
| Embedded and standalone Console | v2.5.1 |
v0.0.0-20261026153924-fff1618d9abb |
| mcli / MC module | RELEASE.2026-10-26T00-00-00Z |
v0.0.0-20261026120214-0ceb298cdcd0 |
| Shared package | github.com/pgsty/silo-pkg/v3 v3.15.2 |
0416eb1f450536ca2cc3592c2c2a06a3df88d4e0 |
| Upstream S3 SDK | github.com/minio/minio-go/v7 |
v7.4.2-0.20261025165926-320af050c445 |
| Optional KES | RELEASE.2026-10-26T00-00-00Z |
e9b4f1acebc11d61b42f876da8df48f548750cc9 |
| Go / platform archives | Go 1.27.3 | Windows ZIP; Linux/macOS tar.gz |
The 20261026 Server, mcli and KES artifacts use package version 20261026000000.0.0. Upgrading a standalone Console deployment does not replace an older Server’s embedded UI or server-side behavior; built-in policy Reset, live bucket inventory reads, and the lifecycle and replication fixes require Server 20261026.
Combined release notes · Console 2.5 · pkg 3.15 · Upgrade checks · Downloads
Official container images are published on Docker Hub under pgsty/silo, pgsty/mc, pgsty/silo-console, and pgsty/kes (package-repository mirrors may update slightly after GitHub Releases). The historical dependency graphs below are retained for upgrade and rollback checks.
Server 20261018 release
The target is RELEASE.2026-10-18T00-00-00Z, package version 20261018000000.0.0, selected source 0cd388c66. The GitHub release is public, with anonymous downloads and Docker Hub images verified on 2026-10-19. See the 20261018 release notes for features, security changes, compatibility and remaining limits.
20261015 remains a historical Draft superseded by the 20261018 release, which delivers the October Object Lock, IAM, and LIST repairs (absent from Server 20260916).
October component graph
| Component | Selected by Server 20261018 |
|---|---|
| Embedded Console | v2.4.2, v0.0.0-20261017112417-ef2dd2c41d21 |
| MC module / image-bundled mcli | 20261018, v0.0.0-20261017101407-fa3b8bf72458 |
| Server shared package | github.com/pgsty/silo-pkg/v3 v3.14.3 |
| Upstream S3 SDK | github.com/minio/minio-go/v7 v7.4.1 |
| Build toolchain | Go 1.27.2; Console frontend uses Node 24.22.0 LTS |
The historical Console v2.4.2 and mcli 20261018 releases declare pkg v3.14.2; Server’s module graph selects v3.14.3. The v3.14.2 dependency baseline and v3.14.3 admin-action classification optimization preserve APIs, policy formats and authorization behavior. Server must select Console and MC replacements explicitly; Go does not inherit dependency replacements. The supported integration target is the PGSTY stack, with best-effort upstream MinIO/MC compatibility.
September 16 dependency graph
Console v2.4.1 selects the released pkg and MC sources below. These identities are verified through the public Go proxy and checksum database.
- Console:
v0.0.0-20260916075814-1360e26d976d→1360e26d976d, tagv2.4.1. - pkg:
v3.14.1→fa657ef431ae22e720df37e5144cf00f67102945. - MC:
v0.0.0-20260916070421-e952aa78f10a→e952aa78f10a, tagRELEASE.2026-09-16T00-00-00Z. - Upstream minio-go:
v7.3.1-0.20260915093545-32e1f32cb176. - JWX / strfmt / React Router: v3.3.0 / v0.27.2 / v7.18.4.
Console’s embedded frontend is rebuilt from this graph. Its Go dependency uses
the canonical pseudo-version because the historical module path has no /v2
suffix. Server embedders must copy both Console and MC replacements; see the
Console integration notes.
Review the password-permission migration
before upgrading: admin:ChangeMyPassword and admin:CreateUser now express
separate operations. The Server changelog
and source links below describe the storage, IAM and HTTP changes needed for
coordinated Server upgrades.
Storage, IAM and HTTP repairs
The following changes shipped in Server 20260916 and are absent from Server 20260903. Use the linked PRs and source records to check each repair’s scope.
| Area | Merged PRs | Operator-visible behavior |
|---|---|---|
| Multi-pool storage | #188 #189 |
Ordinary single-object version DELETE reconciles copies across pools; reconciliation preserves tag state. The opt-in GET-frequency pool-tiering feature was removed. |
| Conditional multipart completion | #190 | Preconditions use the logical current object across all pools, preventing an older pool copy from accepting a stale ETag or rejecting the current one. |
| Multipart discovery and cancellation | #198 #213 |
Server 20260916 defaults to legacy listing and cancellation. Explicit strict mode discovers persistent uploads across pools and sets, continues after native marker uploads disappear, and requires majority cancellation confirmations. Strict mode requires a coordinated writer upgrade and legacy drain; see the upgrade contract. |
| Ordinary conditional PUT | #207 | Public write conditions use the logical current object across all pools, including draining pools. Readability and destination-version changes are detailed below. |
| IAM revocations | #191 #192 |
Peer deletion notifications reload committed state. Durable deletion versions and retained revocation boundaries prevent stale site replay from restoring revoked identities or their older grants. |
| Replicated tags and delete markers | #193 #196 |
SSE-KMS copies preserve tag revision times; tag deletion advances its revision and resists delayed events. Delete-marker purges retain their identity and retry state through MRF recovery. |
| Delete-marker purges | eb4f5e5b3254b19ac0358ab38fb |
An exact-version purge never creates the marker on drives that lack it, a retried purge of a missing version needs a write-quorum majority of absent drives, healed markers keep their replication and purge metadata, and a queued marker creation is re-checked against the source under the replication lock before it is sent. Remaining gaps are listed below and in #217; see the third round of the replication record. |
| Listing under drive disagreement | 8d06424b1 |
A null object version that has listing quorum is kept when a newer minority of drives would otherwise hide it. Rolling restarts with concurrent overwrites can still omit readable keys from a successful LIST; see below, #218 and the design record. |
| Pool migration tags | fced86303 |
Rebalance and decommission carry object tags and their revision fields to the destination pool for ordinary and multipart writes. Tags lost by earlier migrations are not recovered; see multi-pool object consistency. |
| Replica metadata | #194 | Restoring replication metadata no longer reintroduces the transport-only aws-chunked encoding into stored object metadata. |
| Request-header timeout | #196 | --read-header-timeout / MINIO_READ_HEADER_TIMEOUT reaches the HTTP server and imposes an absolute HTTP/1 header-reading deadline, even while bytes keep arriving. HTTP/1 bodies retain the existing rolling idle timeout. |
Upgrade and compatibility requirements:
- IAM requires a coordinated upgrade of all participating servers. Mixed old/new nodes sharing an IAM backend and rolling downgrade are unsupported. Back up complete IAM storage and required encryption material; a live admin export omits deletion history. Older credentials for a recreated parent may need reissuance. Pre-upgrade deletions whose history is already lost cannot be reconstructed automatically. Follow the IAM upgrade and rollback guide.
- Unavailable pools can now fail writes and deletes more consistently.
Conditional multipart completion fails if any pool’s metadata is unreadable,
even when GET/HEAD can use another pool. Ordinary version DELETE also fails
on unreadable pools or cleanup errors; insufficient read quorum returns
503 SlowDownRead. Retry after recovery. Successful deletion does not promise immediate removal from every drive while outbound delete replication is pending. - Access-frequency pool tiering was never in Server 20260903. Only builds containing the experimental feature need its configuration/XML migration. Ordinary lifecycle expiration, remote-tier transitions, rebalance and decommission remain available.
- Purge audit status is normalized from
COMPLETEtoCOMPLETED. Malformed historical tag revisions can fail and retry; this repair does not reconstruct their history. A shorter header timeout also constrains TLS handshake reads; this is not a new total-duration limit for HTTP/1 uploads or downloads. - Objects with recorded tag revisions take one extra metadata COPY per object during explicit resync or heal; when the destination is KMS-encrypted by bucket default, that copy rewrites the object data. Replication rules with tag filters still evaluate target eligibility against the post-deletion (empty) tagging state, and arbitrary site clock skew remains outside the repaired ordering guarantees. Both endpoints of a replication pair must run the repaired build for tombstones to be honored; an old peer still drops empty-value revisions. See Replicated Tag Ordering.
The R4–R8 integration record contains source hashes, local tests and remaining acceptance limits. PR #196’s 11 checks passed before merge.
Console shared downloads
Console #56 and Server #209 resolve the anonymous proxy boundary reported in Console #52. The proxy only accepts object-content GETs at the configured S3 origin and rejects redirects, system paths and query-selected non-download operations. No sharing-disable environment variable was added. Normal public, presigned and versioned downloads remain supported; see the behavior and design tradeoffs.
The fix is released in Console v2.4.1. The exact release source passed the complete CI matrix, vulnerability checks and release workflow. Real API and browser sharing tests passed in standalone and embedded deployments; the downloaded standalone binary also passed the sharing regression against a SILO fixture.
Ordinary conditional PUT
The separate cross-pool conditional PUT defect in
#199 was reproduced on published
Server 20260903. PR #207 was merged as
9b4ae82a29cc
after its head 4620be394b52 passed all eight CI checks on 2026-09-16.
The repair shipped in Server 20260916. Its behavior is:
- Ordinary multi-pool
If-Match/If-None-Matchconditions use the logical current object across all pools. Unreadable metadata can prevent acceptance even when GET still works from another pool; read-quorum failures return 503. Restore readability or heal before retrying. - When a destination
versionIdis supplied, the public condition still compares the current object. The requested destination version is preserved; internal replication keeps its addressed-version checks. Unconditional PUT and single-pool conditions retain their existing behavior. - A successful conditional overwrite does not retire stale copies in other pools, and upgrading cannot recover historical accepted overwrites. Existing modification-time/pool ordering remains in use; this adds no global clock ordering guarantee.
The multipart-completion fix in #190 neither introduced nor repaired this PUT defect. Final packaged-candidate and rollout acceptance remain tracked in #203.
Work still pending
- Upgrade and historical-state readiness: #200 tracks the IAM upgrade/restore rehearsal; #201 tracks historical replica inventory and repair validation.
- Release delivery: #202 collects release notes and component identities; #203 tracks final artifact and multi-process validation.
- Multi-site delete-marker convergence: the source-side re-check in
254b19ac0does not cover creations already in flight or replayed from another site, and minority marker copies left by a crash after a majority-acknowledged purge have no proven persistent cleanup owner. #217 tracks the receiver-side design; see the design record. - Listing during rolling restarts: #239 (shipped in Server 20261018) carries the follow-up to #218: confirm eligible unresolved latest-object entries, distinguish deletion from read failure and preserve terminal errors through pagination. It adds no disk format, client parameter or migration switch. See the current design and acceptance limits.
- Pool migration and historical tags: the final 20261018 artifacts passed four local 2/3-pool rebalance/decommission cases, including tags, versions, markers and interruption/recovery. See the October migration repair. Tags lost before the repair need a separate audit.
- Multipart listing: #79 remains open. Server 20261018 withdraws the default persistent scan and retires the optional strict mode. Public listing again uses a node-local cache or an exact-key lookup; complete discovery, prefix/pagination semantics and delayed creation writes remain unresolved. See the current contract and capacity evidence.
Dependency and release order
- Verify the upstream SDK commit and required fixes. Keep the upstream
github.com/minio/minio-go/v7path; the retiredsilo-gofork is not part of the maintained graph. - Validate and publish pkg under
github.com/pgsty/silo-pkg/v3, including its migration notes. Resolve the tag through the Go proxy and checksum database. - Update MC to that pkg and SDK, validate it, then publish the calendar-tagged mcli release. Go consumers select its canonical pseudo-version.
- Update Console’s direct pkg requirement and explicit MC replacement, validate its embedded frontend and integration, and publish Console when a release is intended. An accepted immutable source commit can also be selected explicitly.
- Update Server’s direct pkg/SDK requirements and both PGSTY replacements, client archive hashes, image and Helm client pins. Validate the complete graph before a separate Server release, image publication and cluster rollout.
Go does not inherit a dependency module’s replace directives. Server must
explicitly select both maintained Console and MC even when Console already
selects MC. Console and MC retain their historical MinIO module paths; pkg uses
its own path directly. Legacy transitive minio/pkg/v3 from colorjson/dperf
is separate from the maintained policy implementation.
Console v2.4.2, mcli 20261018 and the Server 20261018 release use Go 1.27.2; the September stack used Go 1.27.1. The pkg v3.14.x line retains a Go 1.26 library floor and was race-tested with Go 1.26.8. Server 20261026, standalone Console v2.5.1 and v2.6.0, mcli 20261026 and 20261109, and pkg v3.15.2 and v3.16.0 require Go 1.27.3. The Go x/* dependencies were refreshed. The effective go-systemd version stays v22.6.0 because v22.7.0 fails to compile on NetBSD; Console’s tablewriter v0.0.5 replacement preserves its MC API.
Other dependencies are updated for concrete CVE and bug fixes. The September 13 Go scans found no reachable or imported vulnerable package, while retaining module-only GO-2026-5932 in unused OpenPGP code.
The supported integration target is the coordinated PGSTY stack. Compatibility with unmodified upstream MinIO/MC and other S3 implementations is best effort.
September 16 Server source review
Server 20260916 source 2a4d51406b7e selects pkg v3.14.1, upstream minio-go 32e1f32cb176, Console v2.4.1 source 1360e26d976d, and MC e952aa78f10a (mcli 20260916). The published Server tag is RELEASE.2026-09-16T00-00-00Z, package version 20260916000000.0.0.
- #213 retains legacy as the multipart default. Strict requires
MINIO_API_MULTIPART_LISTINGin the process environment and restart, with no shared dynamic key. Majority cancellation confirmation is strict-only. See settings and the upgrade contract. - Multi-pool consistency, conditional DELETE and Object Lock replication ordering distinguish logical current objects, addressed versions and persistence locks.
- Federated CopyObject and SSE-C replica integrity describe logical-byte checksums, encryption and historical-object limits.
- Durable IAM revocation and bucket configuration convergence document coordinated upgrades, deletion history and the default-off
MINIO_SITE_REPLICATION_METADATA_TOMBSTONESswitch. - Request-header timeouts and Go 1.27 TLS/OIDC explain their separate scopes.
- The September security chronicle records SN-2026-012/013/014 and their distinct component delivery boundaries.
These later Server repairs are not in 20260903; original Object Lock fixes and other released prerequisites retain their explicitly identified earlier release boundaries.
Final storage repairs also include migration tag preservation in fced86303, null-version quorum in 8d06424b1, and exact-version purge, marker metadata and queued-creation fixes in eb4f5e5b3 / 254b19ac0 / 358ab38fb. Late creation/crash residue in #217 and restart-time listing omissions in #218 remain known limitations.
October 15 source acceptance
Included in the Server 20261015 draft; not shipped in the public Server 20260916.
PR #233, merged as
d94b3b3c9251,
clears absent fields when reusing legacy xl.meta headers and keeps scanner
updates separate from final caches. Storage formats, RPC fields and dependency
versions are unchanged. The problem and repair record
documents deterministic regression coverage, a two-process scanner check,
post-merge CI, and operator verification. The source repair entered the
Server 20261015 draft and shipped publicly in Server 20261018.
Repairs after 20261015 included in 20261018
20261018 adds commit-metadata error classification (#232), upload-time parsing (#236), checksum and Console errors (#229/#238), IAM Admin authorization and bulk self-listing (#242, 995439260), LIST recovery and directory preservation (#239, 3dd53eb81, 2080b3d03), empty upload-parent cleanup (0f9199e0c) and withdrawal of public persistent scanning (4733e1423). The shared package is now v3.14.3. The final candidate also includes migration overwrite/delete-marker repairs (60c7179dd, 1748f62c7) and durable STS enumeration, revocation propagation and missed-notification recovery (0cd388c66).
The 20261018 release notes cover behavior, upgrade steps and remaining work. Former strict users lose durable listing and strict cancellation; #79 and #217 remain unresolved.
2 - Password Permissions
Applies to Console v2.4.1 with pkg v3.14.1 and the matching SILO Server. The Console button, session capability and Server authorization must use the same action mapping. Follow the coordinated upgrade steps below and the component matrix.
Breaking change: the password-permission split changes the meaning of existing IAM policies. The same stored policy can authorize a request after this update that it denied before, or deny a request it previously authorized, following the adoption of minio/pkg #262.
SILO separates a user’s own password change from creating users or resetting
another user’s password. The same add-user administration endpoint and mcli
commands continue to work; the authenticated caller and target access key
determine which permission is checked.
| Request | Permission | Evaluation |
|---|---|---|
| Change the caller’s own password | admin:ChangeMyPassword |
Allowed for an internal user with an attached policy unless explicitly denied. |
| Create another user or reset another user’s password | admin:CreateUser |
Requires an explicit Allow; an explicit Deny wins. |
The Console’s Change Password button uses admin:ChangeMyPassword.
admin:CreateUser continues to control user administration. The password change
still requires the current password. STS and service-account credentials cannot
change their parent user’s password; root credentials and external identity
provider passwords remain outside this endpoint.
What changes and why
Previously, both operations checked admin:CreateUser. Changing one’s own
password used an implicit grant unless that action was explicitly denied;
managing other users required an explicit Allow. The split retains these two
evaluation rules but checks admin:ChangeMyPassword for the caller’s password,
allowing operators to control self-service password changes and user
administration independently.
The following cases assume an internal user with an attached policy, matching statement conditions, and no other applicable grants or denies:
| Existing policy | Own password before | Own password after | Create/reset another user, before and after |
|---|---|---|---|
| S3 read grant only | Allowed | Allowed | Denied |
Deny admin:CreateUser |
Denied | Allowed | Denied |
Deny admin:ChangeMyPassword |
Allowed | Denied | Denied |
Allow admin:CreateUser |
Allowed | Allowed | Allowed |
Allow admin:CreateUser plus Deny admin:ChangeMyPassword |
Allowed | Denied | Allowed |
Deny both actions, or Deny admin:* |
Denied | Denied | Denied |
Wildcard denies that match admin:CreateUser but do not match
admin:ChangeMyPassword, such as admin:Create*, have the same password
compatibility change as the explicit CreateUser deny. An Allow never overrides
a matching Deny. Granting only admin:ChangeMyPassword does not grant user
administration.
The policy JSON format, stored documents, and endpoint remain unchanged; saved policies are not rewritten automatically and there is no legacy action-mapping toggle.
Preserve the behavior of existing policies
A saved Deny admin:CreateUser still prevents user creation and password resets
for other users. It no longer prevents the caller from changing their own
password. If an existing policy used that deny to lock the caller’s password,
add admin:ChangeMyPassword to the same Deny statement before upgrading.
For example, change that statement’s action list to:
Preserve the statement’s other actions, resource scope, conditions, and all other statements in the policy. Check policies attached through groups as well as directly to users. The preceding SILO package version already recognizes both action names, so this dual deny can be prepared before the Server upgrade. Saved policy files are not migrated automatically; the operator must apply this change where the old password restriction is intended.
To adopt the new split and lock only the caller’s password while allowing
separately granted user administration, deny only admin:ChangeMyPassword.
That finer distinction is enforced only by Servers containing this change.
Built-in read-only policies
The built-in readonly policy now grants its original S3 read operations
without its previous CreateUser deny. This has two compatibility effects:
- A user of the old
readonlypolicy could not change their own password; with the split they can, unless another applicable statement deniesadmin:ChangeMyPassword. - The old built-in
readonlydeny overrode a separate CreateUser Allow. The new built-in definition allows that independently granted user administration. This can broaden effective permissions for users with both policies attached.
The added consolereadonly policy also grants ListBucket for Console browsing
and follows the new split. Neither read-only policy grants user administration
or S3 writes on its own. Their S3 permissions do not implicitly lock passwords.
Saved policies and user overrides of canned policies are preserved on upgrade.
A saved copy of the old read-only policy retains its CreateUser deny and still
blocks a separate CreateUser Allow, even though it no longer blocks self-service
password changes. Where no saved override exists, Server uses the updated
built-in definition. Review the effective policy documents rather than assuming
every policy named readonly has the same contents. To retain both old
restrictions, attach a policy denying both actions or retain the saved readonly
override and add ChangeMyPassword to its existing deny.
Console and package callers
silo-pkg’s Policy.IsAllowedActions now reports admin:ChangeMyPassword as
implicit unless denied, and reports admin:CreateUser only when explicitly
allowed. Its Go signature and the Go compatibility floor are unchanged, but
its returned capabilities change. Console and other consumers must stop using
the CreateUser capability as a proxy for permission to change one’s own password.
Coordinated upgrade and rollback
Upgrade SILO Server, silo-pkg and Console together, including the Console embedded in Server. Update mcli’s shared package and SDK pins as part of the same maintained stack. Mixed versions disagree about the self-service action and may show a button the Server refuses, hide an allowed operation, or fail to enforce a new password-specific deny on an old Server.
Before upgrading, export the affected user/group policy documents, review the cases above, and apply both denies wherever the old combined restriction must survive. Keep both denies throughout a rolling upgrade and any rollback window. Verify self-service password changes and other-user creation/password resets with the affected accounts. Complete the Server rollout and update Console before relying on the new independent permissions.
Rolling back the binaries does not convert policies. An old Server ignores
Deny admin:ChangeMyPassword for this endpoint, so that deny alone cannot lock
the password after rollback. Restore or retain the CreateUser deny when the
password must remain locked; on the old Server it will also deny management of
other users. The old Server cannot represent the new combination of allowing
user administration while denying only self-service password changes.
Upstream MinIO compatibility remains best effort; the supported integration
target is pgsty/silo.
3 - Access-Tiering Removal
Release boundary: the opt-in GET-frequency pool-tiering feature (community PR #60) existed only in main/snapshot builds. Server 20260903 predates it and Server 20260916 was tagged after its removal, so upgrading from either published version needs no access-tier configuration cleanup. The removal merged as PR #188; only deployments that ran a build containing the feature need this page.
Why it was removed
Access tiering was a default-off, opt-in scheduler that moved objects between local server pools according to GET frequency. As shipped it dragged a disproportionate maintenance surface across configuration, lifecycle parsing, the usage cache, statistics, and the core multi-pool write path — for a benefit that was never measured or claimed (no throughput, latency, or lock-traffic numbers existed). The removal decision kept everything that was independently correct:
- ordinary lifecycle expiration and transitions to remote tiers;
- rebalance and decommission;
- the general multi-pool write, metadata, healing and conditional-deletion fixes from PR #178, including shared remote-tier reference protection.
The keep/remove split was validated by isolation controls: reverting the PR #178-era fixes wholesale failed 10 of 13 grouped checks, so they were deliberately preserved while the tiering feature itself went.
Before upgrading a build with access tiering
- Save a copy of the server ILM configuration and each affected bucket’s lifecycle XML. Use an API client that preserves the nonstandard XML; do not rely on a client model that silently omits unknown elements.
- On the old server, set
ilm access_tiering=offand remove or disable any access-tier environment overrides. Let in-progress moves finish before replacing nodes. This reduces movement intermediate states; the removal itself changes no storage RPC protocol. - Remove top-level
AccessTierQuotaand rule-levelAccessTransitionelements. Delete rules whose only action wasAccessTransition. For a mixed rule, retain its filter, status, ID, and ordinary expiration/transition actions. An access-only rule loads harmlessly after upgrade, but becomes an actionless rule and fails validation on the next lifecycle edit. If no rules remain, delete the lifecycle configuration through the S3 API. - Use a coordinated maintenance window: stop the deployment, install the same new binary on every node, then restart all nodes. The bootstrap check compares binary checksums and server environment settings — in a four-node test the first new node could not finish starting among three old nodes, and removing an old environment override on only some nodes can block startup even with matching binaries. Do not replace one node at a time.
- After restarting, verify object reads, bucket listing, ILM worker settings, and a lifecycle edit. Check storage access from every request-serving node to each pool’s drives rather than relying solely on admin disk summaries or a fixed sleep.
What happens to stored state
| State | Behavior after removal |
|---|---|
| Ten old ILM keys | access_tiering, access_pools, access_max_size, access_promote_watermark, access_bin_width, access_bins, access_flush, access_min_residency, access_workers, access_max_tracked are accepted but ignored. Existing transition/expiration worker settings are preserved. |
| Admin configuration | Deprecated keys may still appear in mcli admin config get ilm; setting them may succeed but has no effect, even with access_tiering=on. Remove obsolete environment settings from deployment manifests. |
| Lifecycle XML | AccessTierQuota and AccessTransition are ignored when read and omitted when re-encoded. The same parser handles new PUT requests, so these extensions are also silently discarded there; access-only rules still fail action validation. |
| Data-usage cache | Both v8 and v9 caches are read, preserving ordinary counts, sizes, histograms, and remote-tier statistics. The retired hot-tier byte count is discarded; subsequent writes use v8. No feature-driven full statistics rebuild is required. |
| Objects already moved | Remain in their current pools with the same versions and timestamps. There is no bulk move-back or object metadata rewrite. |
| Internal leftovers | x-minio-internal-ilm-atier and .minio.sys/config/ilm/access/ counter objects may remain unused. They need no cleanup service or object scan. |
Interrupted rebalance/decommission can leave the same version in more than one pool independently of access tiering; removing the scheduler does not remove such existing copies.
Upgrade acceptance and drive-readiness probe
The coordinated-upgrade procedure was validated in a local four-node,
dual-pool experiment (single Docker Linux host, tmpfs drives): three full
upgrade acceptances passed after a readiness-check correction, binding the
production code at 41aa84609. To verify full node-to-drive readiness: from each request-serving node, probe a unique,
never-written object key with GetObjectTagging — a path that waits for all
drives — and match the responses against real storage trace entries
(storage.ReadVersion returns the expected missing-object/version response),
requiring a real response on every node-to-drive path for three consecutive
rounds: 32 responses per round in this four-node, eight-drive topology.
drive not found, timeouts, and missing trace entries fail this check.
Two earlier upgrade attempts before the readiness-check correction encountered a 503 on HEAD/GET after a 204 DELETE.
Version deletion scope
The removal rode together with a repair of ordinary single-object
DELETE ?versionId=..., which now reconciles the addressed UUID, null
version, or delete marker across pools (including unqualified DELETE of a
directory marker). A successful request applies the deletion to every resolved
pool copy under existing per-pool quorum rules; pending outbound delete
replication retains VersionPurgePending until the replication worker
completes the purge. If a pool is unreadable these requests can return
503 SlowDownRead (or retain other error codes) even when another pool has a
readable copy; retry after recovery. Batch DeleteObjects already fans out
across pools. Incoming replicated deletes, lifecycle expiration, free-version
cleanup, and movement-internal calls keep their existing contracts.
Historical record
The feature’s introduction, subsequent fixes, rollback scope, review history,
and the unresolved validation findings are preserved in the archived decision
record at pinned commit
40220bd836cb
and in the repository’s
docs/bucket/lifecycle/access-tiering-removal.md;
this page carries the operator-facing contract so neither is required
reading. Component status lives in the version matrix.
4 - Migrate from MinIO to Silo
Migrating from MinIO to Silo reuses existing object data volumes in place without an object-by-object export and import. Ordinary S3 applications require no code changes; administrators only need to verify deployment settings, authorization rules, and state compatibility.
The current published target is Server 20261026 (review its upgrade requirements and October compatibility delta, especially if your deployment used strict multipart listing). Check the applicable O01–O08 conditions in the three-level compatibility overview before following this guide. RPM/DEB installations are covered in Native Package Migration, and component pairings are listed in the version matrix.
What changes
In order of operational impact:
- Container image:
minio/minio,quay.io/minio/minio, andpgsty/minioare replaced bydocker.io/pgsty/silo. - Package, systemd service, and server executable:
minio→silo. - Upstream services: the in-place self-updater (
mc admin update) and MinIO-operated callhome/SUBNET are disabled; upgrades are managed through packages, container images, or your orchestrator. - Default OS service account:
silo(fresh installations only; migrations continue running under the existing data owner). - Default local configuration directory:
~/.minio→~/.silo(fresh processes only; see server compatibility for the certificate fallback order — an existing~/.minio/certsdirectory continues to be honored). - Branding: startup banners, Console UI, log messages, and documentation links identify the product as Silo.
What stays
- Object layouts, erasure formats, and the
.minio.sysmetadata directory remain unchanged, allowing data drives from compatible baselines to be reused directly (see rollback scope for downgrade constraints). - Existing buckets, object versions, users, access keys, IAM policies, lifecycle rules, and encryption configurations remain active. Specific authorization, replication, and metadata tightening rules are documented in O02, O07, and the target release notes.
- Common S3 APIs, SigV4 signing, SDKs,
mc/mcli, and presigned URLs work as before; checksum validation, conditional requests, and error-code refinements are listed under O04. - Network and mounts: endpoint hostnames, API port
9000, Console ports, and volume mount paths remain unchanged. - Runtime configuration:
MINIO_*environment variables and existing server CLI flags remain unchanged. - Management and metrics paths:
/minio/*HTTP routes,x-minio-*headers, andminio_*Prometheus metric names remain unchanged. - Policy-namespace identifiers:
arn:minio:*ARNs,minio:s3, and other service namespaces in IAM policies, event notifications, and audit logs keep their exact spelling (verify authorization semantics under O02).
These guarantees apply to the comparison baseline and compatible erasure deployments. Note that while data volumes are reused in place, server startup, background tasks, and subsequent writes can update persistent metadata in .minio.sys (such as IAM revocation records or bucket configurations); always take an offline consistent backup before switching binaries.
Single-node field report
Community contributor Paris Y. (@yupoet) documented a single-node production upgrade on 2026-09-26 in SILO #228 (archived report):
| Item | Reported scope |
|---|---|
| Source version | MinIO RELEASE.2025-10-15T17-29-55Z |
| Target version | SILO RELEASE.2026-09-16T00-00-00Z |
| Deployment topology | Docker Compose, single node, single drive, server /data |
| Identity and integrations | Root credentials only; no IAM sub-users, OIDC, notification targets, or reverse proxy |
| Data volume | ~5,900 objects (~34,000 underlying storage chunks); volume opened sequentially across 2025-09, 2025-10, and 2026-09 builds over ~15 months |
| Observed result | Existing volume mounted cleanly without errors; application checks covering multipart uploads, parser-service reads, listing, and deletion passed |
Modern single-drive storage vs. legacy FS mode
The deployment above uses single-node, single-drive erasure storage (xl-single), not the removed legacy fs mode. Because the startup command server /data looks identical for both, inspect .minio.sys/format.json (read-only; do not edit it) before migrating. The erasure loader accepts xl and xl-single and rejects the legacy fs backend.
Legacy fs and gateway modes were removed in MinIO 2022-10-29. Deployments still running those modes must follow the FS/gateway export migration procedure rather than swapping the image in place.
Because a single-process root-only deployment does not exercise peer binary checksums, the removed ReadMultiple internode RPC, sub-user password-policy migration, replication, or external identity providers, multi-node or IAM-heavy deployments should also review the sections below.
Docker migration procedure
The maintained container image repository is:
Pin an exact published RELEASE.YYYY-MM-DDTHH-MM-SSZ tag (and preferably its verified digest) rather than rolling latest. The historical pgsty/minio repository does not redirect to pgsty/silo.
The procedure below covers a single Compose-managed storage process. Keep application writes paused until verification finishes; shared IAM, external etcd, and replicated deployments must also follow their coordinated upgrade procedures.
1. Record the deployment and prepare recovery
- Record runtime identity: save the current source image ID and registry digest, architecture, Compose project and service names, resolved volume or bind-mount paths, runtime UID/GID, port mappings, and server flags. Keep a local or archived copy of the old image.
- Back up configuration and keys: save the original Compose file, environment/secret sources, TLS certificate mounts, and any required KMS/encryption recovery material. Inventory versioning, Object Lock, server-side encryption, lifecycle rules, IAM, replication, and external etcd to determine which post-upgrade checks apply.
- Record representative baseline objects: before upgrading, record keys, version IDs, byte sizes, and independent content checksums for representative existing objects (including old, large, multipart, and encrypted objects; note that multipart and encrypted ETags are not plain MD5 hashes).
- Rehearse in isolation: test the exact upgrade and recovery steps on an isolated copy with outbound access to production IAM, replication, and tiering endpoints blocked.
The snippet below shows the standard image and native health probe. Do not blindly overwrite your existing Compose file with this snippet: preserve your actual Compose project name, volume declarations, runtime user, credential sources, and mounts (changing a Compose project or volume name can silently mount a brand-new empty volume).
In the standard image, the entrypoint script translates a leading minio argument to silo, so command: minio server /data continues to work; hard-coded entrypoint: /usr/bin/minio overrides or scripts invoking that path must be updated. Standard images include curl and mcli (with an mc alias), whereas -distroless images do not. Prefer the native healthcheck across both variants.
2. Stop writes and take a consistent backup
Pre-pull the target image while the old container is still running to minimize downtime:
(Replace minio here and below with your actual Compose storage service name.)
Stop application writers, scheduled jobs, and other clients, then stop the storage container and ensure no orchestrator restarts it during backup:
Confirm that no process has the volume open for writes, then take an offline volume backup or storage snapshot. Stopping only application clients is insufficient because background scanner, lifecycle, and healing workers inside the server continue mutating .minio.sys and object metadata; archiving a live directory produces an inconsistent backup.
Include the entire volume (including .minio.sys), preserve ownership, permissions, and filesystem attributes, and verify that the backup restores cleanly to a separate path. Never run docker compose down -v during an upgrade, as -v deletes named data volumes.
3. Start the replacement storage service
Confirm that the old container is stopped, the existing volume is attached, and the configured UID/GID can access data, certificates, and keys. Start the storage container without starting dependent application containers:
Never run old and new containers against the same volume simultaneously. Complete verification before reopening application traffic.
Distroless variant
pgsty/silo:<RELEASE-tag>-distroless ships only /usr/bin/silo — no shell, mc, curl, or wrapper entrypoint script. Its ENTRYPOINT is /usr/bin/silo, so pass server arguments directly (command: server /data --console-address ":9001") without a leading minio or silo token.
The image includes a built-in native HEALTHCHECK. Because an existing Compose healthcheck block overrides the image default, remove any inherited curl or CMD-SHELL probe or replace it with the exec-form ["CMD", "silo", "healthcheck", "ready"] probe shown above. The default certificate directory is /tmp/.silo/certs; set MINIO_HEALTHCHECK_URL if the listener requires a custom probe URL. Standard and distroless variants of the same release share the exact same on-disk format and can be swapped freely.
Kubernetes
Kubernetes kubelet executes httpGet probes defined in the Pod spec and ignores the Docker HEALTHCHECK instruction, so standard and distroless images behave identically under existing Kubernetes probe configurations. For Helm deployments, preserve the release identity and PVC bindings with nameOverride / fullnameOverride, and diff helm template output before applying (details).
Rollback
On-disk object layout compatibility does not mean an older binary understands new metadata written after an upgrade. Before rolling back an image or binary, verify whether persistent state must be restored as well:
- Durable IAM revocation history: releases with persistent IAM revocations do not support rolling downgrades. Follow IAM upgrade and recovery before upgrading to back up complete IAM storage and key material (standard
mc admin cluster iam exportarchives omit deletion/revocation tombstones). - Bucket metadata convergence and password permissions: check bucket-configuration deletion state and password permission rollback rules.
- Executing recovery: if post-upgrade verification fails, keep clients blocked, stop the new server, and restore the pre-upgrade volume backup, configuration, and pinned image digest. Reconcile any writes, deletes, or credential revocations that occurred after the backup before reopening the endpoint.
Behavior checks for a single-node migration
| Area | What to verify |
|---|---|
| TLS handshakes and CA trust | 20260903 and 20260916 updated TLS behavior; 20260916 removes explicit curve overrides in favor of Go defaults (enabling hybrid post-quantum key exchange) and changes macOS custom CA loading under Go 1.27. Check inbound clients and outbound OIDC, notification, replication, and tiering connections, including TLS-inspecting middleboxes. See Go 1.27 TLS and OIDC discovery. |
| Conditional deletion | Since 20260916, single-object DeleteObject enforces nonempty If-Match headers and returns 412 PreconditionFailed on ETag mismatch (batch DeleteObjects is unchanged). See conditional DELETE. |
| Multipart listing | 20260916 offered an optional MINIO_API_MULTIPART_LISTING=strict disk-scan mode; 20261018 removes it and restores legacy in-memory cache behavior (omitting max-uploads defaults to 1,000, though the legacy cache path does not enforce strict pagination). See 20260916 behavior and 20261018 upgrade requirements. |
| Reverse-proxy source-address trust | When MINIO_API_TRUSTED_PROXIES is unset, the S3 listener preserves historical trust of forwarded client-IP headers. Set none for direct client connections, or explicitly list proxy IPs/CIDRs and ensure proxies overwrite client-supplied headers. Console and LDAP STS follow their own trust rules. See source-address trust. |
| Identity and object state | If your deployment uses IAM sub-user password policies, service accounts, STS, OIDC, Object Lock, or replication, review the target release notes, password permission migration, and IAM recovery. |
Upgrading from RELEASE.2026-08-06
Releases after RELEASE.2026-08-06T00-00-00Z tighten several behaviors that 20260806 accepted. Review this checklist before upgrading:
- Explicit version deletes require
s3:DeleteObjectVersion.DeleteObjectandDeleteObjectsentries carrying aversionIdare authorized againsts3:DeleteObjectVersion, matching AWS S3. Grant this action to principals that delete specific versions, and addDeny s3:DeleteObjectVersionalongside anyDeny s3:DeleteObjectrule intended to block permanent deletion. - Enable and disable are separate admin actions.
admin:EnableUser/admin:DisableUser(and their group equivalents) are checked against the requested target state; policies granting only one action can no longer perform the opposite operation. - New and updated policies reject bare ARN prefixes (such as
arn:aws:s3:::) and statements combiningResourcewithNotResource. Existing stored policies continue loading, but automation that re-applies malformed policies will fail. - Legacy database notification targets require a connection string. An enabled pre-KV PostgreSQL or MySQL target missing
connection_string/dsn_stringhalts startup with a credential-redacted error, whereas 20260806 silently dropped all notification targets in that state. - Checksum headers are strictly validated. Unknown
x-amz-checksum-*algorithms,CRC64NVMEcombined withCOMPOSITE, and checksum-type assertions contradicting the upload return400 BadRequest. Default AWS SDK,minio-go, andmclirequests are unaffected. - Per-bucket CORS configurations take precedence. A bucket with its own CORS rule is served exclusively by that rule;
MINIO_API_CORS_ALLOW_ORIGINapplies only to buckets without one. In site-replication groups, configure bucket CORS only after all sites run the new release so older peers do not ignore the rule and report CORS drift. - Rollback to 20260806 drops per-bucket CORS. Because 20260806 ignores bucket CORS and strips it when rewriting bucket metadata, export bucket CORS rules before rolling back and restore them after re-upgrading.
Console regressions in the September 3 release
RELEASE.2026-09-03T13-18-01Z stopped recognizing forwarded client addresses from unconfigured local proxies and ignored the four CONSOLE_WS_MAX_* WebSocket connection limits, affecting client-IP Allow/Deny rules and locking anonymous connections at the default per-IP limit of 8.
20260916 and later releases (#147, #148) restore loopback TCP-peer trust for the embedded Console unless MINIO_API_TRUSTED_PROXIES=none/off, and honor all four CONSOLE_WS_MAX_* limits from the environment or MINIO_CONFIG_ENV_FILE. Remote proxies still require an explicit IP/CIDR allowlist. See Console settings for full details.
On 20260903, adding the local proxy peer to MINIO_API_TRUSTED_PROXIES restores client-IP attribution, but also switches the port 9000 S3 listener into explicit allowlist mode (requiring all S3 proxies to be listed as well).
One cluster, one binary
Distributed nodes verify each other’s binary checksum during bootstrap. A node started alongside peers running a different binary stays in activating indefinitely and logs:
This check applies between any two distinct binaries — whether MinIO next to Silo, or two different Silo releases. Do not migrate or upgrade a distributed cluster node by node. Switch all nodes in a single pass: stop the old binary on every node, then start the new binary on every node (in Compose, update the image across all nodes and run docker compose up -d once). The same rule applies to rollback.
Single-node deployments are unaffected. Routine rolling restarts of the same binary work normally; gate each node restart with silo healthcheck --maintenance cluster (exit code 0 confirms the node is safe to stop).
Verification
Verify the upgraded service while application traffic is still paused:
- Confirm instance and volume identity: check the running version/image digest, attached volume identity, expected bucket list, and startup logs to ensure the container did not mount a fresh empty volume.
- Run readiness probes: execute
silo healthcheck readyinside the container, or probe the HTTP endpoint from the host (curl --fail --silent --show-error http://127.0.0.1:9000/minio/health/readyfor a default local HTTP listener; supply the proper hostname and CA bundle for TLS). - Verify baseline historical objects: read back the pre-recorded historical objects through your application or SDK and compare byte contents, sizes, and independent checksums. Verify object versions, tags, encryption, retention, and legal holds where used.
- Exercise write and listing paths: using a dedicated test bucket or key prefix, test single-part uploads, multipart completion, listing, read-back, and deletion with your real client (do not delete production objects as a smoke test, and respect bucket versioning/retention when cleaning up test keys).
- Verify identities and integrations: test sub-users, service accounts, STS/OIDC flows, and expected
AccessDeniedresponses. For shared IAM or replicated deployments, verify every participating endpoint following the IAM upgrade procedure. - Confirm restart persistence: restart the storage service once while traffic remains paused, re-verify readiness and signed historical-object reads, and then reopen application traffic.
If any historical metadata or object read fails verification, preserve the error logs and object identifiers for investigation on an isolated copy, and avoid introducing optional new state during the migration window so rollback remains straightforward.
5 - Silo vs. MinIO Server Compatibility
Start with the three-level overview: 12 compatible improvements, 4 minor differences and 8 conditional checks. This page provides server details, preserving the fixed 2026-08-06 audit baseline and separating subsequently released changes from September 16 source changes. The historical audit baseline is separate from the original fork baseline.
Read this before replacing a MinIO deployment. The binary, package, service account, systemd unit, default local configuration directory, container path, Helm resource names, embedded Console, update behavior, several authorization decisions, and some error responses changed. Data disks and the MINIO_* configuration namespace did not receive a matching rename.
Candidate work after Server 20261026
Server 20261026 remains the published Server release. These links collect design and upgrade records for subsequent candidate work. Each record’s status, source commit and validation scope apply separately; inclusion in this index does not establish release acceptance. Independently released Console and mcli versions do not add these changes to older Server artifacts.
| Area | Design and upgrade references |
|---|---|
| Lifecycle, whole-key deletion and restore | Multi-pool lifecycle, exact-key deletion, restored-copy expiry, two-phase restore |
| Locking, reads and healing | Single-pool lock domains, partial-view reads, heal dry runs, failed-write cleanup |
| Batch and configuration reload | Batch authorization and retries, webhook peer reload, IAM revocations |
| Containers | Healthchecks and distroless images |
Server 20261026 upgrade checks
This section covers the Server 20261026 release, source c3ad836aed0d.
| Scenario | Behavior change and upgrade action |
|---|---|
| LDAP users with long normalized DNs | Group-based login continues past missing optional IAM records whose ASCII filename component exceeds 255 bytes; unreadable revocation records and storage errors still fail closed, and writing overlong path records remains unsupported on the object backend. See the LDAP guide. |
| Built-in policy overrides | Reset removes an override and immediately restores the native policy while retaining in-use checks; delayed peer/etcd notifications reload committed state, and custom-policy deletion still deletes the policy. See Console policies. |
| Console bucket creation/deletion | account-info reads the live bucket inventory (older Servers can return a stale cached list after refresh); usage statistics remain scanner-derived. |
| Storage timeouts and rollback | Preserves drive and internode RPC timeout and cancellation propagation, and forwards internal delete options during overwrite rollback (draft mutation guards and unbounded completion waits are not included in this release). |
| Multi-pool deletion | Ordinary and bulk deletion clean up non-authoritative copies before removing the authoritative copy and protect shared remote tier references; retry failed deletes after drive recovery. |
| Versioned remote-tier expiration | Lifecycle expiration that only writes a delete marker preserves the prior version’s remote tier data (including non-null versions in suspended buckets); data deleted on earlier releases must be restored from backups. |
| Multipart replication | Forwards the FULL_OBJECT checksum type for composite CRC algorithms; upgrade the sending Server and verify pending replication queues. |
| Build and packaging | Built with Go 1.27.3, Console v2.5.1, mcli 20261026, pkg v3.15.2, and pinned upstream minio-go; Windows archives use ZIP. See the component matrix. |
Read and failure boundaries: Lease expiration or network partitions can still allow late commits, and partial metadata updates can still expose stale values or tags. Existing read-quorum and healing rules remain unchanged (stricter fencing, rollback and read-recovery diagnostics remain under the consistency_strict build tag); upgrading does not add distributed fencing or automatically repair pre-existing inconsistent objects. The 20261018 Object Lock, IAM, and retired multipart-mode upgrade requirements below continue to apply.
Server 20261018 upgrade checks
This section targets the 20261018 release, now publicly released. Older audits and the 20260916 sections below retain their historical scope.
| Workload | Change and action |
|---|---|
| Object Lock / version-addressed writes | Retention requires its base permission and applicable bypass permission; addressed writes require trusted replication; directory and old multipart targets are rechecked under the write lock and fail closed. Upgrade all write-accepting sites. |
| Service accounts / STS / Admin automation | Explicit self denies and session restrictions apply. Restricted credentials cannot delegate broader access; non-root administrators cannot mutate root-derived credentials. Check the actual actor and external plugin. |
| Object inventories and backup synchronization | Eligible uncertain LIST entries are confirmed or cause an error. Read every page and never turn an error into an empty inventory; no cross-page snapshot is promised. |
| Multipart listing and cancellation | The default remains cache/exact-key lookup. The strict switch is ignored with a warning; former strict users lose durable enumeration and strict cancellation. Restart omissions, prefix/pagination limits and delayed creation remain. |
| Errors / OIDC | Lost staged commit metadata returns 503 SlowDownWrite; checksum message text can change. JOSE rejects RSA JWKs above 4096 bits, so verify login before upgrading. |
See IAM authorization, LIST recovery, the multipart decision and security update. #79 and #217 remain unresolved; disk formats and established configuration names are unchanged. The PGSTY stack is supported, with best-effort upstream compatibility.
Audit scope and method
| Boundary | Audited value |
|---|---|
| Upstream baseline | minio/minio@7aac2a2c5b7c882e68c1ce017d8256be2feea27f, 2026-02-11 |
| Silo snapshot | pgsty/silo@219670d3176a5b27ded60914390d5ee7e763cf58, 2026-08-06 |
| Commit set | 7aac2a2c..219670d3: 96 reachable commits; 93 were on origin/main and the final three were committed locally at audit time |
| Net source diff | 523 files, 36,715 insertions, 21,450 deletions |
| Interpretation | The behavior of the final snapshot. Intermediate changes later replaced or removed are not presented as current behavior |
Every commit in the range was inspected against the final implementation, tests, dependency graph, build recipes, package payloads, container entrypoint, and rendered Helm manifests. The complete commit ledger tracks every commit in the range, including documentation and CI updates.
The range is defined by Git reachability, including d4cd4b433 (authored in December 2025 and merged into the post-baseline graph later).
The tagged RELEASE.2026-08-04T00-00-00Z ends at d88f46cce, 18 commits before this 2026-08-06 audit head (which subsequently shipped in Server 20260806).
Executive compatibility matrix
| Surface | Status | Practical result |
|---|---|---|
| S3 wire API | Compatible with documented exceptions | Routes, XML/JSON schemas, SigV4, S3 headers, ports, and ordinary error codes retain the MinIO contract. Security and correctness fixes below deliberately reject some requests previously accepted |
| Data disks | Compatible | .minio.sys, erasure metadata, bucket/object layout, healing, replication, and encryption formats keep their names and schemas. Poisoned or unusable metadata is now rejected earlier |
| Configuration | Mostly compatible | Existing MINIO_* variables, config keys, KMS/KES, IAM, notification, and storage settings remain. The default per-user directory becomes ~/.silo, with a deterministic ~/.minio fallback |
| Metrics and automation APIs | Compatible | minio_* Prometheus metrics, /minio/* routes, x-minio-* headers, admin/S3 error identifiers, and release tag syntax stay unchanged |
| Binary and distribution | Renamed | minio becomes silo; package, unit, image, chart, archives, checksums, and paths move to the Silo identity. There is no installed server-binary alias |
| Runtime identity | Changed | CLI text, banners, HTTP Server, User-Agent application names, FTP banner, log names, support links, and some human-readable errors say Silo |
| Upstream network services | Disabled | In-place update, update polling, callhome, SUBNET registration, and diagnostic uploads do not contact MinIO services |
| Authorization/security | Intentionally stricter | OIDC HMAC tokens, unsafe LDAP failures, forged replication metadata, object-only grants for protected bucket writes, shadowed policy inputs, ambiguous version IDs, and several malformed internode requests change behavior |
| Embedded UI and Go dependencies | Console and MCLI forked behind compatible import paths; pkg on its own path | Silo Console and MCLI are selected with replace directives while github.com/minio/console / github.com/minio/mc paths remain; Silo Pkg is consumed directly under github.com/pgsty/silo-pkg/v3 |
| Mixed-version cluster | Not supported for this transition | The private ReadMultiple storage-REST operation was removed without bumping storage REST v63. Upgrade all nodes as one build |
What deliberately stays compatible
Protocol, storage, and configuration names
The following MinIO identifiers are compatibility identifiers, not unfinished branding work, and must remain visible:
- the Go module path
github.com/minio/minioand the inheritedgithub.com/minio/...imports; - the
MINIO_*environment namespace, includingMINIO_ROOT_USER,MINIO_ROOT_PASSWORD,MINIO_VOLUMES,MINIO_OPTS, and notification variables; - S3 and admin routes under
/minio/*,x-minio-*headers, MinIO-specific S3 extensions, and established API error codes; - Prometheus metric names under
minio_*; - the
.minio.sysinternal volume and all existing disk metadata names; - the default S3 port
9000, existing--address/--console-addressflags, and release tags of the formRELEASE.YYYY-MM-DDTHH-MM-SSZ; - configuration KV formats, IAM data, KMS/KES configuration, encryption metadata, bucket metadata, replication state, and healing state.
The rebrand guard checks an explicit identifier inventory. The archived 219670d3 manifest contained 137 imports, 436 environment names, 19 metric namespaces, 84 headers, 330 routes, one root, three Grid namespaces, 15 storage identifiers, 58 policy identifiers and 9,014 exported symbols. The current source manifest is version 4 and no longer tracks exported symbols. Run make rebrand-guard against the exact source revision to review intentional inventory changes.
No data copy or metadata rewrite is required when the same disks move from MinIO to Silo. This does not mean every malformed historical object is accepted: the storage hardening described below rejects unsafe paths, invalid erasure geometry, negative part sizes, and poisoned metadata that older code could carry farther into the stack.
Source compatibility
The server module remains github.com/minio/minio, and Console and the client library are selected through replacements without forcing callers to rewrite imports:
The shared package is the deliberate exception. Since pkg v3.13.0 it builds under its own module path, and every maintained component — including the server — consumes it directly:
The historical replace github.com/minio/pkg/v3 => ... arrangement is retired; github.com/minio/pkg/v3 now appears only as a legacy indirect dependency (via colorjson/dperf), separate from the maintained policy implementation. The upstream SDK github.com/minio/minio-go/v7 is the other explicit exception: it is consumed directly from upstream at a verified commit, and the retired silo-go fork is not part of the maintained graph.
This preserves most source compatibility, but it is not an assertion that every private or exported Go symbol is frozen. The internal ReadMultiple storage interface was removed, and the selected silo-pkg release has several developer-visible fixes described in Dependencies.
The maintained source remote is github.com/pgsty/silo on branch main; the former minio branch is archived. go install github.com/minio/minio@... still resolves the upstream project, not Silo, so clone the Silo repository or use an explicit module replace. Contributions no longer require MinIO’s CLA, but commits require DCO sign-off (git commit -s).
Identity, binary, and outbound-service changes
| Surface | Upstream baseline | Silo snapshot | Compatibility consequence |
|---|---|---|---|
| Server executable | minio / minio.exe |
silo / silo.exe |
Scripts and absolute paths must change. Archives, packages, and images do not install a /usr/bin/minio server alias |
| Version output | MinIO identity | Silo release/commit/runtime, AGPL, upstream copyright, PGSTY modification copyright, and MinIO technology lineage | Parsers should rely on stable fields, not grep for MinIO prose |
| Local config home | ~/.minio |
~/.silo for a new home |
See the fallback rules below; data disks are unrelated |
| HTTP identity | Server: MinIO and MinIO application UAs |
Server: Silo; internal batch/fan-out/perf UAs use silo-* / Silo names |
Protocol headers such as x-minio-* remain unchanged; identity-sensitive monitoring may need an update |
| Human text | MinIO banner, help, errors, examples, FTP greeting, support links | Silo identity; examples prefer mysilo |
Exact-string log parsers and snapshots may change, not status/error codes unless listed elsewhere |
| Integration-visible labels | MinIO NATS/Redis connection names and Veeam model | NATS name Silo Notification, Redis CLIENT SETNAME Silo, Veeam model "Silo <release>" |
Broker dashboards, connection-name filters, and Veeam inventory display can change |
| KMS validation prose | MinIO-branded conflict messages | Brand-neutral “both KMS/KES/static-key configuration” messages | Configuration rules are the same; exact-text automation can change |
| Updater | Release polling and in-place update paths | Permanently disabled | MINIO_UPDATE is parsed but cannot re-enable it; admin update routes remain and fail stably instead of disappearing |
| Callhome/SUBNET | Registration, callhome, support uploads, embedded MinIO support key | Configuration is accepted for migration but forced off; no registration/upload/post; no fallback encryption key | Remove automation that expects MinIO-operated services. Requester-key inspect encryption remains |
| Embedded Console | Upstream snapshot had the Console stripped | Silo Console v2.1.1, English/Chinese UI, Metrics V3, no SUBNET UI | Browser behavior and assets change; the S3/Admin API boundary remains the server contract |
| Bundled client in OCI | No maintained fork contract | /usr/bin/mcli plus /usr/bin/mc -> mcli |
This mc is the client compatibility alias, never a server alias |
| Warm-tier probe | Temporary object contains MinIO |
Same-length probe contains Silo! |
Only observable through backend inspection or a failed cleanup; protocol semantics do not change |
| Log rotation default | minio-*.log |
silo-*.log |
Log collectors matching filenames must change |
The deleted /api/health/upload reference was an outbound SUBNET URL path, not a local Silo HTTP endpoint. The compatibility change is that Silo no longer issues that POST; it is incorrect to describe this as removal of a server route.
Default configuration-directory selection
Unless --config-dir is explicit, Silo makes one decision at startup:
| Home-directory state | Selected directory | Message |
|---|---|---|
| Neither directory exists | ~/.silo |
none |
Only ~/.silo exists |
~/.silo |
none |
Only ~/.minio exists |
~/.minio |
informational legacy notice; no files are moved |
| Both exist | ~/.silo |
ambiguity warning |
--config-dir always wins. Unless --certs-dir is also supplied, the certificate directory follows the selected configuration directory. For deterministic automation, set --config-dir instead of depending on filesystem discovery.
The fork adds only three server configuration controls that materially change compatibility behavior; there is no parallel SILO_* replacement namespace:
| Setting | Purpose | Default |
|---|---|---|
MINIO_API_TRUSTED_PROXIES |
General source-address trust boundary | unset: exact historical trust-any behavior |
MINIO_IDENTITY_LDAP_STS_TRUSTED_PROXIES / LDAP key sts_trusted_proxies |
Source buckets used by LDAP STS failure limiting | no trusted proxy; use the socket peer |
MINIO_API_LEGACY_BUCKET_RESOURCE_MATCH |
Temporary rollback for the protected bucket/object IAM boundary | off |
Notification KV registration fixes do not rename their existing environment variables. MINIO_UPDATE, SUBNET, and callhome inputs are retained only as ignored/migration-compatible inputs as described next. SILO_OPTS appears only inside the generated inspect helper, not as a general replacement for MINIO_OPTS.
Updater, callhome, SUBNET, and inspect
- Startup never polls
dl.min.io; the release URL and upstream minisign root are absent. MINIO_UPDATEvalues that request updates produce a warning and are ignored. Public and peer admin update handlers remain registered, returningMethodNotAllowedor the stable “in-place updates are disabled” error.- Legacy
subnetandcallhomekeys remain parseable so an old config can start. Registration state is always false, Console SUBNET variables are unset, callhome is forced off, and no diagnostic or license payload is posted. - Inspect output is encrypted only for a requester-provided public key. It no longer falls back to MinIO’s built-in support key. The helper is
start-silo.sh, invokessilo, and includescluster.infoonly in the requester-key flow. - Upgrade through a package manager, an image rollout, or an orchestrator. Do not call
mc admin update/mcli admin updateagainst Silo as an upgrade mechanism.
Installation and deployment compatibility
RPM, DEB, and APK
The package is silo for Linux amd64/arm64. Its relevant payload is:
The package creates a system silo:silo account without a home. It does not chown existing data, migrate ownership, stop a running MinIO service during installation, or declare package-manager Provides, Obsoletes, Replaces, or Conflicts against the minio package. Both packages can therefore be installed, but their services cannot run together through the shipped units.
silo.service has Conflicts=minio.service, runs as silo:silo, and reads /etc/default/minio first and /etc/default/silo second. The shipped Silo file has no active assignments, so a legacy file continues to work until an administrator overrides it in the later file. It starts:
Before switching the unit, make every data, certificate, KMS credential, and environment-file path readable by silo and every writable path writable by it. An old deployment owned by minio:minio will otherwise fail at startup. Package removal stops/disables silo.service; package upgrades do not deliberately stop the running service in the pre-remove hook.
Container image
The source repository is github.com/pgsty/silo; the intended image name is docker.io/pgsty/silo. There is no registry-level promise that pgsty/minio redirects to it.
- The server exists only at
/usr/bin/silo; an explicit/usr/bin/minio ...command breaks. - The entrypoint translates a first argv word of
miniotosilo, and prependssilowhen argv starts withserver,fmt-gen, or an option. Consequently the commoncommand: minio server /dataform continues to work. - An explicitly requested shell or utility is left alone.
- Every privilege path uses
exec, so the server becomes PID 1 and receivesSIGTERMfor graceful shutdown instead of timing out behind the entrypoint. HOME=/tmpis the image default and is normalized to a writable directory for arbitrary-UID and legacyMINIO_USERNAMEdrop-user execution.- Port
9000,/data, and theMINIO_*interface remain. The amd64/arm64 image manifest also contains checksummed MCLIRELEASE.2026-08-06T00-00-00Z(bumped in the release commit after this audit head) and the client-onlymcsymlink. - OCI license material is under
/licenses/{LICENSE,NOTICE,CREDITS}.
Helm chart
The inherited helm/minio chart, helm-releases, root chart index, and reindex helper were removed. The maintained chart is helm/silo; the 20260806 release ships chart version 7.0.1 (the audit head still carried 7.0.0).
Most values deliberately keep their established names, including minioAPIPort, minioConsolePort, and all MINIO_* environment settings. The changes that matter during migration are:
- image repositories become
pgsty/silo; - generated resource names and labels follow chart name
silo; - default service account becomes
silo-sa; - certificate/client mount paths move from
/etc/minio/{certs,mc}to/etc/silo/{certs,mc}; - no insecure
console/console123user is created by default (users: []); - post-job examples prefer alias
mysilo, whilemyminiois also registered so inheritedcustomCommandscan still resolve it; - the new chart executes
silo, so an image-only rollback to an old MinIO image is not safe.
To preserve the old Kubernetes object identities while adopting the new chart, start from the exact old values and set at least:
Render both charts and compare Services, selectors, StatefulSets/Deployments, PVC templates, Secrets, service account, storage mounts, environment, and ports before applying. Roll back chart and image together.
Archives, provenance, and legal files
Release archives are named silo_<version>_<os>_<arch> and contain the executable, README, LICENSE, and NOTICE. The checksum manifest is silo_<version>_checksums.txt; each archive receives an SPDX JSON SBOM, and the checksum set is accompanied by a keyless Sigstore bundle. Static, CGO-disabled, kqueue-tagged binaries are published only for Linux, macOS, and Windows on amd64/arm64; formerly compile-checked but unshipped architectures are no longer release gates. RPM/DEB/APK are built for Linux amd64/arm64, with timestamp package versions such as YYYYMMDDHHMMSS.0.0 (RPM release 1) and a separate package checksum manifest. Normal builds no longer stamp the build host’s GOPATH/GOROOT, improving reproducibility and removing path leakage.
Unlike the upstream baseline Docker recipe, which downloaded and verified a prebuilt dl.min.io server, the Silo release image consumes the exact source-built, attested release archive at the selected tag. Image publication is a separate, explicitly dispatched workflow after a GitHub release; build success alone does not publish it.
CREDITS is regenerated from the modules actually linked into the server and guarded in CI. It is included in the OCI image; it is intentionally omitted from packages and archives because of its size. Upstream AGPL and copyright notices remain, alongside PGSTY’s modification notice.
Runtime and security behavior changes
These are user-visible changes even when they close a vulnerability. “Stricter” means a request, policy, token, configuration, or corrupted internal message that formerly succeeded or failed differently can now be rejected.
Authentication, IAM, and request identity
| Change | Final behavior | Who must act |
|---|---|---|
OIDC JWT verification (d24f449e0) |
The client secret is no longer a verification key. Only asymmetric JWKS algorithms RS256/384/512, ES256/384/512, RS3256/3384/3512, and ES3256/3384/3512 are accepted. HS256/384/512 tokens fail; unknown kid still triggers the established JWKS refresh/retry path |
An IdP signing Silo tokens with HMAC must migrate to an asymmetric JWKS key |
LDAP STS errors (3b950f8fa) |
Unknown user and bad password share one external InvalidParameterValue authentication failure. LDAP infrastructure failures remain server errors and are logged |
Clients must not distinguish account existence from response text |
LDAP STS rate limiting (18b712d49, 9e10f6d9a, f44110890, 5e40665ac) |
Per-source, per-node in-memory bucket: burst 10, refill one per 6 seconds, idle TTL 15 minutes. Only authentication failures consume tokens; success and infrastructure failures refund them. Exhaustion returns HTTP 429, ThrottlingException, Retry-After: 6 |
Proxies should configure MINIO_IDENTITY_LDAP_STS_TRUSTED_PROXIES; it is separate from the general source-address setting |
| LDAP trusted-proxy source | For an allow-listed socket peer, clean X-Real-IP is preferred; otherwise XFF is walked right-to-left past trusted hops. RFC 7239 Forwarded is ignored. A proxy must overwrite X-Real-IP |
Review ingress header sanitation; the limiter is not a distributed account lockout |
| LDAP service-account lookup | “User DN not found” matching is case-insensitive, preserving the intended Admin no-such-user / login-name error classification across dependency message capitalization | Only brittle clients that depended on the accidental misclassification see a difference |
Bucket/object IAM boundary (97b7d2804) |
Twelve protected bucket-write actions no longer inherit an Allow from only arn:aws:s3:::bucket/*; the bare bucket ARN is required. Deny/NotResource and built-in * policies retain their semantics |
Add arn:aws:s3:::bucket to custom policies that legitimately perform protected bucket writes, or use the temporary global escape hatch MINIO_API_LEGACY_BUCKET_RESOURCE_MATCH=on |
| Protected actions | Delete/ForceDelete bucket; put/delete bucket policy; put replication, lifecycle, object-lock, versioning, or CORS; delete CORS; put bucket QoS or inventory configuration | Read/list, create bucket, tags, encryption, and notifications keep the inherited matching behavior |
Effective policy inputs (2f55347f7) |
Server-derived conditions cannot be shadowed by a same-named header/query parameter. Exact keys win in the policy package. Existing/request tags, storage class, content/copy/checksum, object-lock, signature age, and list parameters are sourced from the value actually consumed by the operation | Policies that accidentally depended on attacker-controlled shadow values stop matching |
| Request tags | PutObject, CreateMultipartUpload, and PutObjectTagging bind s3:RequestObjectTag/* to the parsed input. ExistingObjectTag comes only from stored metadata. Other operation paths keep the inherited header fallback |
Re-test tag-conditioned write policies |
s3:signatureAge |
Present only for verified presigned SigV4 requests | Raw x-amz-signature-age injection no longer creates the condition |
s3:versionid (744a9dcd7) |
Absent means absent, whitespace is normalized, and DeleteObject/MultiDelete uses the effective per-object version. A URL version cannot decoy a different XML version | Re-test version-conditioned delete policies and any policy relying on Null |
Replication metadata (56fa63bfd) |
Ordinary PUT/COPY cannot inject internal replication status/time metadata. An authenticated replica request must have ReplicateObjectAction; multipart and Snowball replication flows retain their legitimate metadata |
Custom replication callers must use the authorized replication path |
The bucket-boundary escape hatch is startup-global and all-or-nothing. It exists for migration, not as a permanent mixed-policy mode. An empty or unparseable LDAP source is deliberately not placed into one shared limiter bucket; it is not throttled until a usable source can be derived.
General source-address trust
fe6dc4780 adds MINIO_API_TRUSTED_PROXIES, because the chosen client address feeds aws:SourceIp, audit remotehost, event Host, admin trace, and node-to-node forwarding.
| Value | Result |
|---|---|
| unset | Exact inherited behavior: trust source headers from any peer; left-most XFF, then X-Real-IP, then Forwarded |
none or off |
Ignore all three source-address headers and use the TCP peer |
| IP/CIDR list | Trust headers only when the TCP peer is listed; walk XFF right-to-left past trusted hops (maximum 100), then use the last X-Real-IP line, then walk Forwarded right-to-left |
Malformed entries, a non-empty list naming no proxy, or a remote env:// read failure fail closed and stop startup rather than reverting to trust-any. Invalid values inside a received chain are skipped; no usable address falls back to the peer. Loopback is implicitly trusted only for the FTP/SFTP peer bridge, not skipped as an arbitrary hop inside a client chain.
The inherited _MINIO_API_XFF_HEADER=off retains its exact old semantics and initialization timing: it disables only XFF parsing, not X-Real-IP or Forwarded, and therefore is not a security boundary. Configure every cluster peer that legitimately forwards requests and ensure edge proxies overwrite or strip client-provided source headers.
S3 request and response behavior
| Area | Change and compatibility effect |
|---|---|
| Presigned streaming auth | A query/presigned SigV4 request declaring STREAMING-UNSIGNED-PAYLOAD-TRAILER is rejected with SignatureVersionNotSupported; it cannot fall through to anonymous authorization. Header SigV4 is verified before body processing |
| Snowball | Authorization happens before tar extraction; the streaming-trailer bypass cannot write objects before a later failure |
| S3 Select record limits | CSV input, JSON Lines input, and output records over 1 MiB return an OverMaxRecordSize event. JSON Lines always uses the bounded reader (possibly slower on SIMD-capable CPUs), JSON parse errors use JSONParsingError, and already completed records can precede the terminal error |
| Streaming responses | The tracking writer implements Flush; Write/Flush records an implicit HTTP 200. ListenBucketNotification/watch streams and S3 Select keepalives reach clients, while audit/status metrics record the committed status correctly |
| Multipart full-object checksum | FULL_OBJECT CRC32/CRC32C/CRC64NVME completion may omit every per-part checksum. If any are supplied they are still checked; COMPOSITE still requires every part. Explicit completion type is validated even without an object checksum value: mismatch is BadDigest, unknown type is InvalidArgument. CRC64NVME + COMPOSITE is rejected by the later #93/#96 repairs, included in Server 20260903. A zero-byte multipart object’s checksum is retained correctly |
| CopyObject checksum response | CopyObject XML and HTTP headers report the committed checksum using destination encryption context. SSE-C source key A and destination key B remain separate; same-object key rotation reports checksum fields under the new key. Stored data and checksum format are unchanged |
| Multipart part ordering | Duplicate or non-increasing part numbers fail with InvalidPartOrder before assembly. Gaps and a first part other than 1 remain legal; the upload stays available for retry |
| Erasure read pooling | Correct shard-buffer ownership is restored, avoiding loss of pooling and a wrong-buffer association that could cause hangs, corruption, or severe performance loss |
| Update buffers | Returned download buffers are owned correctly; the public updater was subsequently disabled, so no current supported upgrade path exercises this code |
Distributed storage and private APIs
These changes are normally invisible to an S3 client but are compatibility changes for mixed clusters, custom internal callers, corrupted disks, and adversarial peers.
ReadMultipleand its private storage-REST/rmplendpoint, client method, exported Go types, and metric were removed. External S3 List/Get operations are unchanged.storageRESTVersionremains v63, so do not infer mixed-node compatibility from the version number.- Every remote StorageAPI path field, nested metadata name, and raw-volume sink is validated at the storage boundary, including peer-S3 Grid messages. Lexical traversal, volume-root aliases, and Windows separator/drive forms are rejected.
- Erasure geometry, non-positive blocks, negative part sizes, and unusable stored erasure metadata are rejected at all decoded sinks and during CheckParts/VerifyFile.
- Internode allocation declarations are bounded: AppendFile caps preallocation at 1 MiB while still accepting the body; DeleteVersions grows as it decodes and rejects negative declarations; legacy ReadFile is capped at 5 GiB.
- Deadline-bounded work converts worker panics into errors and logs a bounded stack instead of taking down the process.
ReadPartskeeps the actual backend error across keepalive frames. An empty part list returns a successful empty result without a trace panic or leaked goroutine.- HTTP stream helpers left orphaned by
ReadMultiplewere deleted later; that cleanup creates no additional public behavior change.
This containment is lexical. It does not resolve filesystem symlinks, and native Windows server CI was not available for the audit. Upgrade every node together and keep untrusted clients away from the internode port even though root credentials and validation protect it.
Notification configuration and audit output
- NATS now registers parser-consumed
user_credentials,nkey_seed, andtls_handshake_first; AMQP registersimmediate. Existing environment variable names stay unchanged. - The old literal
MINIO_NOTIFY_NATS_USER_CREDENTIALSremains accepted for NATS only. Precedence is environment, new key, then old migration key. - AMQP legacy migration maps
immediatecorrectly. Invalid notification errors identify names but no longer echo credential values. - PostgreSQL and MySQL notification configuration uses the canonical
connection_stringordsn_string; explicit strings pass through unchanged. The retained internal PostgreSQL fallback builder quotes values and uses the correctuserkeyword, but discrete fields are not a supported KV input. - Dangling-object deletion audit events once again include per-drive errors under
merrs.
The inherited database-notification migration risk is closed by f1ba68358: migration no longer writes unregistered discrete connection keys. An enabled pre-KV target without connection_string or dsn_string now stops startup with a credential-free error. Convert, disable, or remove such targets before upgrading; diagnostic bundles exported before this fix may contain a historical plaintext database password and should be treated accordingly.
Toolchain, dependencies, and embedded components
The build declaration moved from Go 1.24 plus a 1.24.8 toolchain to go 1.26.5. That can change TLS, HTTP, DNS, scheduler, garbage-collector, and standard-library edge behavior even where no Silo source line changed. Security-sensitive dependencies were also advanced, including Go-Jose, OpenTelemetry, Go crypto/network modules, cloud SDKs, etcd, NATS, and compression libraries.
Observable edge corrections inherited through those updates include Go TLS/X.509/URL/archive fixes; MQTT oversized UTF-8 packet encoding; malformed Azure NTLM challenge handling; Thrift framed transport and 32-bit compilation; NATS authentication, authorization, identity, and denial-of-service fixes; and Prometheus remote-read/write and UI hardening. The jsonparser CVE-2026-32285 investigation produced no patch: the resolved v1.1.2 already contained the fix and no vulnerable reachable symbol was found, so it creates no compatibility delta in this range.
Important deliberate dependency decisions are:
| Component | Final selection | Compatibility rationale / effect |
|---|---|---|
| Console | pgsty/silo-console v2.1.1 behind github.com/minio/console |
Restores the embedded UI, applies Silo branding and bilingual text, adds Metrics V3, removes SUBNET flows, and fixes untranslated metric legends |
| Client library | pgsty/mc behind github.com/minio/mc |
Keeps Console’s import path while consuming the maintained MCLI fork |
| Shared package | pgsty/silo-pkg/v3, consumed directly at its own module path (v3.14.1; behind minio/pkg/v3 via replace up to v3.12.x) |
Breaking module-path move in v3.13.0; supplies the IAM exact-match half, LDAP TLS/StartTLS/deadline/close fixes, certificate-watcher cleanup, and RNG fixes |
| Kafka | Sarama 1.45.1 | Pinned to avoid a breaking broker-negotiation drift |
| PostgreSQL | lib/pq 1.10.9 | Pinned to avoid a nil-[]byte / PostgreSQL-before-14 behavior regression; generated DSN quoting is fixed in server code |
| Compression | klauspost/compress 1.18.7 | Explicit security/correctness upgrade |
| Thrift | 0.24.0 | Fixes 32-bit builds |
| systemd library | require 22.7, replace with 22.6 | Retains NetBSD compilation until the monotonic-clock regression is fixed upstream |
The LDAP package now honors TLS fields for ldaps://, keeps StartTLS active even with server_insecure, avoids a nil-TLS panic, applies a StartTLS deadline, and closes a connection after failed StartTLS. Certificate-watcher shutdown no longer leaks; on Windows, polling can delay reload by up to about ten seconds. RNG subkey entropy/reset behavior is corrected, although the server does not exercise the reset path.
For external Go consumers of silo-pkg, two changes are broader than this server’s own call paths: xtime.Duration JSON moves from integer nanoseconds to duration strings, and some AIStor action vocabulary / protected-action helpers differ from upstream. In particular, Policy.IsAllowedActions can disagree on protected actions; the server does not call it. The server stores the relevant state through YAML/msgp and does not call the differing AIStor/action helper paths, so no server data migration or authorization delta was found from those library changes.
Generated String() files were regenerated under the new toolchain. Valid enum output remains the same; the diff is generator provenance and invalid-value formatting machinery, not a separately claimed S3 behavior change.
Changes since the 2026-08-06 audit
The sections above describe the 219670d3 snapshot. The table below records the behavior changes merged to main after it and consolidated in the 20260903 release.
| Area | Change | Where |
|---|---|---|
| Authorization | Explicit version deletes are authorized as s3:DeleteObjectVersion and DeleteObjects authorizes each entry; user and group status changes require the action matching the target status; policy writes reject bare ARN prefixes |
#104, 58735ee38, 229fe2b3c, eee05a17c (SN-2026-005, -009, -010) |
| Replication trust | Replication-only headers grant replication semantics only with the exact marker and s3:ReplicateObject / s3:ReplicateDelete; otherwise they are removed after signature verification |
#101 (SN-2026-008) |
| SSE-C | Zero-byte objects and GetObjectAttributes authenticate the customer key; null-version and in-place key-rotation copies no longer rewrite objects into unreadable ciphertext; CopyObject reports checksums under the destination key |
b73581b05, 474cd5801, 05df6e70d, ffb70eb37, e73436c99 (SN-2026-006, -007) |
| Checksums | Server-side part checksums, federated UploadPartCopy, ChecksumType in CompleteMultipartUpload, AWS-aligned completion errors, rejection of unknown algorithms and of CRC64NVME with COMPOSITE |
7fea6d5a5, 8d76a255c, d014a12cf, 5d152416d, 7c103389f, d28885d0e |
| Listing | ListObjects on a missing bucket returns NoSuchBucket on the shortcut paths that previously returned an empty listing |
e9c5340be |
| Per-bucket CORS | Real ?cors API; a bucket configuration overrides the global policy; the pre-authentication lookup reads resident metadata only and otherwise applies the global policy; site replication converges CORS with a last-writer-wins register |
#71, #80, #101 |
| Bucket metadata | metadata.lock serializes every bucket-configuration writer; ForceCreate and site adoption keep existing configuration; a locked bucket always carries plain Enabled versioning |
#103, dd3bdb808 |
| Site replication | Object Lock configuration replicates in its own field (the legacy Tags carrier is still accepted); status is accounted per site; validity probes verify permissions under the rule prefix |
3861f33cb, fb406fdc9, c9ad74673, 5db7be4ee |
| Configuration | Legacy database notification targets require a DSN; MINIO_CONFIG_ENV_FILE uses a dedicated parser that keeps named targets |
f1ba68358, 6b0998157, 2aea7fe9c |
| Toolchain and components | Go 1.27.1; upstream minio-go at 0e78d3f18efe (silo-go retired); silo-pkg v3.13.2; Console v2.3.0 (see the Console page); bundled mcli 20260903 |
43f4bb7ed, 4d6e1ea8e, final dependency refresh |
The shared package’s own-module move (v3.13.0, a breaking change for Go consumers) was already adopted by published Server 20260903: that tag directly requires github.com/pgsty/silo-pkg/v3 v3.13.2. The September 13 refresh moved the maintained stack to v3.14.0 and Server 20260916 requires v3.14.1; neither is the first Server adoption of the new path. See the component matrix for each build.
Changes released in Server 20260916
These changes shipped in Server 20260916 and are not included in Server 20260903. See the version matrix for per-repair scope and companion-component status, and the overview IDs for user impact.
| Area | Observable change | Classification and details |
|---|---|---|
| New administration APIs | GET /minio/admin/v3/multipart-preflight checks multipart migration readiness without modifying state. New peer GET/PUT /minio/admin/v3/site-replication/peer/iam-revisions operations synchronize revisions and revocations. |
O06 / multipart preflight; O07 / IAM protocol |
| S3 conditions and multipart listing | Multi-pool conditional writes, multipart completion and version deletion verify metadata more strictly. Strict multipart pages are capped at 1,000 and require explicit opt-in and preparation. Legacy remains the default; its cache path can still exceed the requested page size. | O04, O06 / multi-pool consistency, listing modes |
| Authorization and persistent state | Self-service password changes and user creation use separate permissions. Persistent IAM revocations and bucket-configuration deletion state affect coordinated upgrades and recovery; general bucket-configuration tombstone export is off by default. | O02, O07 / password migration, IAM upgrades, configuration convergence |
| Data and replication | Further fixes cover multi-pool replicas, tag and Object Lock ordering, delete-marker recovery, SSE-C and federated copies; they do not reconstruct lost historical state automatically. | G07–G10 / replica audit, version notes |
| HTTP and TLS | Request headers use an absolute read deadline. TLS defaults need checking with older proxies and identity providers; bodies retain the existing rolling idle timeout. | O03 / header timeouts, TLS/OIDC |
Distinguish API additions, removals and behavioral changes: per-bucket CORS implements existing placeholder APIs; ReadMultiple is a removed private storage operation; online updates retain an entry point with the function disabled. silo healthcheck and mcli checksum verify are new commands that call existing health endpoints and read objects for local auditing, respectively.
Runtime and observability
See G12 and B03. The minio_* namespace and existing metric paths remain, but the metric set and value meanings are not frozen:
- MRF drop counters are exposed through administration statistics and Prometheus v2/v3. They count queue entries; several may refer to one object, so they are not counts of lost objects.
- IAM revocation metrics report retained deletion records, reconciliation failures, duration and last success. Interpret them using each node’s collection semantics.
- Bucket-quota metrics report effective quotas, and CPU metric collection gains concurrency protection. These affect monitoring values and runtime stability respectively; no universal speedup is implied.
Review affected dashboards and alerts when upgrading.
Known residual risks and non-fixes
- Source IP is still forgeable by default. Unset
MINIO_API_TRUSTED_PROXIESdeliberately retains upstream trust-any behavior. Setnonefor direct deployments or an exact proxy allowlist for proxied deployments. - Version-condition follow-up. #104 repaired per-entry governance-bypass version authorization in Server 20260903. The Snowball PAX-version ordering observation concerns a separate path. Some identity conditions still retain present-empty semantics.
- Multipart parser defense is not complete. Handler-level ordering is fixed, but the object layer has no independent uniqueness defense; XML-root validation and the inherited nonnumeric-part error mapping were not changed.
- Legacy notification migration remains risky. Review it as described above.
- Storage path validation is lexical. Symlinks are not resolved; native Windows execution was not independently covered.
- Private APIs are not a stable compatibility promise.
ReadMultipleproves that a same-numbered storage REST protocol can still lose an operation. Do not run a rolling mixed build across this boundary. - Informational HTTP responses remain imperfectly tracked. The response-tracking layer treats a 1xx response as final. The Flush/implicit-200 change did not introduce this behavior and does not claim to fix it.
- Conditional deletion has a release boundary. Server 20260903 ignores
If-MatchonDeleteObject; Server 20260916 implements it through #145/#178, comparing the explicitly addressed version when one is given.DeleteObjectsstill ignores per-item<ETag>, and no extras3:GetObjectcondition permission is implemented. See the implemented contract. - Conditional deletion has a release boundary. Server 20260903 ignores
If-MatchonDeleteObject; Server 20260916 implements it through #145/#178, comparing the explicitly addressed version when one is given.DeleteObjectsstill ignores per-item<ETag>, and no extras3:GetObjectcondition permission is implemented. See the implemented contract. - Historical convergence limit for multi-site bucket-configuration deletion. In earlier versions before PR #180 (shipped in Server 20260916), deleting a bucket policy, SSE, tag, or quota configuration on one site could be restored by a peer that still held it (#77); only per-bucket CORS used a tombstone-aware register at that point. Full deletion recovery requires every node to be upgraded and deletion export (
MINIO_SITE_REPLICATION_METADATA_TOMBSTONES) to be enabled consistently; see the bucket configuration convergence record.
Migration checklist
For a MinIO-to-Silo move, use this order:
- Record the exact MinIO binary/tag, chart and values, image digest, package payload, service unit, environment files, config directory, data ownership, IAM policies, OIDC/LDAP settings, notification targets, and proxy topology.
- Back up configuration and IAM metadata. Silo reads existing disks in place, but a rollback still needs the old executable/config/chart and unchanged data ownership available.
- Replace invocations of the server binary with
silo; do not assume/usr/bin/minioexists. In containers, argv-levelminio serveris translated, but the absolute path is not. - Decide the config directory explicitly. Reuse
--config-dir ~/.minioor let the legacy-only fallback select it; do not create an empty~/.siloaccidentally and then wonder why the old configuration is ignored. - For packages, grant
silo:siloaccess to data, certificates, secrets, and logs. Move deliberate overrides into/etc/default/silo; understand that it overrides/etc/default/minio. - For Helm, render the old and new charts with the complete old values, preserve names with
nameOverride/fullnameOverride/serviceAccount.namewhere required, and change chart plus image atomically. - Remove updater, callhome, SUBNET-registration, and support-upload automation. Replace it with package/image/orchestrator rollout and your own diagnostic transfer path.
- Change HMAC-signed OIDC tokens to asymmetric JWKS. Exercise success, bad-password, unknown-user, backend-failure, and rate-limit LDAP paths.
- Audit custom IAM policies for protected bucket writes (requiring bare bucket ARNs unless
MINIO_API_LEGACY_BUCKET_RESOURCE_MATCH=on),s3:DeleteObjectVersionon versioned deletes, split user/group enable/disable actions, and the password-permission split. - Set
MINIO_API_TRUSTED_PROXIES=noneor an exact allowlist, sanitize all three source-address headers, and include cluster peers that forward authenticated requests. - Test oversized S3 Select records, streaming notifications, unsigned-trailer rejection, multipart full-object checksums, duplicate parts, replication, healing, KMS, every notification target, audit ingestion, and graceful container shutdown.
- Upgrade all distributed nodes as one build. Keep the old chart and image paired for rollback; never roll back only one of them.
Verification evidence
Verification at the recorded 2026-08-06 snapshot:
| Check | Result / boundary |
|---|---|
| Commit enumeration | 96/96 commits classified in the ledger below; origin/main accounted for 93 and the local prepared head for three more |
| Net diff review | All 523 changed paths classified across server runtime, internal protocol, dependency, delivery, documentation, tests, or superseded changes |
| Rebrand compatibility guard | Passed; compatibility manifest and delivery/runtime assertions unchanged, including Docker argv tests |
| Go test suite | Full go test ./... passed against 219670d31, including cmd, OIDC, LDAP, notify, event targets, Grid, handlers, hash, and all S3 Select packages |
| Helm migration | buildscripts/verify-helm-migration.sh passed: lint, render, legacy upgrade, archive, and identity comparison across seven rendered resources |
| Package lifecycle | buildscripts/package/lifecycle_test.sh passed; package payload/provenance assertions cover empty DEB conflict metadata, unit/default paths, and legal files |
| Site | Strict make check passed: module verification, warning-fatal Hugo render (617 EN / 615 ZH pages), and 388,962 internal references across 1,084 HTML files; git diff --check, bilingual anchors, and 96/96 commit coverage also passed |
Security articles contain deeper threat models and test vectors, but their historical “released/unreleased” labels describe their publication date. Where they conflict with the final snapshot, this page’s audited boundary is authoritative.
Complete commit coverage ledger
The hashes are in graph order. Merge, documentation, test, and CI commits are included because delivery behavior and the strength of a compatibility claim are themselves user-relevant; “no independent runtime delta” means exactly that, not “not reviewed.”
| Class | Commits | Verified net effect |
|---|---|---|
| Initial fork, Console, CI, dependency base | d4cd4b433, 8630937e7, 68521b37f, 00f3cf74f, 5abd9a80f, 377fc616d, f2f9a40dc, ee55e5391, ce1c537eb, 68e0ba997, 1869bd30b, ff58df949, e4fa06394 |
Go/SDK evolution; embedded Console restoration/fork; MCLI in OCI; replacement CI; LDAP TLS regression fix; security dependency upgrades. The two merge commits add no delta beyond their parents |
| April security series | d24f449e0, 3b950f8fa, 56fa63bfd, 3252d5b7f, f444b6f37, efb6e5b00, db4c0fd5e, 18b712d49, 9e10f6d9a, f44110890, f48dbe777 |
OIDC, LDAP STS, replication metadata, S3 Select, unsigned-trailer/Snowball, Go 1.26.2, limiter accounting/source hardening, and security documentation |
| May–June reliability and private API | 65795ee1f, 5e40665ac, fd69c89d0, 73ac52472, df627ff89, 3e61b1d3a, d495d30d5 |
HTTP Flush, final LDAP bucketing, full S3 Select bound, ReadMultiple removal, Go 1.26.4/dependency update, and documentation-link change |
| Pre-August component integration | ce01ccbdc, 4dfc27ce3, b7f52ca43, 7babc0c39, c1aec0518, 15fcc3c8a, 3f192f3f0 |
Historical chart image switch, security dependency upgrade, notification-stream merge, portable dependency pins, compression, MCLI replacement, Console v2.0 |
| August runtime correctness/security | c8590413f, 3e14733f1, 924717926, 89d346bf5, 8069a32ac, a36fd8fff, ca7baa670, 80e8eaa42, b6f70ab08, 1af351a70, 38366f654, 22c1e41fd, 97b7d2804, 2f55347f7, 744a9dcd7, fe6dc4780, 162ded343, 0c14d8151, 9dd1dc172, 2602177ef |
Multipart, erasure buffers, response commits, panic containment, path/metadata/allocation/ReadParts containment, orphan cleanup, IAM/effective values/version ID/source trust, notification/libpq, and audit details |
| Chart hardening and audit documentation | dfe669862, 5f4513fd4, b42ee4e8a, 8eae745ab |
Secure chart user default, portal/doc routing, maintainer ignore rules, and advisories; only the chart default changes runtime delivery behavior |
| Release engineering through the 20260804 tag | 9c799f42d, 10c7670b8, cf7df097b, 32863c852, 632ade111, 1814ae52f, 475236c79, 11d79fddc, 3b8a55dee, ca674a696, 4c185d5a6, 2ca4971d9, e064b5555, aa5139369, 021110b45, d88f46cce |
RPM/DEB/APK, provenance, OCI publishing gates, pinned lint/generation, S3 Select test-race fix, broad CI, PID-1 signal fix, published-target cross-builds, safe release dispatch, reproducibility, stale-config removal, systemd location, honest gates, and runtime-image shutdown assertion |
| Silo cutover and 2026-08-06 prepared head | 15def34dc, 77bdc4c0c, 15ab10833, 30749911b, e071bb77e, bd8df5166, 6613c2a3c, fd2ca1c6d, c46b16ec6, c47733abc, f1c77d5a2, 62717d7bf, 6740e6978, b57275be3, 05be686b8, a6d6d9b02, 6bd9cf77e, 219670d31 |
Removes unpublished MinIO delivery residue; Silo runtime identity/offline boundary; renamed packages, OCI and Helm with migration guards; pinned fixtures; docs/repository cutover; Console 2.1.0 then 2.1.1; Node 24 actions; DCO/legal/docs polish; regenerated CREDITS; LICENSE/NOTICE delivery |
The ledger totals 96 unique commits. Changes replaced inside the range—such as Console 2.0 → 2.1.0 → 2.1.1, the historical pgsty/minio image/chart state, and updater-buffer code after the updater was disabled—are described only where they leave a final compatibility consequence.
See also
- MCLI — client artifact, config-directory, update, SUBNET, and command compatibility
- Silo 20260804 release notes — the earlier tagged release boundary, not the full 2026-08-06 audit head
- Silo Pkg 3.11.0 — shared IAM, LDAP, watcher, RNG, and developer-facing changes
- OIDC JWT hardening, LDAP STS hardening, replication metadata, S3 Select limits, and ReadMultiple removal
- Unsigned-trailer query auth, Snowball authentication, and the jsonparser no-change finding
- Internode path containment, duplicate multipart parts, bucket/object IAM boundary, version-ID conditions, source-address trust, and notification key registration
6 - MCLI Client Compatibility Notes
mcli is Silo’s build of the MinIO Client (mc). This page records where the two are interchangeable and where they differ.
pgsty/mc forked from the upstream minio/mc at its final commit, 77f82e18 (2025-11-06). The upstream repository was archived in July 2026 without ever cutting a release that contains that commit — so every mcli release is strictly newer than any official mc binary ever published. Documented fork releases: 20260313, 20260321, 20260417, 20260804, 20260806, 20260903, 20260913, 20260916, 20261018, and 20261026. The latest is 20261109; historical notes retain their own comparison baselines.
Current release: mcli 20261109, package 20261109000000.0.0, source e1d4cc5abfe0. Windows uses ZIP; Linux/macOS use tar.gz, with Linux RPM/DEB/APK and Docker Hub images.
Current release changes
The 20261109 release notes cover the fixes in pgsty/mc#74, #76 and #77. The source changelog records the complete changes.
cpandmvcontinue processing independent sources after a preparation error and return nonzero overall. Later arguments can still be copied or moved; successful moves delete their sources. There is no rollback.- Per-object JSON start records contain cumulative prepared object counts and bytes in
totalCountandtotalSize. These values grow while sources are listed; they are not the final workload or a denominator for completion percentage. Check the final exit status, error records and final byte summary. share uploadgenerates POSIX-shell commands with single-quoted arguments and--form-stringfor literal fields. After substituting correctly quoted placeholder values, execute the command in a POSIX shell. If it is stored inupload, usesh -c "$upload"; executing$uploaddirectly does not interpret its embedded quotes. On Windows, use Git Bash or WSL instead ofcmd.exe.- Invalid
--attrfailscpandmvbefore any transfer. Separate pairs with semicolons:--attr 'key=value;other=value'. Empty entries, including trailing or repeated semicolons, are ignored bycp,mv,mirrorandpipe; an input with no key/value entry still fails. - Removal permission and WORM protection errors, and per-item IAM or bucket metadata import failures, return nonzero while preserving reports. Legacy IAM fallback is restricted to
NotImplementedandXMinioAdminVersionMismatch. mvrejects resolved self-moves and ambiguous same-bucket/key moves between endpoints that share a scheme, port and IP address, including--resolvemappings. Shared-IP virtual hosts may be rejected. Establish that their stores are independent before copying, verifying and removing explicitly; matching bytes through two aliases alone do not establish independence. Different addresses, ports, proxy routing, virtual-host addressing or changing DNS can still hide a shared backend. Configure aliases to one store with a consistent endpoint URL; DNS is not a general storage-identity service. A local DNS failure does not reject an endpoint that the transport reaches through an HTTP proxy; unresolved direct endpoints still fail. Proxy selection honorsNO_PROXY.- Container images pre-create
/.mcwith mode1777for non-root UIDs without a passwd entry (HOME=/) and install legal files under/licenses/(#77);--config-dirandMC_CONFIG_DIRstill take precedence.
Changes inherited from 20261026
Built with Go 1.27.3, silo-pkg 3.15.2, and pinned upstream SDK 320af050c445. Since 20261018, this release fixes identity and connection-cache isolation, admin replicate resync JSON output and terminal states, throughput and batch IOPs counter overflow, job status polling when live metrics are absent, partial-failure exit codes, alias region configuration, and SDK multipart, read, and bulk-delete handling. Automation scripts should handle the added pending state and non-zero exit codes when resync mutations partially fail (querying an already failed job still exits 0). Alias region support adds no new CLI flags, and existing configurations continue to work unchanged.
The 20261026 module version is github.com/pgsty/mc v0.0.0-20261026120214-0ceb298cdcd0. Client upgrades require no Server protocol changes, with SILO as the maintained target. See the complete changes and component matrix.
Changes inherited from 20260916
20260916 uses pkg v3.14.1, upstream SDK 32e1f32cb176, JWX v3.3.0 and Go 1.27.1.
The SDK retries and reports S3 errors embedded in single-request CopyObject
HTTP 200 responses (objects smaller than 64 MiB or explicit --disable-multipart;
larger copies use ComposeObject), so a failed copy cannot authorize mv to
remove its source. JWX fixes JSON field-name escaping; pkg v3.14.0
password-policy semantics are unchanged.
- Per-object
mirrorpermission errors are reported and accumulated while later objects continue; finite jobs exit 1. Failed local destination removals are no longer swallowed. These permission failures do not require--skip-errorsto continue; watch listing/watcher errors retain their existing cancellation/retry behavior. - Failed mirrors suppress normal final success statistics. Explicit
--summarystill reports statistics, with JSONstatus: failure; text errors go to stderr and statistics to stdout. - Failed legalhold set/clear, partial recursive retention failures, and
mvsource-deletion failures now return nonzero. Successful objects are not rolled back. Retention reports each failed object once. - Empty retention durations and invalid find regexes produce normal errors instead of panics. Very short transfers no longer produce infinite speeds in JSON statistics.
Automation migration: affected failure paths that returned 0 now return 1;
check the final exit code and error records. Configuration, aliases and MC_*
variables are unchanged. See the release notes.
Changes inherited from 20260913
20260913 uses pkg v3.14.0, upstream SDK 60bd07042d49 and Go 1.27.1, with
refreshed Go x/* modules. It preserves mirror destination history and fixes
restart dry runs, noninteractive behavior, transfer/SQL failure exits, explicit
checksums for empty uploads and quiet JSON output. Boolean environment values
accept on/off. Deny/NotResource and bounded wildcard fixes preserve clauses that
could previously be lost; recover already-lost clauses from the original policy.
The password-permission split requires
Console v2.4.1 and a matching Server. Review the existing policy denies as
part of the coordinated upgrade.
Changes inherited from 20260903
The 20260903 client keeps the upstream command, configuration, protocol, and JSON contracts, with these deliberate additions and tightenings since 20260806:
- Read-only checksum audit:
mc checksum verifystreams object data and compares storedFULL_OBJECTCRC32, CRC32C, CRC64NVME, SHA1, or SHA256 values. It supports one object, recursive prefixes, exact or all versions, JSON Lines manifests, dry runs, bounded workers, size/time filters, SSE-C, machine-readable output, private report files, and explicit failure policies. It never repairs or mutates an object. - Credential output is fail-closed:
--debug,admin trace, errors, redirects, trailers, JSON documents, SSE-C paths, custom headers, and registered runtime secrets are scrubbed before display. Support and cluster-export artifacts, including rotated copies, are created with mode0600on POSIX systems. - Correctness repairs: JSON Prometheus metrics no longer panic; empty
pipeinput uses a regular zero-byte PUT; S3 Select responses are no longer closed twice after valid rows are printed; malformed app-level global flags fail instead of being ignored; and service-account policy clearing works again. - Strict writes, permissive reads: named policy and service-account write paths reject empty or malformed policies, bare ARN namespaces, and mixed S3/admin action statements. Existing stored policies remain readable; an empty service-account policy still means “clear the inline policy.”
- Dependency and toolchain floor: builds require Go 1.27.1 and directly consume
github.com/pgsty/silo-pkg/v3v3.13.2.minio-gois pinned to upstream master commit0e78d3f18efe; gRPC is 1.83.2 andx/cryptois 0.56.0. Darwin binaries require macOS 13 or newer. - Verifiable delivery: the public release is immutable, contains 19 checksum-checked assets, and binds archives plus DEB/APK packages to the signed tag and exact source with Sigstore attestations. RPMs carry the PGSTY GPG signature. The release and
latestcontainer tags publish matching linux/amd64 and linux/arm64 manifests.
Two low-level compatibility changes are worth testing in specialized deployments: JWX 3.2 rejects non-compliant JWT crit processing more strictly, and updated Unicode width handling can slightly change table/progress alignment for Indic or ZWJ-heavy strings.
Principles
The fork follows one rule: the shipped artifact and its channels are renamed; the tool you use is not.
- Renamed / replaced — the artifact name on disk (
mcli), the product identity in--versionand--help, the distribution channels (GitHubpgsty/mc, the Pigsty repository,docker.io/pgsty/mc), and the signing keys. Not the command syntax, and — depending on how you install it — not even the name you type. - Preserved interfaces — familiar command syntax, S3/admin protocols and
x-minio-*identifiers; versioned output, authorization and error-handling changes are documented above; the configuration file format and alias semantics;MC_*environment variables (includingMC_HOST_<alias>); the.part.minioresume suffix; and the Go module pathgithub.com/minio/mc. - Severed — every connection to MinIO-operated services: the release/update feed, the SUBNET support and licensing portal, telemetry, and the pre-seeded
playdemo alias. Affected commands remain in the CLI for script compatibility and fail with a stable error rather than disappearing. - Preserved — upstream copyright and the AGPL-3.0 license. Runtime output credits both MinIO, Inc. and PGSTY.
A configuration written by upstream mc is readable by mcli unchanged, and vice versa. SILO releases test mcli against Silo; operation against upstream MinIO and other S3-compatible endpoints is retained on a best-effort basis and should be verified with the exact versions in use.
What changed
Ordered by how likely each change is to affect you, most likely first.
1. The name — what you type, and where the config lives
For many users nothing changes here: the container image keeps mc as its entrypoint, and a binary installed under the name mc behaves identically to upstream. What changed is what we ship — archives and Linux packages install the binary as /usr/local/bin/mcli (package name mcli).
Neither name is hardcoded anywhere. Since 2016 the upstream client has derived its runtime identity from the name it is invoked as, and mcli is the exact rename upstream’s own CONFLICT.md recommended (issue #873) for the Midnight Commander clash — this fork merely promoted that suggestion to the official shipping name, with zero code changes. What that mechanism means in practice:
| Follows the invoked name | Fixed, regardless of the name |
|---|---|
Configuration directory: ~/.mc vs ~/.mcli (Windows: %USERPROFILE%\mc\ vs …\mcli\) |
Environment variables: always MC_* — there is no MCLI_CONFIG_DIR |
| Program name shown in help and usage text | config.json format — identical and interchangeable in both directions |
| Shell-completion registration | All commands, flags, JSON output, exit codes |
User-Agent application suffix (mc/… vs mcli/…) |
--config-dir and MC_CONFIG_DIR overrides |
The one real trap: run mcli for the first time and your existing mc aliases are not there — it starts from an empty ~/.mcli. Either keep invoking it as mc (a symlink suffices — argv[0] is what counts), or copy the state once with cp -a ~/.mc ~/.mcli. For automation and configuration templates, set MC_CONFIG_DIR explicitly: the environment prefix does not follow the name, so one template serves both. Details in Migration.
Get it from GitHub Releases (SHA-256 mcli_<version>_checksums.txt), the Pigsty repository (RPMs GPG-signed, key fingerprint 9592A7BC7A682E7333376E09E7935D8DB9BD8B20), or docker.io/pgsty/mc. Upstream’s minisign key does not sign these artifacts, and dl.min.io is never contacted. Release tags (RELEASE.YYYY-MM-DDTHH-MM-SSZ) and package versions (YYYYMMDDHHMMSS.0.0) keep their upstream schemes.
The container image keeps aliases in /root/.mc when it runs as root. To run it as another UID (--user, Kubernetes runAsNonRoot), with a read-only root filesystem, or with aliases that outlive the container, point MC_CONFIG_DIR at a directory that UID can write — a bind mount it owns (create it first), an emptyDir, or a volume:
2. mcli update always fails — on purpose
Self-update is removed. mcli update never contacts the network and never replaces its binary; it prints an explicit notice and always exits 1. Upstream mc update exited 0 when already current, so any cron job or script that calls it and treats a non-zero exit as failure will start failing — drop the call and upgrade through your package manager or GitHub Releases instead. The per-invocation version probe against upstream release feeds is also gone, and MC_UPDATE / MINIO_UPDATE are no longer consulted.
(mcli admin update ALIAS — updating the server — still exists, but Silo servers reject in-place updates server-side.)
3. SUBNET, licensing, and telemetry commands
Everything that reached MinIO SUBNET is disabled at build time. Affected commands keep their names and flags, print a stable notice — “MinIO SUBNET services (registration, licensing, uploads) are disabled in this Silo build of mc; diagnostics remain available locally.” — and exit 1:
| Command | Behavior now | Use instead |
|---|---|---|
mcli license register |
notice, exit 1 |
— |
mcli license update ALIAS (online renewal) |
notice, exit 1 |
mcli license update ALIAS license.key (offline, still works) |
mcli support upload |
notice, exit 1 |
share files through your own channels |
mcli support proxy set |
notice, exit 1 |
proxy remove still clears a legacy setting |
mcli support callhome enable |
notice, exit 1 |
disable / status still work |
The diagnostics themselves stay: mcli support diag / perf / profile / inspect always run in local (airgap) mode — results are written to local files, nothing is uploaded, and SUBNET registration is no longer a prerequisite. Two related hardening changes: inspect no longer falls back to encrypting output with an embedded MinIO public key (your archives stay decryptable by you), and since 20260804 --debug output redacts SUBNET credentials — if you ever shared debug logs from older builds, rotate the keys in them. mcli license info and unregister work locally.
4. The play demo alias is no longer pre-seeded
Fresh configurations seed local, s3, and gcs — not play. Tutorials and smoke scripts that assume the demo alias need it added explicitly: mcli alias set play https://play.min.io <access-key> <secret-key> restores the old behavior, since nothing blocks deliberate access to any S3 endpoint. Existing configuration files are never modified.
5. Output text carries the Silo identity
mcli --version keeps its machine-readable first line and adds an identity line plus dual copyright; --help says “Silo client” and examples use mysilo. Command syntax is untouched — only scripts that grep for upstream identity strings (e.g. “MinIO Client”) need adjusting.
6. For developers
The module path stays github.com/minio/mc, so imports compile unchanged — but go install github.com/minio/mc@latest installs the archived upstream, not this fork. Build from source (git clone https://github.com/pgsty/mc && cd mc && make) or consume it via a replace directive. Contributions need no CLA but require a DCO sign-off (git commit -s). Upstream being archived also means inherited defects are only ever fixed here — most notably minio/mc#5139 (mirror --remove --watch on versioned buckets).
Migration
Moving from an official mc binary to mcli:
- Install
mclifrom one of the fork’s channels (see §1 for verification): GitHub Releases archive,yum install mcli/apt install mclifrom the Pigsty repository, ordocker pull pgsty/mc. - Decide what to call it — this determines which configuration it reads:
- Keep the
mcname (least friction): after confirming no upstream binary remains (command -v mc), install it asmc— e.g.ln -s /usr/local/bin/mcli /usr/local/bin/mc. Invoked asmc, it reads your existing~/.mcuntouched; nothing else to migrate. - Adopt the
mcliname: carry your state over once withcp -a ~/.mc ~/.mcli, or setMC_CONFIG_DIR=~/.mc. Both clients can also coexist side by side, each with its own directory.
- Keep the
- Clean up automation:
- remove
mc updatecalls — they now always exit1; - remove
license register,support upload,support callhome enable, andsupport proxy set— same stable failure; support diag/perf/profile/inspectkeep working and write local files; drop any step that expected a SUBNET upload;- review anything that greps
--versionoutput beyond the first line.
- remove
- Re-check
playusage in tutorials and smoke scripts (§4). - Verify:
mcli --version,mcli alias ls, thenmcli ls <alias>andmcli ping <alias>against your servers. - Rollback stays trivial: the configuration format is identical in both directions, so keeping the old
mcbinary around lets you switch back at any time.
See also
- Silo vs. MinIO — how the
siloserver compares tominio
7 - Native Package Migration
Version scope: commands and illustrative output target Server 20261026, now publicly available. See downloads and the release upgrade requirements.
Silo publishes silo packages for RPM, DEB, and APK on amd64/arm64 via GitHub Releases, with SHA-256 sums and build-provenance attestations. This page records what changes relative to a minio package installation: the file layout, the service account, and the caveats. General migration scope is in the migration guide.
Installing
Download the package that matches your platform from the release assets, then verify its checksum before installing:
This example selects the x86_64 RPM. For ARM64 use .aarch64.rpm; on Debian/Ubuntu select the .deb file and matching checksum from the download page.
If you use the Pigsty package repository, dnf install silo / apt install silo resolves the same artifacts (the repository may lag GitHub Releases). The package intentionally provides no minio alias or Provides: relationship — minio and silo are separate packages that coexist, and the takeover happens at the systemd level, not through package replacement (see Takeover).
File layout
| MinIO installation | Silo package |
|---|---|
/usr/bin/minio |
/usr/bin/silo (also provides silo healthcheck) |
minio.service |
/usr/lib/systemd/system/silo.service |
/etc/default/minio |
Still read, first; /etc/default/silo overrides per variable (noreplace/conffile — upgrades never overwrite edits) |
service account minio-user (upstream) / minio (Pigsty) |
silo:silo, declared in /usr/lib/sysusers.d/silo.conf, created on install |
| — | /usr/share/doc/silo/LICENSE, NOTICE (AGPL-3.0-or-later) |
Two package properties:
- Installation never starts or enables the service;
postinstallonly creates thesiloaccount and reloads systemd. - The package installs alongside the
miniopackage — no file conflicts, so the old package stays available for rollback.
Service account
The unit defaults to User=silo, but existing data, TLS keys, and KMS credentials belong to the old MinIO user. Do not chown the data. Run Silo as the current owner via a drop-in:
This also keeps TLS working: Silo resolves certificates from the runtime user’s home (~/.silo/certs, falling back to the legacy ~/.minio/certs), so the existing public.crt/private.key/CAs/ are found without copying. Without the drop-in, a TLS deployment fails to start:
Adopting the silo account is an optional later change: move the certificates to a silo-readable path, set --certs-dir in MINIO_OPTS, and transfer data ownership outside the migration window.
Takeover and rollback
The unit is a takeover unit:
Conflicts=minio.service: systemd never runs both units; starting one stops the other.- The
EnvironmentFilechain meansMINIO_VOLUMES,MINIO_OPTS, credentials, and KMS settings from/etc/default/minioapply unchanged. Type=notify:systemctl startreturns success only after the server is actually ready.
Switch over:
Before rolling back, review the state and IAM requirements in rollback scope. To switch systemd units on a node:
After validation completes and the rollback window closes, optionally mask the old unit so nothing but an explicit systemctl unmask can bring it back:
Caveats
- Coordinate the switch across all cluster nodes. Prepare every node first (install packages and create drop-ins), stop all old processes, confirm they have exited, then start all new processes. Avoid old and new versions accessing storage together. See migration notes for mixed versions and binary consistency; for shared IAM or site replication, coordinate every participant under the IAM upgrade procedure. Rollback must also meet the applicable state-recovery requirements.
- Non-packaged installations work the same way. A
/usr/local/bin/miniowith a custom unit is taken over identically, as long as its configuration lives in/etc/default/minio. - Crash loops rate-limit. A misconfigured start (for example, missing certificates) repeats under
Restart=alwaysuntil systemd’s start limit trips (Start request repeated too quickly). Fix the cause, thensystemctl reset-failed silo && systemctl start silo. - An old
minio.servicestop can hang. Legacy units commonly setTimeoutSec=infinity. If graceful shutdown remains stuck after traffic is drained and the shutdown allowance expires, an operator can force it withsudo systemctl kill --signal=SIGKILL minio.service, then confirm the old process has exited before starting Silo. This interrupts any remaining requests; plainsystemctl killdefaults to anotherSIGTERMand does not resolve a process that ignores it. - The environment chain can surprise you during the bridge period.
/etc/default/miniois still read while/etc/default/siloexists: deleting a variable from/etc/default/silodoes not disable it — the old value from/etc/default/minioapplies again. Remove the variable from both files, or comment it out in the file that still carries it. - Keep the rollback window. Leave the
miniopackage, unit, and binary installed until validation completes; a disabled unit costs nothing. Remove the old package afterwards if desired. - Rolling restarts after migration: gate each with
silo healthcheck --maintenance cluster; exit0means stopping this node keeps write quorum, HTTP412means it does not.
8 - Console Compatibility Notes
Latest published release: Console v2.7.0 (2026-11-09, UTC+8) adds ten UI languages, Arabic RTL, container CA/temporary-directory fixes, usage-worker TLS and scheduling fixes, folder sizes over plain HTTP, silo-pkg v3.16.0, official mcli 20261109, and updated Swagger documentation. Console v2.6.0 added usage scans, retained-version totals, and bucket-administration fixes. Upgrading standalone Console does not replace Server’s embedded Console; built-in policy Reset and live bucket inventory reads require the Server 20261026 release (absent from published Server 20261018). See the component matrix.
SILO Console is Silo’s build of the MinIO Console. This page records where the two are interchangeable and where they differ. Start with the three-level overview, then use the details below to check UI and automation behavior.
pgsty/silo-console continues the upstream minio/console history from its final commit, feff71e4 (2026-04-16); the rebrand begins at 50797deb (2026-08-04). The upstream repository is no longer published — github.com/minio/console now returns 404, where minio/mc was merely archived — so the source lineage survives only in this fork. The Go module path still resolves, because the module proxy continues to serve the versions it already cached. Earlier release records: v2.0.0, v2.1.0, v2.1.1, v2.2.0, v2.2.1, v2.3.0, v2.4.0, v2.4.1, v2.4.2, and v2.5.1.
2.6.0 usage scans and bucket administration
Console 2.6.0 adds cancellable folder and bucket usage scans with separate current and retained-version totals, an optional PostgreSQL-backed durable usage worker, strict integer range validation for retention, quota, and replication bandwidth inputs, Object Lock versioning-exclusion guards during bucket creation, inert object-detail/version controls during route transitions (#81), and a three-minute keep-alive default for embedded Console servers.
2.5.1 administration repairs
Status-only group updates now preserve existing members, while explicit member arrays still replace the member list. OIDC string-array group claims preserve mapped permissions, and secure session cookies are set only when HTTPS or a trusted proxy is configured. Credential exports produce valid UTF-8 JSON, and streaming ZIP downloads preserve the underlying S3 error status if they fail before writing the first byte. Keyboard focus, accessible names, and tab scrolling are also fixed. See the complete 2.5 release notes.
Principles
The fork prioritizes existing integration contracts and explicitly documents authorization, response and interface changes.
- Renamed — the artifact on disk (
silo-console), the product identity in the interface and in--version, the distribution channels, and the signing keys. - Retained core contracts — the Go module path
github.com/minio/console, existingCONSOLE_*environment variables (includingCONSOLE_MINIO_SERVERandCONSOLE_MINIO_REGION), and packaging identifiersminio-console.service,console-userand/etc/default/console. REST APIs retain the existing architecture; see response and operation changes below. - Severed — automatic self-update, telemetry, analytics, beacons, external scripts and fonts, and call-home. A release catalog is contacted only when one is explicitly configured, through
SILO_RELEASE_SERVICE_HOSTwithRELEASE_SERVICE_HOSTretained as a fallback. - Preserved — upstream copyright and the AGPL-3.0 license. Runtime output credits both MinIO, Inc. and PGSTY.
SILO Console is not a generic S3 browser. Its administrative features need the MinIO-compatible administration APIs that Silo implements in addition to the S3 API.
What changed
1. The full administration console is retained
This is the largest functional difference, and it runs opposite to the usual direction of a fork. Upstream reduced its community console to an object browser. SILO Console keeps the complete administrative interface: dashboards, health, logs, diagnostics, and speed tests; bucket, object, lifecycle, replication, notification, and tier management; users, groups, service accounts, policies, identity providers, and KMS setup; and server configuration.
2. The dashboard targets Metrics V3
Dashboard widgets query the MinIO Metrics V3 catalog, the metric set current deployments actually scrape, with guards for its zero-value and per-node export semantics so a panel distinguishes a real zero from missing data. The mapping is recorded in docs/metrics-v3.md.
3. A smaller, quieter payload
The embedded frontend went from roughly 10 MB to under 3 MB, rebuilt reproducibly byte for byte and enforced by a release gate. There is no telemetry of any kind, and no external network dependency in the page itself.
4. Interface languages
Released Console 2.6.0 offers English and Chinese. The 2.7.0 source line adds French, German, Russian, Spanish, Brazilian Portuguese, Japanese, Arabic and Turkish through a shared menu, with bundled dictionaries and Arabic RTL. English remains the default and fallback. Documentation stays bilingual; server messages and log contents remain intact. See the language design for scope and maintenance tiers.
5. Object listings operate one page at a time
See B02. Console v2.4.0 introduced cursor pagination, retained in v2.5.1. Pages default to 100 entries, with 50, 100, 250, 500 and 1,000 available. Navigation offers first, previous and next; there is no load-all mode or arbitrary page jump.
Sorting, name filtering and select-all apply to the current page. They cover the whole directory only when the first page already contains it. Paging retains filter text and clears selection; changing page size returns to the first page, and changing directories clears the filter. A failed page leaves the previous page available for retry.
Rewind and deleted-object listings have no cursor. They return at most 1,000 versions within a time budget and disclose incomplete results when either limit is reached. See the object-browser contract.
6. API responses and automation
See B04 and O05. Session responses add accountAccessKey, and object listings explicitly return size: 0 for empty objects. Session capabilities also reflect effective permissions. Custom parsers should tolerate added fields, and Console automation should handle pagination, expired sessions and WebSocket errors.
User enable/disable gains a separate PUT /api/v1/user/{name}/status; the old PUT /api/v1/user/{name} remains but is deprecated. Multi-object downloads retain the existing endpoint and also accept a bounded form request for native streaming ZIP downloads. See password-permission migration and the v2.4.1 sharing changes; use the current API definition for custom integrations.
Console v2.4.2 preserves wrapped S3 errors from bucket-versioning operations and returns InvalidBucketState explicitly when Object Lock prevents suspension. This makes the existing Server restriction visible to the caller.
7. For developers: the module graph
Console v2.5.1 requires Go 1.27.3, directly selects github.com/pgsty/silo-pkg/v3 v3.15.2 and
upstream SDK v7.4.2-0.20261025165926-320af050c445, and retains the historical
github.com/minio/console module path. Embedders explicitly select these released sources:
Go does not inherit dependency replacements. Server must select both PGSTY
Console and MC. go-systemd v22.6.0 preserves NetBSD compatibility and tablewriter
v0.0.5 preserves the MC API. Legacy transitive minio/pkg from colorjson remains
separate from the maintained silo-pkg policy implementation. The old
minio/pkg => silo-pkg and minio-go => silo-go replacements are unsupported.
See the component matrix and
embedding guide.
The release-gating target is the coordinated SILO, Console, mcli and pkg stack. Upstream MinIO/MC probes remain non-blocking compatibility signals and do not require pkg downgrades or duplicate APIs.
Migration
The official image is docker.io/pgsty/silo-console. Linux amd64/arm64 images are provided. See release details in the maintenance announcement. Published GitHub binaries and packages are also available.
The image runs as UID 1000 with no home directory, so its default certificate directory is /.console/certs: mount private CA certificates into /.console/certs/CAs/, or choose another path with --certs-dir. Images up to v2.6.0 also ship without the system CA bundle; to reach HTTPS endpoints with public certificates (SILO, OIDC providers) from them, mount the host bundle read-only at /etc/ssl/certs/ca-certificates.crt.
An existing MinIO Console deployment upgrades in place. The service unit, service account, and configuration file keep their names, and every CONSOLE_* variable is read unchanged, so the usual path is to install the silo-console package over the old one and restart.
Two behaviors change on first start and are worth expecting:
silo-consolewill not update itself. Roll out new versions through packages, images, or your orchestrator.- Any workflow that relied on the console reaching MinIO-operated services — the update feed, licensing, or telemetry — no longer has anything to reach.
When upgrading from a version earlier than v2.4.1, review the password-policy migration.
Linux packages use /etc/silo-console/certs; migrate existing certificates or
retain their old path in CONSOLE_OPTS in /etc/default/console before restarting.
The service and configuration names are retained. Shared downloads require no new
setting. See the v2.4.1 release notes.
See also
- Silo server compatibility — the server this console administers
- MCLI client compatibility — the command-line client
- Console release notes and
CHANGELOG.md
9 - Feature Notes
The component pages in this section record where Silo matches MinIO. This subsection records where Silo extends beyond upstream: each page is a design note written before implementation and kept afterwards as the record of what was decided and why. Each page notes its implementation and release status at the top.
9.1 - Native Health Checks and the Distroless Image
Status: P1 (silo healthcheck subcommand, 2ff594f4b), P2 (distroless image + CI gate, 4c34d2309, with UBI CA bundle parity in #295), and P4 (healthcheck probe guide) are shipped; P3 (Helm probes) remains pending · Decided: 2026-08-06 · Owner: pgsty/silo (command, images, Helm chart), this site (docs)
Silo is getting a native silo healthcheck subcommand and, alongside the existing container image, a new distroless image variant that contains exactly one file that matters: the silo binary. This note records the reasoning and the design decisions before implementation, so the code has a specification to be checked against — and so that “why is it built this way?” has a permanent answer.
Background
Today’s release image (docker.io/pgsty/silo) is built on ubi-micro and ships four moving parts: the silo server, the mcli client (with an mc alias), a statically linked curl, and a POSIX-shell entrypoint script. The compose examples check container health with the bundled client:
That arrangement is inherited from upstream MinIO, and its fragility is a matter of record: when mc was briefly missing from the image, users found their health checks failing “with no option but to disable” (#9). Upstream’s own history rhymes — when MinIO moved to ubi-micro in 2023 and lost curl, the maintainers’ answer was to lean harder on mc ready local (minio/minio#18373, #18389), and upstream minio/minio has since been archived with server as the only subcommand its binary ever had. Nobody upstream is going to fix this.
A distroless image forces the question. There is no shell, no curl, no mc — by design. The only program guaranteed to exist inside the container is the server binary itself. If Docker-level health checking is to exist at all in that image, the binary has to provide it.
Why mc ready had to be retired as a probe
Reading the actual mc implementation (cmd/ready-main.go) shows the current health check works by accident, not by design. Four independent defects:
- It never fails on its own.
mc readyis a wait-until-ready loop: it retries every 5 seconds forever and only ever exits zero, on success. Connection refused does not break the loop. As a Docker healthcheck, the “unhealthy” verdict is produced entirely by Docker’stimeoutkilling the process — the probe semantics are a side effect of SIGKILL. - It checks the wrong scope.
mc readyrequests/minio/health/cluster— cluster-wide write quorum. Every container’s “health” therefore reflects the state of the whole cluster, which is precisely the cascading-failure anti-pattern the Kubernetes documentation warns about: lose quorum, and every node is marked unhealthy simultaneously. - It has hidden failure modes. It requires a writable
~/.mcconfig directory (on a read-only rootfs or an arbitrary OpenShift UID, the probe fails while the server is perfectly healthy), it prints config-creation noise on first run, and its built-inlocalalias is hardcoded tohttp://localhost:9000— wrong the moment TLS is enabled or the port changes, a limitation users complained about upstream. - It is the last functional reason to bundle a second binary. Both
mcliand the pinned staticcurlcarry ongoing supply-chain and maintenance cost (the curl pin is stuck on v8.11.0 because a later release dropped the aarch64 build) for what a subcommand of the existing binary can do in ~150 lines.
The decisions
Three tracks, deliberately decoupled:
| # | Decision |
|---|---|
| D1 | The silo binary gains a healthcheck subcommand — a thin, anonymous HTTP client for the server’s existing /minio/health/* endpoints. It ships in every build, so every image and bare-metal install gains the capability. |
| D2 | The existing image does not change. It keeps mcli, curl, the shell entrypoint, and the mc ready local examples. Users of the current image who want the new probe can opt in by overriding their healthcheck.test — nothing is taken away and no default behavior moves. |
| D3 | A new distroless variant is published alongside it, as a pilot: single binary, no shell, native HEALTHCHECK baked in. If the pilot proves out, it becomes the recommended default later and the switch completes; the classic image remains for compatibility either way. |
D2 and D3 answer the obvious “why not just slim the main image?” — because the main image’s contents are a compatibility surface. #9 exists because that surface was changed underneath people once before. The distroless image is a new name with a new contract, so nobody’s existing healthcheck, docker exec mc habit, or entrypoint assumption breaks while it is evaluated.
The silo healthcheck command
The design principles behind that shape:
- Thin client, single source of truth. The command is only ever an HTTP client of the canonical health API. It never re-implements a check in-process, so the semantics of “healthy” live in exactly one place: the server handlers.
- CLI vocabulary = API vocabulary. The check names are the endpoint paths. No new concepts to learn, nothing to keep in sync.
- Share the server’s own configuration. The port comes from the same
--address/MINIO_ADDRESScontract the server uses, and http-vs-https is decided by the same certificate check the server itself performs at startup (public.crt+private.keyin the certs directory). This is the Traefikhealthcheckpattern — the closest prior art, which resolves its ping endpoint from the same static configuration as its server — with the TLS handling Traefik left as a// TODOactually implemented. It is also the direct fix formc ready’s port-guessing defect. - The default check is node-local.
liveanswers “is this process serving,” which is the only question a per-container health status should answer. Cluster-scope checks exist, but only behind explicit arguments, mirroringmc ready’s--cluster-read/--maintenanceso the operational vocabulary carries over. - Exit codes are 0 and 1, nothing else. The Dockerfile reference explicitly reserves exit code 2 (
vault status, which uses 2 for “sealed”, is the cautionary tale). Rich diagnostics belong in the single output line instead — Docker stores the first 4096 bytes of probe output indocker inspect, and the command decodes the server’s diagnostic headers (x-minio-server-status,x-minio-write-quorum,x-minio-healing-drives) into it, which is exactly the detail a barecurl -fthrows away. - Skip TLS certificate verification, with no opt-out in v1. This is a loopback self-probe of an anonymous endpoint carrying no data — and the kubelet’s documented behavior for HTTPS
httpGetprobes is precisely the same. Matching it means one TLS deployment produces one verdict across Docker and Kubernetes; verifying by default would only manufacture false negatives, since self-signed server certs rarely carry a127.0.0.1SAN.
Two key implementation constraints from the server source:
- The request must be strictly anonymous. The health routes are exempted from the reserved-path guard only for requests the server classifies as anonymous; attaching an
Authorizationheader reclassifies the request and gets it rejected (ErrAllAccessDisabled) instead of answered. - The HTTP transport must set
Proxy: nil. Containers routinely inheritHTTP_PROXYwithout aNO_PROXYentry for127.0.0.1; a loopback probe must never route through a corporate proxy. (Traefik’s healthcheck does this deliberately, for the same reason.)
The 15-second default timeout for cluster checks accounts for the server’s own 10-second cluster_deadline when evaluating cluster health — a client that gives up at 5s abandons the request before the server delivers its 503 response and diagnostic headers. Two additional details complete the contract: --url is environment-backed (MINIO_HEALTHCHECK_URL) because a probe process cannot see the server’s command line — it is the documented way to point a baked-in HEALTHCHECK at a server whose address or TLS setup comes from CLI arguments; and any outer (Docker) timeout must exceed the probe’s own deadline, or the probe is SIGKILLed before it can print its diagnostic line.
What the endpoints really do
The table below is verified against the handler source, not quoted from documentation — and it corrects a common misreading:
| Endpoint | Returns 200 when… | Fails with… | Notes |
|---|---|---|---|
/minio/health/live |
almost always — even before the object layer is initialized (that state is only signaled via the x-minio-server-status: offline header) |
503 when the request queue is saturated | touches no external system; the only endpoint quiet enough for high-frequency probing |
/minio/health/ready |
as live, plus KMS can generate a key and etcd answers a read — each only if configured |
KMS/etcd failure; queue saturation | without KMS or etcd, ready and live are the same code path |
/minio/health/cluster |
object layer, bucket metadata and IAM are initialized, and every erasure set has write quorum | 503 with quorum diagnostic headers; with ?maintenance=true, failure is 412 |
each failed evaluation writes a server-side log line — do not poll it tightly |
/minio/health/cluster/read |
the read-quorum version of the above | as above |
Consequences worth spelling out: live and ready are liveness-grade signals — they do not tell you the node can serve objects; only the cluster pair does. That is exactly why the cluster pair must stay out of per-container probes (scope, log noise, cascading restarts) and why it is the right tool for operational questions like “may I take this node down?” (--maintenance, where 200 means safe and 412 means you would lose HA).
The distroless variant
Base: gcr.io/distroless/static-debian12 — which is sufficient because silo builds with CGO_ENABLED=0. The base ships tzdata, /tmp, and an /etc/passwd with root/nonroot entries, and #295 copies the UBI-derived /etc/ssl/certs/ca-certificates.crt trust bundle from the verify stage so the classic and distroless images share the exact same root CA trust store (for KMS/webhook/STS egress). It ships no shell, no package manager, no libc.
Sketch of the contract:
The decisions folded into that sketch:
ENTRYPOINTis the binary itself.docker run pgsty/silo:distroless server /data— no argv-translation script, because there is no shell to run one. The classic image’sMINIO_USERNAME/MINIO_GROUPNAMEprivilege-drop path (which needs GNUchrootand a writable/etc/passwd) is not supported in this variant; the supported mechanism is--user/ KubernetesrunAsUser./datais created in the layer, mode 0777, and the default user stays root for the pilot. Issue #55 demonstrated that declaringVOLUME ["/data"]without creating it breaks every non-root invocation, and that no entrypoint can repair it after the fact — in distroless there is no entrypoint at all. Creating it world-writable in the layer is the one option that makes all privilege modes work (--userincluded), keeps drop-in parity with the classic image, and its exposure is bounded by the image running a single process. A nonroot-by-default posture (uid 65532) was considered and deferred: it would break the documented bind-mount workflow on UID mismatch, and the pilot’s job is to measure friction, not maximize it. Revisit at promotion time, possibly as a-nonroottag.- The health check is baked in, exec-form. Shell-form
HEALTHCHECKstrings need/bin/shand are impossible here; the JSON-array form is mandatory. Compose inherits an image’sHEALTHCHECKautomatically (withdisable: trueas the escape hatch), so compose users of this variant get workingdepends_on: condition: service_healthywith zero configuration.readyrather thanlivebecause Docker’s health status feeds gating (start ordering), which is readiness semantics — and the two are identical anyway unless KMS/etcd are configured. - One release-blocking verification:
HEALTHCHECKis a Docker extension, absent from the OCI image spec (opencontainers/image-spec#749 is still open), and OCI-media-type builds drop it silently. The publish pipeline must assertdocker inspectshows theHealthconfig on the pushed manifest, or adjust the build’s media types until it does. - Naming:
docker.io/pgsty/silo:<RELEASE>-distroless, plus a rollingdistrolesstag. A newDockerfile.distrolessin the server repo — which, having no download stages, is fully offline and can therefore be built and asserted in CI on every release, closing the “the gate tests a synthetic image, not the shipped one” coverage gap that #55 documented for the classic Dockerfile. - What a user gives up in the variant, stated honestly in its docs: no
docker exec <container> shdebugging (usedocker debug/kubectl debugephemeral containers), no in-imagemc(use thepgsty/mcimage or a host-installedmcli), noMINIO_USERNAMEpath (use--user).
Kubernetes needs no image support at all
Worth stating explicitly, because it bounds the problem: Kubernetes ignores Dockerfile HEALTHCHECK entirely — kubelet probes are configured in the pod spec and executed from outside the container as httpGet requests. Both image variants are therefore probed identically:
Three cautions that belong next to any such config: the cluster endpoints must never appear in probes (liveness would restart the whole fleet on quorum loss; readiness would tangle with bootstrap — the chart’s headless service correctly sets publishNotReadyAddresses: true for exactly that reason); live deliberately returns 503 under sustained request-queue saturation, so a saturated node restarts after ~90s by design; and with scheme: HTTPS the kubelet skips certificate verification, so self-signed deployments need nothing extra.
The Silo Helm chart currently ships no probes at all (neither does upstream’s, despite its docs). Adding the three probes above to the chart is planned as an independent follow-up — it depends on neither image track, and it is a differentiator over upstream rather than a compatibility risk.
Rollout
| Phase | Scope | Repo |
|---|---|---|
| P1 | silo healthcheck subcommand + tests; ships in the next release binary (all images inherit the capability, no image behavior changes) |
pgsty/silo |
| P2 | Dockerfile.distroless + CI build/health gate + publish -distroless tags as a pilot |
pgsty/silo |
| P3 | Helm chart: add the three probes; refresh the stale default image tag | pgsty/silo |
| P4 | Docs: command reference, probe guide, distroless migration notes; pilot feedback → decide on promoting distroless to the recommended default | this site |
Compatibility promises across all phases: the classic image’s contents and examples do not change; mc ready local keeps working everywhere it works today; the health HTTP API is untouched (the subcommand is purely additive); and the /minio/health/* paths remain frozen as compatibility surface, same as every other /minio/* route in the fork.
Deferred decisions
Recorded so they are not re-litigated from scratch:
--waitmode (block-until-healthy, the one thingmc ready’s loop is genuinely for): deferred — no in-repo consumer needs it yet, and adding it later is backward-compatible. The flag namespace is reserved.- Nonroot-by-default for the distroless image: deferred to promotion time, as above.
- JSON output schema for
--json: follows the global-flag convention; the exact schema is fixed at implementation time and documented in the command reference.