Design Records
Design records document the engineering decisions behind PGSTY SILO: root causes, S3 and IAM protocol invariants, trade-offs against rejected alternatives, operational limits, and regression coverage.
November 9: Console Languages
- Adding Multilingual Support to SILO: Ten UI languages across Console 2.6.0 and Console 2.7.0 (including Portuguese from #83 and Turkish requested in silo#277), maintenance tiers, English fallback and Arabic RTL, with measured gzip and cache tradeoffs behind per-language assets and eager loading.
November 6: Found in Release Acceptance
- A Partial View Must Not Report an Existing Object as Missing: Why a node that reaches only some drives can answer 404 for an object that exists, and why “not found” must require an answer from every drive.
- A Failed Multipart Completion Can Strand the Upload: Why a completion that misses write quorum can strand the upload until stale-upload cleanup, and why the repair waits for fencing of storage operations.
November 4: Release-Blocking Lifecycle Fixes
- Multi-Pool Lifecycle Expiry: Decide on the Whole Object (#278, #279): Why an expiry decided on the versions of one pool hides, deletes, or resurrects versions in another pool, and why a recheck on the merged version list under the object lock is the minimum complete repair.
- Restored Copies: Expire the Copy, Not the Object (#281, #282): Why the expiry of a restored current version hid the object or stripped an older null version, and which storage and native-restore safeguards ship with the exact-version fix.
- Native Restore in Two Phases (#284): Why a native restore held the object lock for its whole tier download, why the first #281 safeguards also stranded failed ordinary uploads, and how staging outside the lock with a short, checked commit replaces the private upload.
- Audit Webhooks After a Peer Reload (#259): Why a runtime audit_webhook change leaves every server but the serving one with a stopped target, which logger subsystems share it, and why building logger targets on the server context plus opening the queue store first is the minimum complete repair.
- Failed Writes and Phantom Empty Prefixes (#285, #286): Why a write whose staged data vanished before the remote rename leaves an empty object directory that later lists as an empty prefix, and why removing it on both rename failure paths is sufficient.
- Heal Dry Run Must Not Delete Data (from #280): Why a dry-run heal could delete the last metadata copies of a dangling object and run a real bucket heal on remote nodes, and why an early return plus forwarding the dry-run flag to peers closes both paths.
- Lifecycle Delete-All Must Stay on One Key (found during review of #278): Why ExpiredObjectAllVersions, DelMarkerExpiration, and the default batch-expiration purge delete every other key under the expired key’s name, and why purging the key’s own versions by ID after a recheck is the minimum safe repair.
- Bucket Quota in the Console: List Display and Numeric Validation (silo-console#72, silo-console#73): Why a Console-set quota showed as “-” in the bucket list, why bucket creation could turn a huge quota into one byte, where both came from, and what the merged Console fix covers.
- Lock Domains of Large Single-Pool Clusters: Why DELETE and CopyObject fail to exclude PUT on the same key in single-pool clusters of 18 or more hosts, which layouts that covers, and the routing change that closes it.
November 1: Pool Migration Integrity Review
- Pool Migration Integrity Review: #261, #263, and #265: A maintainer review of three reports and their fixes, including the contributor merge of #266 and maintainer follow-up #276.
- Mixed EC Layouts and Migration Listing (#261, #262): Why a readable object can disappear from decommission and rebalance listings after an interrupted degraded overwrite, and why the narrow recount in #262 is sufficient.
- Tier Free Versions Are Internal Cleanup Records (#263, #264): Why a free version must stay visible to healing but must not become a user-visible delete marker or a migration version.
- ILM Expiry and Pool Migration Ordering (#265, #266, #276): Why a versioned Expiration marker must not cause the current version to be skipped, and how the maintainer follow-up preserves tier cleanup and completion semantics.
October 24: FULL_OBJECT Checksum Replication
- FULL_OBJECT Checksum Replication: Preserve the Type at Upload Initiation (
#254,#255): Why dropping the checksum type makes valid multipart CRC objects fail replication, how one shared header fix also restores the existing CopyObject path, and the source-upgrade and acceptance boundaries.
October 24: Multi-Pool Deletion and Placement
-
Multi-Pool Deletion: Preserve the Latest Copy on Failure (
#256,#257): Why an online-drive preflight cannot protect against later I/O failures, how staged cleanup preserves each item’s authoritative copy, and the retry, tier-reference, performance, and compatibility boundaries of the merged fix. -
Overwrite Pool Affinity: Why #245 Was Not Adopted (
#245): The original placement problem, request-aware quorum and health improvements, unversioned-only scope, two rounds of m3 performance measurements, and the decision to stop after small-object overwrite regressions remained. The experimental changes were not merged.
October 23: Alias Regions
- Alias regions: Persistent per-alias regions, client-cache isolation, and upgrade, downgrade and credential-rotation boundaries, reconciled with mcli 20261109.
October 23 IAM & KMS Design Notes
- LDAP Long DNs: Restoring Login Without Hiding Revocation Errors (
#250): How\xxhex escaping inNormalizeDNtriggers the 255-byte single-component filename limit on the object IAM backend, and whyloadIAMConfigBytesWithMetadataclassifies only impossible pure-ASCII path components (> 255bytes) aserrConfigNotFound. - IAM Policy Reloads: Lock Scope, Cancellation, and Time Budgets (
#251): Moving policy reload and KMS decryption outside the authentication-cache lock while retainingwriteMu, threading request cancellation end-to-end toKES.Decrypt, and keepingetcd30-second RPC deadlines separate from total decryption budgets.
Server 20261018 Key Design Notes
For the Server 20261018 release, see these core architecture notes:
- Object Lock Governance & Overwrite Protection (
SN-2026-015,SN-2026-016) - IAM Admin Credential Authorization (
SN-2026-017) - Durable IAM & Scoped STS Revocations (
SN-2026-013) - LIST Recovery Lock Cost
- ListMultipartUploads Rollback & Capacity Boundaries
- Multi-Pool Consistency & Rebalance Protection
- KES & OpenBao Integration
-
Adding Multilingual Support to SILO
Console PR #83, contributed by Eduardo Andrade Machado, supplies Brazilian Portuguese. silo#277, opened by siyahkut, requests Turkish. These contributions led to a configurable ten-language UI. Status as of 2026-11-09: the ten-language UI, reviewed …
Console PR #83, contributed by Eduardo Andrade Machado, supplies Brazilian Portuguese. silo#277, opened by siyahkut, requests Turkish. These contributions led to a configurable ten-language UI. Status as of 2026-11-09: the ten-language UI, reviewed …
-
Native Restore in Two Phases
This record covers issue #284 and two regressions of the first native-restore safeguards for #281, described in Restored Copies: Expire the Copy, Not the Object. It was evaluated against main at 69fcf50cd and the integration branch for the next …
This record covers issue #284 and two regressions of the first native-restore safeguards for #281, described in Restored Copies: Expire the Copy, Not the Object. It was evaluated against main at 69fcf50cd and the integration branch for the next …
-
A Partial View Must Not Report an Existing Object as Missing
This record covers a defect found during local acceptance of the next release. It reproduces on the local integration of the next release and on 69fcf50cd. It has no GitHub issue yet. Status: fix approved by an adversarial review (with nits, all …
This record covers a defect found during local acceptance of the next release. It reproduces on the local integration of the next release and on 69fcf50cd. It has no GitHub issue yet. Status: fix approved by an adversarial review (with nits, all …
-
A Failed Multipart Completion Can Strand the Upload
This record covers a defect found during local acceptance of the next release. It reproduces on 69fcf50cd and is inherited from upstream MinIO. It has no GitHub issue yet. Status: analyzed and reproduced. A fix was drafted, but an adversarial review …
This record covers a defect found during local acceptance of the next release. It reproduces on 69fcf50cd and is inherited from upstream MinIO. It has no GitHub issue yet. Status: analyzed and reproduced. A fix was drafted, but an adversarial review …
-
Lock Domains of Large Single-Pool Clusters
This record covers a lock-domain defect found during review of the restored-copy fix (Restored Copies: Expire the Copy, Not the Object), evaluated against main at 69fcf50cd. No issue has been filed yet. Status: fix approved by review and committed …
This record covers a lock-domain defect found during review of the restored-copy fix (Restored Copies: Expire the Copy, Not the Object), evaluated against main at 69fcf50cd. No issue has been filed yet. Status: fix approved by review and committed …
-
Restored Copies: Expire the Copy, Not the Object
This record covers issue #281 and PR #282 by Denis Babchuk, evaluated against main at 69fcf50cd. The fix ships in the same maintainer integration PR as the multi-pool lifecycle fix (Multi-Pool Lifecycle Expiry, #278). That PR keeps the contributor …
This record covers issue #281 and PR #282 by Denis Babchuk, evaluated against main at 69fcf50cd. The fix ships in the same maintainer integration PR as the multi-pool lifecycle fix (Multi-Pool Lifecycle Expiry, #278). That PR keeps the contributor …
-
Failed Writes and Phantom Empty Prefixes
This record covers issue #285 and PR #286 by Denis Babchuk, evaluated against main at 69fcf50cd and the PR head ed05d982a. Status: fix approved by review and committed locally. Tested together with the other fixes for the next release on one …
This record covers issue #285 and PR #286 by Denis Babchuk, evaluated against main at 69fcf50cd and the PR head ed05d982a. Status: fix approved by review and committed locally. Tested together with the other fixes for the next release on one …
-
Multi-Pool Lifecycle Expiry: Decide on the Whole Object
This record covers issue #278 and PR #279 by Denis Babchuk, evaluated against main at 69fcf50cd. The fix will ship through a maintainer integration PR that keeps the contributor commits unchanged and adds the maintainer follow-ups described below. …
This record covers issue #278 and PR #279 by Denis Babchuk, evaluated against main at 69fcf50cd. The fix will ship through a maintainer integration PR that keeps the contributor commits unchanged and adds the maintainer follow-ups described below. …
-
Heal Dry Run Must Not Delete Data
This record covers the dry-run part of PR #280 by Jasper Bernhardt (@dihmandrake), evaluated against main at 69fcf50cd. The same PR also verifies an object’s drives concurrently during deep scans. That optimization is reviewed separately and is …
This record covers the dry-run part of PR #280 by Jasper Bernhardt (@dihmandrake), evaluated against main at 69fcf50cd. The same PR also verifies an object’s drives concurrently during deep scans. That optimization is reviewed separately and is …
-
Lifecycle Delete-All Must Stay on One Key
This record covers a defect found during review of the #278 fix (Multi-Pool Lifecycle Expiry). It was evaluated against main at 69fcf50cd and against the maintainer integration branch for #278 and #281, which is not yet published. Status: fix …
This record covers a defect found during review of the #278 fix (Multi-Pool Lifecycle Expiry). It was evaluated against main at 69fcf50cd and against the maintainer integration branch for #278 and #281, which is not yet published. Status: fix …
-
Bucket Quota in the Console: List Display and Numeric Validation
This record covers silo-console#72 (a quota set in the web Console shows as a dash in the bucket list) and silo-console#73 (numeric validation in the bucket-creation form). It is evaluated against the Console source before the fix, 3d264aea9, and the …
This record covers silo-console#72 (a quota set in the web Console shows as a dash in the bucket list) and silo-console#73 (numeric validation in the bucket-creation form). It is evaluated against the Console source before the fix, 3d264aea9, and the …
-
Audit Webhooks After a Peer Reload
This record covers issue #259, reported by Henrik Feldt, evaluated against main at 69fcf50cd. The fix completes earlier, uncommitted maintainer work. It adds the Kafka call site, a handoff of records queued in a target that never started, and Kafka …
This record covers issue #259, reported by Henrik Feldt, evaluated against main at 69fcf50cd. The fix completes earlier, uncommitted maintainer work. It adds the Kafka call site, a handoff of records queued in a target that never started, and Kafka …
-
ILM Expiry and Pool Migration Ordering
This record covers issue #265, the contributor change PR #266, and maintainer follow-up #276. The merged #264 and #267 are relevant predecessors. The review was updated against main after #276 on 2026-11-02. #266’s decommission commit is 19910230a, …
This record covers issue #265, the contributor change PR #266, and maintainer follow-up #276. The merged #264 and #267 are relevant predecessors. The review was updated against main after #276 on 2026-11-02. #266’s decommission commit is 19910230a, …
-
Tier Free Versions Are Internal Cleanup Records
This record covers issue #263 and PR #264, updated against main at 0fc5a0b5, with the #264 merge recorded at 0cfca29c. The PR is three linked repairs, not three unrelated workarounds. Short answer The report is real. A free version is an internal …
This record covers issue #263 and PR #264, updated against main at 0fc5a0b5, with the #264 merge recorded at 0cfca29c. The PR is three linked repairs, not three unrelated workarounds. Short answer The report is real. A free version is an internal …
-
Mixed EC Layouts and Migration Listing
This record covers issue #261 and PR #262, updated against main at 0fc5a0b5 after the #276 follow-up. The fix was merged through maintainer PR #267; #262 was the superseded duplicate. Short answer The report is real. A successful degraded overwrite …
This record covers issue #261 and PR #262, updated against main at 0fc5a0b5 after the #276 follow-up. The fix was merged through maintainer PR #267; #262 was the superseded duplicate. Short answer The report is real. A successful degraded overwrite …
-
Pool Migration Integrity Review: #261, #263, and #265
This record reviews #261, #263, and #265, together with their fixes #262, #264, and #266. It also records the maintainer follow-up #276. It is a design and review record, not a release note. The review was updated against the post-#276 main branch on …
This record reviews #261, #263, and #265, together with their fixes #262, #264, and #266. It also records the maintainer follow-up #276. It is a design and review record, not a release note. The review was updated against the post-#276 main branch on …
-
Multi-Pool Deletion: Preserve the Latest Copy on Failure
Decision, October 24, 2026: merge the corrected PR #257 (shipped in Server 20261026). Commit aafe4f8886 fixes the multi-pool deletion failures reported in issue #256 and closes the in-flight failure window after preflight checks. The core invariant …
Decision, October 24, 2026: merge the corrected PR #257 (shipped in Server 20261026). Commit aafe4f8886 fixes the multi-pool deletion failures reported in issue #256 and closes the in-flight failure window after preflight checks. The core invariant …
-
Overwrite Pool Affinity: Why #245 Was Not Adopted
Decision, October 24, 2026: do not merge the tested pool-affinity changes (PR #245). The experiment addressed a real space-efficiency problem, but neither the broad policy nor the narrower unversioned-only policy achieved an acceptable performance …
Decision, October 24, 2026: do not merge the tested pool-affinity changes (PR #245). The experiment addressed a real space-efficiency problem, but neither the broad policy nor the narrower unversioned-only policy achieved an acceptable performance …
-
FULL_OBJECT Checksum Replication: Preserve the Type at Upload Initiation
Issue #254 reported a replication failure affecting multipart objects uploaded with a FULL_OBJECT CRC32 or CRC32C checksum: the object uploads and reads normally on the source, but the replication target rejects CompleteMultipartUpload with 400 …
Issue #254 reported a replication failure affecting multipart objects uploaded with a FULL_OBJECT CRC32 or CRC32C checksum: the object uploads and reads normally on the source, but the replication target rejects CompleteMultipartUpload with 400 …
-
LDAP Long DNs: Restoring Login Without Hiding Revocation Errors
Server PR #250 (merged at d43adc0f2 and shipped in Server 20261026) fixes a login failure affecting LDAP users whose normalized Distinguished Name (DN) exceeds the 255-byte single-component filename limit of the object-storage IAM backend, while …
Server PR #250 (merged at d43adc0f2 and shipped in Server 20261026) fixes a login failure affecting LDAP users whose normalized Distinguished Name (DN) exceeds the 255-byte single-component filename limit of the object-storage IAM backend, while …