[codex] Harden PackageNormalize ZIP handling

DEVELOPER_GUIDE.md

@@ -116,6 +116,8 @@ bash tools/build-install-sh.sh
 | Reproducible OPC metadata (#2756) | NuGet's OPC package writer generates a random `package/services/metadata/core-properties/*.psmdcp` part name on each pack run. The normalizer rewrites that part to `package/services/metadata/core-properties/core-properties.psmdcp`, updates the matching content-type and relationship references, and gives ZIP entries stable timestamps. This is the package reproducibility boundary for `.nupkg` and `.snupkg` archives. |
 | Work bounds (#2892) | Before rewriting, the normalizer rejects packages with more than 4096 ZIP entries, any entry above 128 MiB uncompressed, total uncompressed content above 512 MiB, or XML reference text above 16 MiB so crafted packages cannot force unbounded normalization work. |
 | Unsafe ZIP names (#2894) | Before creating the destination archive, the normalizer rejects absolute paths, Windows drive roots, backslash separators, empty path segments, parent-directory segments, empty normalized names, and destination names that collide after path normalization. Those entries are not preserved into normalized packages. |
+| Unsafe ZIP attributes (#3552) | Before copying entries, the normalizer rejects POSIX symlink/device/special-file types and unsafe DOS attributes, then writes normalized entries with scrubbed deterministic external attributes instead of preserving source permission bits. |
+| Failure diagnostics (#3458) | The CLI accepts at most 1024 package paths per run, reports bounded package path and ZIP entry diagnostics instead of raw path-heavy exception text, and emits cleanup deletion failures as per-package `warnings` in JSON output. |
 
 When you intentionally update a dependency (or add a new direct `PackageReference`), regenerate the lock files locally and commit the diff in the same change:
 
@@ -2299,6 +2301,8 @@ bash tools/build-install-sh.sh
 | 再現可能な OPC metadata (#2756) | NuGet の OPC package writer は `package/services/metadata/core-properties/*.psmdcp` part 名を pack ごとにランダム生成します。normalizer はその part を `package/services/metadata/core-properties/core-properties.psmdcp` に書き換え、対応する content-type / relationship 参照も更新し、ZIP entry timestamp を固定します。これが `.nupkg` / `.snupkg` archive の package 再現性境界です。 |
 | 作業量の上限 (#2892) | 書き換え前に、normalizer は 4096 を超える ZIP entry、128 MiB を超える uncompressed entry、512 MiB を超える合計 uncompressed content、または 16 MiB を超える XML 参照テキストを持つ package を拒否し、細工された package が無制限の normalize 作業を強制できないようにします。 |
 | unsafe ZIP name (#2894) | destination archive を作る前に、normalizer は absolute path、Windows drive root、backslash separator、空の path segment、parent-directory segment、空に正規化される名前、path 正規化後に衝突する destination 名を拒否します。これらの entry は normalized package に保持されません。 |
+| unsafe ZIP attributes (#3552) | entry のコピー前に、normalizer は POSIX symlink / device / special-file type と unsafe DOS 属性を拒否し、source の permission bit を保持せず deterministic に scrub した external attributes で normalized entry を書き込みます。 |
+| failure diagnostics (#3458) | CLI は 1 回の実行で受け付ける package path を最大 1024 件に制限し、raw な path-heavy exception text ではなく bounded な package path / ZIP entry diagnostics を報告し、cleanup 削除失敗を JSON 出力の package ごとの `warnings` として出します。 |
 
 依存を意図的に更新する（あるいは直接 `PackageReference` を追加する）場合は、ローカルで lock ファイルを再生成し、同じ変更でコミットしてください:
 

changelog.d/unreleased/3458.fixed.md

@@ -0,0 +1,17 @@
+---
+category: fixed
+issues:
+  - 3458
+affected:
+  - DEVELOPER_GUIDE.md
+  - tools/CodeIndex.PackageNormalize/PackageNormalizeCli.cs
+  - tests/CodeIndex.Tests/ReleaseWorkflowTests.cs
+---
+
+## English
+
+- **PackageNormalize now reports bounded friendly failures and cleanup warnings (#3458)** — package normalization errors now avoid raw path-heavy exception text, cap package arguments per run, bound ZIP entry diagnostics, and include structured cleanup warnings in JSON output.
+
+## 日本語
+
+- **PackageNormalize が bounded で friendly な失敗内容と cleanup warning を報告するようになりました (#3458)** — package normalization error は raw な path-heavy exception text を避け、1 回の実行で受け付ける package 引数数を制限し、ZIP entry diagnostics を bounded にし、JSON 出力に構造化された cleanup warning を含めます。

changelog.d/unreleased/3552.fixed.md

@@ -0,0 +1,17 @@
+---
+category: fixed
+issues:
+  - 3552
+affected:
+  - DEVELOPER_GUIDE.md
+  - tools/CodeIndex.PackageNormalize/PackageNormalizeCli.cs
+  - tests/CodeIndex.Tests/ReleaseWorkflowTests.cs
+---
+
+## English
+
+- **PackageNormalize no longer preserves unsafe ZIP external attributes (#3552)** — normalized packages now scrub entry external attributes to a deterministic safe value after rejecting POSIX special-file types and unsafe DOS attributes.
+
+## 日本語
+
+- **PackageNormalize が安全でない ZIP external attributes を保持しないようになりました (#3552)** — 正規化後の package は、POSIX special-file type や unsafe DOS 属性を拒否したうえで、entry external attributes を deterministic な安全値へ scrub します。

tests/CodeIndex.Tests/ReleaseWorkflowTests.cs

@@ -294,6 +294,110 @@ public void PackageNormalizeCli_JsonContinueOnErrorReportsAggregateSummary()
         }
     }
 
+    [Fact]
+    public void PackageNormalizeCli_RejectsTooManyPackageArguments()
+    {
+        var args = Enumerable
+            .Range(0, PackageNormalizeOptions.MaxPackageArgumentCount + 1)
+            .Select(index => $"package-{index}.nupkg")
+            .ToArray();
+
+        var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() => PackageNormalizeCli.Run(args));
+
+        Assert.Equal(1, exitCode);
+        Assert.Empty(stdout);
+        Assert.Contains($"at most {PackageNormalizeOptions.MaxPackageArgumentCount} package paths", stderr);
+    }
+
+    [Fact]
+    public void PackageNormalizeCli_JsonReportsBoundedFriendlyFailure()
+    {
+        var projectRoot = TestProjectHelper.CreateTempProject(nameof(PackageNormalizeCli_JsonReportsBoundedFriendlyFailure));
+        try
+        {
+            var missingPackagePath = Path.Combine(projectRoot, "missing.nupkg");
+
+            var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() =>
+                PackageNormalizeCli.Run(["--json", missingPackagePath]));
+
+            Assert.Equal(1, exitCode);
+            Assert.Empty(stderr);
+            using var doc = JsonDocument.Parse(stdout);
+            var package = doc.RootElement.GetProperty("packages").EnumerateArray().Single();
+            var error = package.GetProperty("error").GetString();
+            Assert.Contains("missing.nupkg", error);
+            Assert.DoesNotContain(projectRoot, error);
+            Assert.True(error!.Length <= 512);
+            Assert.Empty(package.GetProperty("warnings").EnumerateArray());
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(projectRoot);
+        }
+    }
+
+    [Fact]
+    public void PackageNormalizeCli_JsonBoundsZipEntryDiagnostics()
+    {
+        var projectRoot = TestProjectHelper.CreateTempProject(nameof(PackageNormalizeCli_JsonBoundsZipEntryDiagnostics));
+        try
+        {
+            var packagePath = Path.Combine(projectRoot, "unsafe-entry.nupkg");
+            var longEntryName = new string('a', 260) + "\\payload.txt";
+            CreatePackageWithEntries(
+                packagePath,
+                ("package/services/metadata/core-properties/random.psmdcp", ""),
+                (longEntryName, "payload"));
+
+            var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() =>
+                PackageNormalizeCli.Run(["--json", packagePath]));
+
+            Assert.Equal(1, exitCode);
+            Assert.Empty(stderr);
+            using var doc = JsonDocument.Parse(stdout);
+            var error = doc.RootElement.GetProperty("packages").EnumerateArray().Single().GetProperty("error").GetString();
+            Assert.Contains("aaa", error);
+            Assert.Contains("...", error);
+            Assert.DoesNotContain(longEntryName, error);
+            Assert.True(error!.Length <= 512);
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(projectRoot);
+        }
+    }
+
+    [Fact]
+    public void PackageNormalizer_ReportsCleanupWarningsWhenTempDeleteFails()
+    {
+        var projectRoot = TestProjectHelper.CreateTempProject(nameof(PackageNormalizer_ReportsCleanupWarningsWhenTempDeleteFails));
+        try
+        {
+            var packagePath = Path.Combine(projectRoot, "cleanup-warning.nupkg");
+            CreateMinimalNuGetPackage(packagePath, "random.psmdcp");
+            var limits = PackageNormalizeLimits.Default with { MaxXmlTextChars = 5 };
+            var warnings = new List<string>();
+
+            var exception = Assert.Throws<InvalidOperationException>(() =>
+                PackageCorePropertiesNormalizer.NormalizePackage(
+                    packagePath,
+                    limits,
+                    warnings,
+                    _ => throw new IOException("delete failed at /private/path")));
+
+            Assert.Contains("[Content_Types].xml", exception.Message);
+            var warning = Assert.Single(warnings);
+            Assert.Contains("Could not delete temporary normalized package", warning);
+            Assert.Contains("cleanup-warning.nupkg.normalize-tmp", warning);
+            Assert.DoesNotContain(projectRoot, warning);
+            Assert.DoesNotContain("/private/path", warning);
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(projectRoot);
+        }
+    }
+
     [Fact]
     public void PackageNormalizer_RejectsPackageThatExceedsEntryCountLimit()
     {
@@ -452,7 +556,76 @@ public void PackageNormalizer_RejectsDestinationNamesThatNormalizeToDuplicates()
 
             var exception = Assert.Throws<InvalidOperationException>(() => PackageCorePropertiesNormalizer.NormalizePackage(packagePath));
             Assert.Contains("docs/./readme.txt", exception.Message);
-            Assert.Contains("duplicate destination name docs/readme.txt", exception.Message);
+            Assert.Contains("duplicate destination name 'docs/readme.txt'", exception.Message);
+            Assert.False(File.Exists(packagePath + ".normalize-tmp"));
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(projectRoot);
+        }
+    }
+
+    [Fact]
+    public void PackageNormalizer_ScrubsSafeExternalAttributes()
+    {
+        var projectRoot = TestProjectHelper.CreateTempProject(nameof(PackageNormalizer_ScrubsSafeExternalAttributes));
+        try
+        {
+            var packagePath = Path.Combine(projectRoot, "external-attributes.nupkg");
+            CreatePackageWithAttributedEntries(
+                packagePath,
+                ("package/services/metadata/core-properties/random.psmdcp", "", UnixRegularFileAttributes(493)),
+                ("payload.bin", "payload", 0x20));
+
+            PackageCorePropertiesNormalizer.NormalizePackage(packagePath);
+
+            using var archive = ZipFile.OpenRead(packagePath);
+            Assert.All(archive.Entries, entry => Assert.Equal(0, entry.ExternalAttributes));
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(projectRoot);
+        }
+    }
+
+    [Fact]
+    public void PackageNormalizer_RejectsPosixSymlinkExternalAttributes()
+    {
+        var projectRoot = TestProjectHelper.CreateTempProject(nameof(PackageNormalizer_RejectsPosixSymlinkExternalAttributes));
+        try
+        {
+            var packagePath = Path.Combine(projectRoot, "symlink-attributes.nupkg");
+            CreatePackageWithAttributedEntries(
+                packagePath,
+                ("package/services/metadata/core-properties/random.psmdcp", "", 0),
+                ("payload.bin", "payload", UnixSymlinkAttributes()));
+
+            var exception = Assert.Throws<InvalidOperationException>(() => PackageCorePropertiesNormalizer.NormalizePackage(packagePath));
+            Assert.Contains("payload.bin", exception.Message);
+            Assert.Contains("unsafe POSIX file type symlink", exception.Message);
+            Assert.False(File.Exists(packagePath + ".normalize-tmp"));
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(projectRoot);
+        }
+    }
+
+    [Fact]
+    public void PackageNormalizer_RejectsUnsafeDosExternalAttributes()
+    {
+        var projectRoot = TestProjectHelper.CreateTempProject(nameof(PackageNormalizer_RejectsUnsafeDosExternalAttributes));
+        try
+        {
+            var packagePath = Path.Combine(projectRoot, "dos-attributes.nupkg");
+            CreatePackageWithAttributedEntries(
+                packagePath,
+                ("package/services/metadata/core-properties/random.psmdcp", "", 0),
+                ("payload.bin", "payload", 0x04));
+
+            var exception = Assert.Throws<InvalidOperationException>(() => PackageCorePropertiesNormalizer.NormalizePackage(packagePath));
+            Assert.Contains("payload.bin", exception.Message);
+            Assert.Contains("unsafe DOS attributes 0x04", exception.Message);
             Assert.False(File.Exists(packagePath + ".normalize-tmp"));
         }
         finally
@@ -559,14 +732,34 @@ private static void CreatePackageWithEntries(string packagePath, params (string
             WriteZipEntry(archive, entry.EntryName, entry.Content);
     }
 
-    private static void WriteZipEntry(ZipArchive archive, string entryName, string content)
+    private static void CreatePackageWithAttributedEntries(string packagePath, params (string EntryName, string Content, int ExternalAttributes)[] entries)
+    {
+        using var archive = ZipFile.Open(packagePath, ZipArchiveMode.Create);
+        foreach (var entry in entries)
+            WriteZipEntry(archive, entry.EntryName, entry.Content, entry.ExternalAttributes);
+    }
+
+    private static void WriteZipEntry(ZipArchive archive, string entryName, string content, int? externalAttributes = null)
     {
         var entry = archive.CreateEntry(entryName);
+        if (externalAttributes.HasValue)
+            entry.ExternalAttributes = externalAttributes.Value;
+
         using var stream = entry.Open();
         using var writer = new StreamWriter(stream, new UTF8Encoding(encoderShouldEmitUTF8Identifier: false));
         writer.Write(content.Replace("\r\n", "\n", StringComparison.Ordinal));
     }
 
+    private static int UnixRegularFileAttributes(int permissions)
+    {
+        return unchecked((int)((0x8000u | (uint)permissions) << 16));
+    }
+
+    private static int UnixSymlinkAttributes()
+    {
+        return unchecked((int)((0xA000u | 511u) << 16));
+    }
+
     private static string ReadZipEntryText(ZipArchive archive, string entryName)
     {
         var entry = archive.GetEntry(entryName) ?? throw new InvalidOperationException($"Missing ZIP entry: {entryName}");