Bad output on patch archive extraction

[heinermann]

When extracting storm.dll from a patch archive using mpqcli, it appears to be corrupted with extra bytes before the header. MPQEditor extracts the file without the extra bytes.

In a powershell,

curl -L -o patch.exe http://ftp.blizzard.com/pub/broodwar/patches/PC/BW-1161.exe
curl -L -o mpqcli.exe "https://github.com/TheGrayDot/mpqcli/releases/download/v0.9.6/mpqcli-windows-amd64.exe"
.\mpqcli.exe extract -f storm.dll -o . patch.exe

Source	Filesize
mpqcli	409,624 bytes
MPQEditor	409,600 bytes

Extra bytes before the magic MZ (PE header).
18 00 04 01 00 00 00 00 00 00 00 00 00 40 06 00 F2 7C 83 AE AD 72 C9 01

Do you know if this is actually part of the file that MPQEditor strips out magically or if it's a bug in mpqcli? The 4th dword seems to be the filesize.

[heinermann]

For now I'm using a workaround to trim those extra bytes until MZ which works for me so this is not pressing.

[thomaslaurenson]

Hey, this is an interesting scenario. From what I know about MPQ archives and patches.... this looks like a normal "Blizzard patch file" - not sure what the exact term is for them. Basically, files in a patch usually have a custom header that is used by the patching program to apply the patch files to the game client. And the actual file contents, follows the header.

I know the most about Vanilla WoW, so this is the example from there. But probably similar for other games. For early WoW clients, the BNUpdate.exe application ingests an MPQ archive, iterates each file, and applies the file based on the patch header (e.g., copy, apply diff, replace etc.). There are also "special" files in the archive to control how the patch is applied (e.g., patch.cmd). The start of the patch header is:

struct PatchFileHeader {
    uint16_t header_size = 0;
    uint8_t  version = 0;
    uint8_t  override_mode = 0;
    ...
};

So, the header size is an uint16_t, which is 2 bytes. The value stored in your example is 24 bytes. So, like you witnessed, move in 24 bytes, or remove the 24 byte header, and you get the actual file.

So, back to the actual issue reported.... the file is being extracted correctly, because the actual file in the MPQ is extracted as it is stored. The source for MPQEditor is not available, so I cannot determine if/how it handles this scenario. I did look through the Stormlib source code, and couldn't find any methods for handling these patch files. But I can open an issue in the Stormlib repo and see if the author has any insight.

Solution: If Stormlib provides an easy helper function, I could add in an argument to "strip" the patch file header. Otherwise, there are automated ways to do this, due to the nature of how mpqcli operates:

mpqcli read storm.dll BW-1161.exe | dd bs=1 skip=24 > storm.dll

Or a dynamic version that reads the header size:

mpqcli read storm.dll BW-1161.exe | { s=$(dd bs=2 count=1 2>/dev/null | od -An -tu2); dd bs=1 skip=$((s-2)); } > storm.dll

I know... this one is a bit of a mess! These are Linux (bash) examples, but pretty sure you can do something similar in PowerShell.

I will leave this issue open and report back any more findings.

[heinermann]

Thanks, I missed this reply. Yes I am stripping the header in my use case. Since this is intended and the workings of MPQEditor caused some confusion on my end then it's a non issue.

[thomaslaurenson]

Thanks for the reply. Closing the issue - but feel free to open or reply if there is anything else to add