Skip to content

target/i386: Fix decoding of 0xC6/0xC7 (group 11) instructions - #2361

Open
unknown-1-0 wants to merge 1 commit into
unicorn-engine:devfrom
unknown-1-0:i386-fix-group-11-decoding
Open

target/i386: Fix decoding of 0xC6/0xC7 (group 11) instructions#2361
unknown-1-0 wants to merge 1 commit into
unicorn-engine:devfrom
unknown-1-0:i386-fix-group-11-decoding

Conversation

@unknown-1-0

@unknown-1-0 unknown-1-0 commented Jul 7, 2026

Copy link
Copy Markdown

According to both table A-6 in Volume 2 of Intel 64 and IA-32 Architectures Software Developer's Manual and table A-6 in Volume 3 of AMD64 Architecture Programmer's Manual, group 11 instructions are decoded as MOV only if ModRM.reg == 0.

Although Unicorn checks for that, it handles some things incorrectly:

  • In register-only operand form (ModRM.mod == 3), decoding of MOV instruction also succeeds if ModRM.reg == 7. However, according to table A-6 in Volume 2 of Intel SDM, this should encode XBEGIN/XABORT, not MOV!
  • Before the check, ModRM.reg field is ORed with the value of REX.R << 3, leading to an invalid opcode (#UD) exception if REX.R bit is actually set. Real Intel hardware seems to ignore the REX.R bit with group 11 instructions.

This commit fixes both issues.

@unknown-1-0
unknown-1-0 force-pushed the i386-fix-group-11-decoding branch 2 times, most recently from 9ba7ee0 to 7314158 Compare July 7, 2026 13:21
@unknown-1-0
unknown-1-0 force-pushed the i386-fix-group-11-decoding branch 2 times, most recently from e3973bc to 1493f07 Compare July 17, 2026 13:39
@unknown-1-0

unknown-1-0 commented Jul 17, 2026

Copy link
Copy Markdown
Author

Updated the tests and the commit message, fixed a typo in one of test names

According to both table A-6 in Volume 2 of Intel 64 and IA-32 Architectures
Software Developer's Manual and table A-6 in Volume 3 of AMD64 Architecture
Programmer's Manual, group 11 instructions are decoded as MOV only if
ModRM.reg == 0.

Although Unicorn checks for that, it handles some things incorrectly:
- In register-only operand form (ModRM.mod == 3), decoding of MOV
  instruction also succeeds if ModRM.reg == 7. However, according to
  table A-6 in Volume 2 of Intel SDM, this should encode XBEGIN/XABORT, not MOV!
- Before the check, ModRM.reg field is ORed with the value of REX.R << 3,
  leading to an invalid opcode (#UD) exception if REX.R bit is actually set.
  Real Intel hardware seems to ignore the REX.R bit with group 11 instructions.

This commit fixes both issues.

Signed-off-by: Andrey Polivoda <apolivodaa433@gmail.com>
@unknown-1-0
unknown-1-0 force-pushed the i386-fix-group-11-decoding branch from 1493f07 to 07c5a7f Compare July 22, 2026 16:21
@unknown-1-0

Copy link
Copy Markdown
Author

Force-pushed to sign the commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant