Add missing elements from newer versions of CodeView.

[LightBender]

The code for PR was generated using an LLM.

LLM Model : Opus 4.8 (Extra High)
Prompt:
I've attached files for D's CodeView implementations. Using these files, we would like to add a new implementation that uses the latest information from Microsoft (here: https://github.com/microsoft/microsoft-pdb) and LLVM (here: https://llvm.org/docs/PDB/). We would like to add a new D file called "pdb.d" to the backend folder that implements the latest PDB information from LLVM with the goal being to achieve similar PDB quality and coverage as LLVM does. This new implementation should not touch the existing CodeView4/8 implementations but should be entirely self-sufficient with the intent being to eventually retire the existing implementations. This implementation can modify cgcv.d as needed, but edits that would effect CV4/CV8 should be kept minimal. This new PDB emitter should be hidden behind a preview compiler flag called "-preview=newpdb". This flag is to be threaded through the backend config. Please be sure to add pdb.d to the appropriate build scripts and VC++ projects.

I have attached two detailed analysis of features missing from D's CodeView generation. Both analysis uncovered different things so path attention to both. Considering that D may not support all of these features (example: we know that live sites and S_DEFRANGE are not supported in DMD) please make a plan to implemented as many of these as possible in pdb.d. Please included a detailed list of what you plan to implement and what you determine cannot be implemented with existing DMD infrastructure. The overall goal of this plan is to implement as much of CodeView as is possible with the existing DMD infrastructure, and then use the list of what cannot be implemented to potentially add that infrastructure at a later time. Please put the list of unimplemented CodeView features into a markdown file in the backend folder.

Things we would like to implement from the analysis that are not currently implemented:

1. Full TLS support as TLS is used extensively in D.
2. All Missing or Incomplete Symbol Records
3. All Missing or Incomplete Type Records (except anything Managed/.NET specific)
4. More pointer/modifier variants or numeric leaves.
5. Line/file enhancements. Thinks like Blake3 file hashing, Full section contributions, source file checksums/embedded sources.
6. Advanced line number encodings, build environment details.
7. D-specific extensions beyond basic structures/OEM (e.g., more precise array/delegate metadata) if possible.
8. Newer primitive type mappings or full CV_SIGNATURE_C13+ handling nuances
9. New basic symbols/types.
10. Anything recommend by the analysis and implementable without disturbing the rest of the compiler but not listed here.

Second Prompt:
Let's do one final check to make sure that we added everything that we can use for DMD from the latest CodeView/PDB implementations from Microsoft (here: https://github.com/microsoft/microsoft-pdb) and LLVM (here: https://llvm.org/docs/PDB/). If there are additional items to add, please add them.

Bug-fix prompts:
I received the following error while attempting to build an app:
core.exception.ArrayIndexError@src\dmd\common\outbuffer.d(282): index [192] is out of bounds for array of length 192

I am now getting the following linker warning:
coalesce.obj : warning LNK4209: debugging information corrupt; recompile module; linking object as if no debug info
I have attached the .obj file for you to use in your diagnosis.

[rainers]

I might be biased, but I'm not sure this is the right approach:

• it is unclear what the actual improvements are, the description is too vague
• there is quite a bit of duplication from cv8.d
• no test cases for the (new?) features
• you should enable the preview-flag temporarily to see what the test suite says (especially testpdb.d)

I'd rather prefer focused fixes to the existing code for specific bugs.

[LightBender]

@rainers The duplication is intentional. There have been a lot of new features added to CodeView since CV8, and branching all of them out for a preview switch would have made CV8 a much more complicated file to review and maintain.

Which gets to why this PR in the first place. This isn't about fixing a bug or two. There are a lot of new features that CV8 does not implement, we're on CV13 now according to documents from LLVM, and this adds as much as makes sense for D.

As for test cases, I can certainly add those. I can try to enable the preview switch for testing, but I strongly suspect that all the additions will break the current tests. Should be fun to watch the failure though.

Here is full explanation of everything added/fixed.

1. New symbol records (not emitted by cv8.d)

Record	Opcode	Purpose	cv8.d
S_COMPILE3	0x113C	Modern compiland flags: language, machine, front/back-end versions	Uses legacy S_COMPILE (0x0001)
S_ENVBLOCK	0x113D	Build environment (cwd, command), double-null terminated	— none —
S_BUILDINFO	0x114C	References an LF_BUILDINFO ID record	— none —
S_FRAMEPROC	0x1012	Per-function frame info; encodes RBP as local/param base pointer on x64	— none —
S_RETURN	0x000D	Marks the epilogue so the debugger's "step out" works	Explicitly skipped ("VC doesn't, so we won't")
S_CONSTANT	0x1107	Enum members / manifest constants (SC.const_)	No SC.const_ handling
S_UDT alias via LF_ALIAS	0x150A	SC.typedef_ named typedefs	No SC.typedef_ handling

---

2. New type / ID-stream records (not in cv8.d)

cv8.d emits no ID-stream records. pdb.d adds helpers that build them in the
shared debtyp pool:

Helper	Leaf	Opcode	Purpose
pdb_string_id	LF_STRING_ID	0x1605	Interned string → type index
pdb_buildinfo	LF_BUILDINFO	0x1603	cwd, tool, source, pdb, args
pdb_func_id	LF_FUNC_ID	0x1601	Ties a function type to its name
pdb_alias	LF_ALIAS	0x150A	Named typedef of an underlying type
pdb_udt_src_line	LF_UDT_SRC_LINE	0x1606	Source file/line where a UDT is defined

---

3. Enhanced source-file & line info

Feature	pdb.d	cv8.d
File checksums (F4)	Real 32-byte SHA256 slot computed with blake3	Writes an empty checksum (kind 0, no hash)
Checksum kind constants	CHKSUM_NONE/MD5/SHA1/SHA256 named enum	Magic 0/0x0110 inline
Subsection kinds	Named DEBUG_S_SYMBOLS/LINES/STRINGTABLE/FILECHKSMS	Magic 0xF1..0xF4 inline
First record	Correctly named S_OBJNAME (0x1101)	Same opcode, mislabeled S_COMPILAND_V3

---

4. Full TLS support

Item	pdb.d	cv8.d
Thread-local data	Named S_GTHREAD32 / S_LTHREAD32 constants	Hard-coded magic 0x1113 / 0x1112

Values are identical, but pdb.d uses documented named constants and treats TLS
as a first-class path.

---

5. Named constants instead of magic numbers

pdb.d defines a local, documented enum block for all symbol/leaf opcodes
(S_GPROC32, S_LPROC32, S_REGREL32, S_REGISTER_V3, S_LDATA32,
S_GDATA32, S_BLOCK32, S_ENDARG, S_END, etc.) rather than the inline
literals used in several cv8.d paths (e.g. register-relative 0x1111,
S_ENDARG written as raw 0x000A only in the block helper).

---

6. Structural / robustness changes

Change	Detail
Buffer init	pdb_initfile uses a single initBuf helper (alloc + reset) instead of repeated __gshared blocks
Buffer growth safety	S_FRAMEPROC header uses growing write16 (not write16n) to avoid an out-of-bounds write past the reserved region
UDT routing	cv8_udt forwards to pdb_udt when config.newpdb is set, so struct/enum/class UDTs emitted by the front-end glue land in the PDB buffers
Name encoding	pdb_writename mirrors cv8_writename (D .→@ under -gms) with an early-out structure

---

7. Behavior shared / reused (not reimplemented)

pdb.d intentionally does not duplicate these; they come from the shared
cgcv.d/cv4.d/cv8.d type pool via cv_typidx:

• D dynamic arrays (cv8_darray), delegates (cv8_ddelegate),
  associative arrays (cv8_daarray)
• Struct/union/class forward refs (cv8_fwdref)
• Register-number mapping (pdb_regnum mirrors cv8_regnum)
• The .debug$T type stream writer (cv_term)

---

8. Not yet implemented

Tracked in pdb_unimplemented.md; blocked on missing
DMD backend infrastructure:

• S_LOCAL + S_DEFRANGE_* (no per-instruction variable live-range data)
• S_INLINESITE / S_INLINESITE_END (no inline-site metadata)
• S_HEAPALLOCSITE, S_FRAMECOOKIE, FPO/PGO data
• All managed/.NET records (out of scope for D)

---

Summary

pdb.d reaches modern CodeView parity by adding the ID stream
(LF_STRING_ID/LF_BUILDINFO/LF_FUNC_ID/LF_UDT_SRC_LINE/LF_ALIAS),
real file checksums, and the module records modern debuggers expect
(S_OBJNAME, S_COMPILE3, S_ENVBLOCK, S_BUILDINFO, S_FRAMEPROC,
S_RETURN), plus first-class TLS, enum-constant, and typedef handling — all
behind -preview=newpdb and without disturbing the legacy CV4/CV8 paths.

[rainers]

I have compared pdb.d (a misnomer as this is still CV written to an obj file) and cv8.d, most of the changes are just renaming and reformatting. Using identifiers instead of magic numbers is good.

pdb.d reaches modern CodeView parity by adding the ID stream

Not much materializes, though:

(LF_STRING_ID/LF_BUILDINFO/LF_FUNC_ID/LF_UDT_SRC_LINE/LF_ALIAS),

The only string added is "dmd".

real file checksums,

It hashes the filename, not the content of the file!

and the module records modern debuggers expect
(S_OBJNAME, S_COMPILE3, S_ENVBLOCK, S_BUILDINFO,

Indeed, the header is a bit more verbose. It ignores the configured language, though.

S_FRAMEPROC, S_RETURN),

These could be some actual improvements, but it only emits dummy values. It also omits the "member function type info" in cv8.

plus first-class TLS

No change

enum-constant and typedef handling

These are not available in the backend AFAICT.

[rikkimax]

I have compared pdb.d (a misnomer as this is still CV written to an obj file) and cv8.d, most of the changes are just renaming and reformatting. Using identifiers instead of magic numbers is good.

I covered that at beerconf, but didn't insist on a name change. Even though it is required.

Adam was under the belief that the pdb file would be emitted directly, when it's the linker that does it.

[LightBender]

@rainers That bit of feedback made the LLM work out hard. Hopefully it looks better. What are we thinking for names? IMO, at this point we really probably want to just merge everything into one file since we will never be emitting CV4 (OMF) again, and we're a bit beyond CV8.

[rainers]

I see some improvements (language selection restored, frameproc now has some info).

The hash is calculated from the file content now, but it should not have to read the file again. It is already in Module.src and should be passed down somehow or retrieved from FileManager.getFileContents().

So far there are still only tiny improvements that you won't notice in a debugging session. I think these could just be applied to cv8.d.

Long term, we could ditch CV4 support, but there are still some remnants for OBJ_OMF that should be removed first. Then cv8.d might be renamed codeview.d while only using a cv_ prefixes for function names.

There are a couple of more files involved (tocvdebug.d, cgcv.d). Not sure if merging these is useful or whether there is a sensible separation.

[LightBender]

Alright @rainers here it is. I've merged cv4.d+cv8.d+pdb.d into a single file called codeview.d and kept the relevant additions from pdb.d. I didn't merge cgcv.d because that was a bigger project than I wanted to tackle in a single PR, but it could be done we if deem it a reasonable refactoring. I did not remove the last elements of OBJ_OMF as that is for a different PR. This commit will also try to get the file contents and hash those instead of reloading from the disk.

[rainers]

Unfortunately this is growing to become even harder to review. I was hoping for smaller PRs rather than larger ones ;) Taking step 3 (renaming to codeview and cv_-functions) before the other 2 (eliminating OMF and CV4) now produces the strange call chain that a cv-function calls a cv4-function that conditionally calls a cv_ function for CV8 to do the actual work.

Keeping cv8 for both module and function names would probably be good enough to make this step manageable.

[rainers]

BTW: I think it needs Walters approval to remove all the ancient CodeView code that is inherited from the DMC compiler backend.
@WalterBright are you ok with support for CV4 (only used by obsolete OMF) being removed from the backend? This should also include the even older types CVOLD (Codeview 1), CVSYM (Symantec format) and CVTDB (Symantec format written to file).

[LightBender]

@rainers I've reworked this PR to only included the CodeView additions/improvements to the cv8.d emitter. I also made sure that the magic numbers were replaced with constants. I also made sure that LF_FUNC_ID and LF_UDT_SRC_LINE functions were correctly wired in instead of just sitting un-used. LF_UDT_SRC_LINE was particularly tricky to get correct, but it is now. Finally, I added a bunch of tests to testpdb.d.

This PR should be a bit smaller now, but the added tests will definitely add some weight to the review.

[rainers]

Thanks, looks a lot better, especially with the added tests. Just a few nits remaining...

[LightBender]

Note: FreeBSD 13.2 is an out-dated test that we replaced last weekend.