Fix #12796: proper diagnostic for invalid custom attribute array element types

Previously, using an array of a user-defined type as a custom attribute
argument (e.g. [<DefaultValue([||] : A[])>]) caused an internal error
FS0192 in encodeCustomAttrElemType. The ECMA 335 metadata format requires
the array element type to be encoded even for empty arrays, and only
primitive types, enums, string, System.Type, and System.Object are valid.

This change adds a validation check in GenAttribArg (IlxGen.fs) that
detects unsupported element types before reaching the IL writer, and
reports a proper diagnostic (FS3885) instead of crashing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: T-Gro

docs/release-notes/.FSharp.Compiler.Service/11.0.100.md

@@ -19,6 +19,7 @@
 * Fixed how the source ranges of warn directives are reported (as trivia) in the parser output (by not reporting leading spaces). ([Issue #19405](https://github.com/dotnet/fsharp/issues/19405), [PR #19408]((https://github.com/dotnet/fsharp/pull/19408)))
 * Fix UoM value type `ToString()` returning garbage values when `--checknulls+` is enabled, caused by double address-taking in codegen. ([Issue #19435](https://github.com/dotnet/fsharp/issues/19435), [PR #19440](https://github.com/dotnet/fsharp/pull/19440))
 * Fix completion inconsistently showing some obsolete members (fields and events) while hiding others (methods and properties). All obsolete members are now consistently hidden by default. ([Issue #13512](https://github.com/dotnet/fsharp/issues/13512), [PR #19506](https://github.com/dotnet/fsharp/pull/19506))
+* Fix internal error `FS0192: encodeCustomAttrElemType` when using an array of user-defined type as a custom attribute argument. Now reports a proper diagnostic (FS3885). ([Issue #12796](https://github.com/dotnet/fsharp/issues/12796), [PR #19472](https://github.com/dotnet/fsharp/pull/19472))
 
 ### Added
 

src/Compiler/CodeGen/IlxGen.fs

@@ -10223,6 +10223,18 @@ and GenAttribArg amap g eenv x (ilArgTy: ILType) =
     // Detect '[| ... |]' nodes
     | Expr.Op(TOp.Array, [ elemTy ], args, m), _ ->
         let ilElemTy = GenType amap m eenv.tyenv elemTy
+
+        // Validate element type is encodable in custom attribute metadata (ECMA 335).
+        // Only primitive types, enums, string, System.Type, and System.Object are valid.
+        match ilElemTy with
+        | ILType.Boxed tspec when
+            tspec.Name <> "System.String"
+            && tspec.Name <> "System.Object"
+            && tspec.Name <> "System.Type"
+            ->
+            error (Error(FSComp.SR.ilCustomAttrInvalidArrayElemType (tspec.Name), m))
+        | _ -> ()
+
         ILAttribElem.Array(ilElemTy, List.map (fun arg -> GenAttribArg amap g eenv arg ilElemTy) args)
 
     // Detect 'typeof<ty>' calls

src/Compiler/FSComp.txt

@@ -1814,3 +1814,4 @@ featurePreprocessorElif,"#elif preprocessor directive"
 3882,lexHashElifMustBeFirst,"#elif directive must appear as the first non-whitespace character on a line"
 3883,lexHashElifMustHaveIdent,"#elif directive should be immediately followed by an identifier"
 3884,tcFunctionValueUsedAsInterpolatedStringArg,"This expression is a function value. When used in an interpolated string it will be formatted using its 'ToString' method, which is likely not the intended behavior. Consider applying the function to its arguments."
+3885,ilCustomAttrInvalidArrayElemType,"The type '%s' is not a valid custom attribute argument type. Custom attribute arrays must have elements of primitive types, enums, string, System.Type, or System.Object."