Dataclass-like types accept the same values whether or not added as a group

CHANGELOG.rst

@@ -44,6 +44,9 @@ Added
 - Support ``Collection``, ``Container`` and ``Reversible``, validated as a list,
   and ``AbstractSet``, validated as a set (`#950
   <https://github.com/mauvilsa/jsonargparse/pull/950>`__).
+- ``shtab`` completion scripts now include the fields of a dataclass-like type
+  that is not added as a group, e.g. ``Optional[SomeDataclass]`` (`#952
+  <https://github.com/mauvilsa/jsonargparse/pull/952>`__).
 
 Fixed
 ^^^^^
@@ -103,6 +106,23 @@ Fixed
   input data from keyword arguments``. Now the nearest class docstring in the
   method resolution order is used, skipping base classes that only provide
   machinery (`#951 <https://github.com/mauvilsa/jsonargparse/pull/951>`__).
+- A dataclass-like type accepting the ``class_path`` of a subclass when it is
+  not added as a group, e.g. ``Optional[SomeDataclass]``, even though subclasses
+  are disabled for these types by default. The error now says how to enable
+  subclass support for the type, see :ref:`subclasses-disabled` (`#952
+  <https://github.com/mauvilsa/jsonargparse/pull/952>`__).
+- Signature parameters typed as a dataclass-like type that is not added as a
+  group, e.g. ``Optional[SomeDataclass]``, ``list[SomeDataclass]`` and
+  ``dict[str, SomeDataclass]``, not accepting a path to a sub-config file even
+  when added with ``sub_configs=True``, failing with e.g. ``No module named
+  'data'``. Also, the path of a loaded sub-config was not kept, so ``save`` with
+  ``multifile=True`` did not write it back to its own file. See
+  :ref:`sub-config-files` (`#952
+  <https://github.com/mauvilsa/jsonargparse/pull/952>`__).
+- The value of an environment variable not validating for a dataclass-like typed
+  argument of a subcommand, e.g. ``APP_SUB__DATA='{"p1": 2}'`` failing with
+  ``Not a valid subclass of ...`` (`#952
+  <https://github.com/mauvilsa/jsonargparse/pull/952>`__).
 
 Changed
 ^^^^^^^
@@ -138,6 +158,12 @@ Changed
   that they no longer prevent the remaining subtypes from being attempted. See
   the new documentation section :ref:`union-types` (`#949
   <https://github.com/mauvilsa/jsonargparse/pull/949>`__).
+- A dataclass-like type added as a group now accepts a subclass spec that has
+  the ``class_path`` of the type itself, e.g. ``{"class_path": "Data",
+  "init_args": {...}}``, instead of failing with ``Group 'data' does not accept
+  option 'init_args....'``. Dataclass-like types now accept the same values
+  whether or not they are added as a group, see :ref:`subclasses-disabled`
+  (`#952 <https://github.com/mauvilsa/jsonargparse/pull/952>`__).
 
 
 v4.50.0 (2026-07-22)

DOCUMENTATION.rst

@@ -2370,6 +2370,14 @@ moved around without needing to modify them. Furthermore, :meth:`save
 <.ArgumentParser.save>` with ``multifile=True`` writes back each sub-config to
 its own file, preserving the original structure.
 
+Dataclass-like types, see :ref:`subclasses-disabled`, also accept a sub-config
+file, the difference being that its content are the fields of the type, without
+``class_path`` and ``init_args``. Note that this only applies when the type is
+not added as an argument group, i.e. when it is part of a larger type, e.g.
+``Optional[SomeDataclass]`` or ``list[SomeDataclass]``. When added as a group,
+the group's own config argument accepts the path, e.g. ``--data=data.yaml``,
+independent of ``sub_configs``.
+
 
 .. _instance-factories:
 
@@ -2675,6 +2683,16 @@ classes technically support subclassing, subclass support can be enabled as
 described below. Subclass support has been kept disabled for these types by
 default to avoid introducing breaking changes.
 
+A type with subclasses disabled is added as an argument group when it is the
+entire type of an argument, such that each of its init args is an individual
+argument, e.g. ``--data.number``. This is not the case when the type is part of
+a larger type, e.g. ``Optional[FinalClass]`` or ``list[FinalClass]``, since then
+a single argument must accept the entire value. Independent of this, the
+accepted values are the same. A subclass spec is accepted, though only with the
+``class_path`` of the type itself, i.e. ``--data={"class_path": "FinalClass",
+"init_args": {"number": 8}}``. The ``class_path`` of a subclass is not accepted,
+unless subclass support is enabled for the type as described next.
+
 
 .. _enable-disable-subclasses:
 

jsonargparse/_actions.py

@@ -229,6 +229,10 @@ def is_print_config_requested(parser):
         return False
 
 
+class SubclassesDisabledError(TypeError):
+    """Raised when a class_path is given for a type that has subclasses disabled."""
+
+
 class _ActionConfigLoad(Action):
     def __init__(self, basetype: type | None = None, **kwargs):
         if len(kwargs) == 0:
@@ -253,14 +257,56 @@ def __call__(self, *args, **kwargs):
         namespace[self.dest] = loaded_value
         return None
 
+    def resolve_subclass_spec(self, value):
+        """Resolves a subclass spec given for a subclasses disabled type, e.g. a dataclass.
+
+        These types are added as a group, such that their init args are individual arguments. Still a
+        subclass spec is accepted, so that the accepted values are the same as for the optional
+        counterpart of the type, which is added as a single typehint argument.
+        """
+        from ._typehints import is_subclass_spec, resolve_class_path_by_name, subclasses_disabled_message
+
+        if self.basetype is None or not is_subclasses_disabled(self.basetype):
+            return value
+
+        def resolve_class(class_path):
+            try:
+                return import_object(resolve_class_path_by_name(self.basetype, class_path))
+            except Exception:
+                return None
+
+        if isinstance(value, str):
+            if resolve_class(value) is None:
+                return value  # not a class path, e.g. a path to a config file
+            value = Namespace(class_path=value)
+        elif not is_subclass_spec(value):
+            return value
+        class_path = value["class_path"]
+        if resolve_class(class_path) is not self.basetype:
+            raise SubclassesDisabledError(subclasses_disabled_message(self.basetype, class_path))
+        resolved = Namespace()
+        for key in ["init_args", "dict_kwargs"]:
+            sub_value = value.get(key)
+            if isinstance(sub_value, dict):
+                sub_value = Namespace(sub_value)
+            if isinstance(sub_value, Namespace):
+                resolved.update(sub_value)
+        return resolved
+
     def _load_config(self, value, parser):
         try:
-            cfg, cfg_path = parse_value_or_config(value)
-            if not isinstance(cfg, dict):
+            cfg = self.resolve_subclass_spec(value)
+            cfg_path = None
+            if cfg is value:
+                cfg, cfg_path = parse_value_or_config(value)
+                cfg = self.resolve_subclass_spec(cfg)
+            if not isinstance(cfg, (dict, Namespace)):
                 raise TypeError(f'Parser key "{self.dest}": Unable to load config "{value}"')
             with load_config_path_context(cfg_path), change_to_path_dir(cfg_path):
                 cfg = parser._apply_actions(cfg, parent_key=self.dest)
             return cfg
+        except SubclassesDisabledError as ex:
+            raise TypeError(f'Parser key "{self.dest}":\n{indent_text(str(ex))}') from ex
         except (TypeError,) + get_loader_exceptions() as ex:
             str_ex = indent_text(f"- {ex}")
             raise TypeError(f'Parser key "{self.dest}":\nUnable to load config {value!r}\n{str_ex}') from ex

jsonargparse/_completions.py

@@ -14,7 +14,12 @@
 from typing import Literal, Union
 
 from ._actions import ActionConfigFile, ActionFail, _ActionConfigLoad, _ActionHelpClassPath, remove_actions
-from ._common import NonParsingAction, get_optionals_as_positionals_actions, get_parsing_setting
+from ._common import (
+    NonParsingAction,
+    get_optionals_as_positionals_actions,
+    get_parsing_setting,
+    is_subclasses_disabled,
+)
 from ._optionals import get_pydantic_path_type
 from ._parameter_resolvers import get_signature_parameters
 from ._typehints import (
@@ -23,6 +28,7 @@
     get_all_subclass_paths,
     get_callable_return_type,
     get_typehint_origin,
+    is_single_subclass_or_closed_type,
     is_subclass,
     type_to_str,
 )
@@ -347,6 +353,13 @@
             choices = add_subactions_and_get_subclass_choices(typehint, prefix, parser, skip, added_subclasses)
             return choices, True, False
 
+        if is_single_subclass_or_closed_type(typehint, origin) and is_subclasses_disabled(typehint):
+            # a closed type, e.g. a dataclass, only inlined as a group when not in a union,
+            # so its init args need to be added as options for them to be completed
+            added_subclasses.add(typehint)
+            add_subactions_and_get_subclass_choices(typehint, prefix, parser, skip, added_subclasses, closed_type=True)
+            return [], False, True
+
         if origin in callable_origin_types:
             return_type = get_callable_return_type(typehint)
             if return_type and ActionTypeHint.is_subclass_typehint(return_type):
@@ -363,26 +376,31 @@
     return choices, require_prefix
 
 
-def add_subactions_and_get_subclass_choices(typehint, prefix, parser, skip, added_subclasses) -> list[str]:
-    choices = []
-    paths = get_all_subclass_paths(typehint)
+def add_subactions_and_get_subclass_choices(
+    typehint, prefix, parser, skip, added_subclasses, closed_type: bool = False
+) -> list[str]:
+    choices: list[str] = []
     init_args = defaultdict(list)
     subclasses = defaultdict(list)
-    for path in paths:
-        choices.append(path)
+    # a closed type is not a choice, since only its init args are accepted
+    classes: list = [typehint] if closed_type else get_all_subclass_paths(typehint)
+    for class_or_path in classes:
+        name = class_or_path if isinstance(class_or_path, str) else class_or_path.__name__
+        if isinstance(class_or_path, str):
+            choices.append(class_or_path)
         try:
-            cls = import_object(path)
+            cls = import_object(class_or_path) if isinstance(class_or_path, str) else class_or_path
             params = get_signature_parameters(cls, None, parser._logger)
         except Exception as ex:
-            parser._logger.debug(f"Unable to get signature parameters for '{path}': {ex}")
+            parser._logger.debug(f"Unable to get signature parameters for '{name}': {ex}")
             continue
         num_skip = next((s for s in skip if isinstance(s, int)), 0)
         if num_skip > 0:
             params = params[num_skip:]
         for param in params:
             if param.name not in skip:
                 init_args[param.name].append(param.annotation)
-                subclasses[param.name].append(path.rsplit(".", 1)[-1])
+                subclasses[param.name].append(name.rsplit(".", 1)[-1])
 
     if prefix is not None:
         for name, subtypes in init_args.items():
@@ -394,8 +412,9 @@
                         subtype, option_string, parser, skip, added_subclasses
                     )
                     if shtab_shell.get() == "bash":
-                        message = f"Expected type: {type_to_str(subtype)}; "
-                        message += f"Accepted by subclasses: {', '.join(subclasses[name])}"
+                        message = f"Expected type: {type_to_str(subtype)}"
+                        if not closed_type:
+                            message += f"; Accepted by subclasses: {', '.join(subclasses[name])}"
                         add_bash_typehint_completion(
                             parser,
                             action,

jsonargparse/_core.py

@@ -367,6 +367,7 @@ def _parse_common(
         skip_required: bool = False,
         skip_subcommands: bool = False,
         fail_no_subcommand: bool = True,
+        nested_parse: bool = False,
     ) -> Namespace:
         """Common parsing code used by other parse methods.
 
@@ -378,6 +379,8 @@ def _parse_common(
             skip_required: Whether to skip check of required arguments.
             skip_subcommands: Whether to skip subcommand processing.
             fail_no_subcommand: Whether to fail if no subcommand given.
+            nested_parse: Whether the result is to be merged into an ongoing parse of another parser,
+                in which case the internal representation of values must be preserved.
 
         Returns:
             A config object with all parsed values.
@@ -406,7 +409,7 @@ def _parse_common(
             if not skip_validation:
                 self.validate(cfg, skip_required=skip_required)
 
-        if not lenient_check.get():
+        if not lenient_check.get() and not nested_parse:
             cfg = subclasses_disabled_remove_class_path(cfg)
 
         return cfg
@@ -607,7 +610,9 @@ def parse_env(
         Raises:
             ArgumentError: If the parsing fails and ``exit_on_error=False``.
         """
-        skip_validation, skip_subcommands = get_private_kwargs(kwargs, _skip_validation=False, _skip_subcommands=False)
+        skip_validation, skip_subcommands, nested_parse = get_private_kwargs(
+            kwargs, _skip_validation=False, _skip_subcommands=False, _nested_parse=False
+        )
 
         try:
             cfg = self._parse_defaults_and_environ(defaults, env=True, environ=env)
@@ -617,6 +622,7 @@ def parse_env(
                 "defaults": defaults,
                 "skip_validation": skip_validation,
                 "skip_subcommands": skip_subcommands,
+                "nested_parse": nested_parse,
             }
             if skip_validation:
                 kwargs["fail_no_subcommand"] = False
@@ -1383,6 +1389,7 @@ def _apply_actions(
             with parser_context(parent_parser=self, lenient_check=True):
                 value = self._check_value_key(action, value, action_dest, prev_cfg, append=append)
             if isinstance(action, _ActionConfigLoad):
+                value = action.resolve_subclass_spec(value)
                 config_keys.add(action_dest)
                 keys.append(action_dest)
             elif isinstance(action, ActionConfigFile):

jsonargparse/_signatures.py

@@ -455,11 +455,12 @@ def _add_signature_parameter(
                 prefix = f"{name}.init_args."
                 nested_skip = {s[len(prefix) :] for s in skip or [] if s.startswith(prefix)}
                 sub_add_kwargs["skip"] = nested_skip
+            # also_closed since dataclass-like types accept a sub-config when not added as a group
             enable_path = sub_configs and (
-                is_subclass_typehint
+                ActionTypeHint.is_subclass_typehint(annotation, all_subtypes=False, also_closed=True)
                 or is_return_subclass_typehint
                 or is_list_pathlike(annotation)
-                or is_subclass_container_typehint(annotation)
+                or is_subclass_container_typehint(annotation, also_closed=True)
             )
             args = ActionTypeHint.prepare_add_argument(
                 args=args,
@@ -588,6 +589,8 @@ def _create_group_if_requested(
             name = obj.__name__ if nested_key is None else nested_key
             group = self.add_argument_group(strip_title(doc_group), name=name)
             if config_load and nested_key is not None:
+                if config_load_type is None and inspect.isclass(obj):
+                    config_load_type = obj
                 group.add_argument("--" + nested_key, action=_ActionConfigLoad(basetype=config_load_type))
             if inspect.isclass(obj) and nested_key is not None and instantiate:
                 group.dest = nested_key.replace("-", "_")

jsonargparse/_subcommands.py

@@ -262,7 +262,9 @@ def handle_subcommands(
         subnamespace = None
         key = prefix + subcommand
         if env:
-            subnamespace = subparser.parse_env(defaults=defaults, _skip_validation=True)
+            # nested_parse so that the values keep the same internal representation as the ones
+            # already in cfg, otherwise merging the two gives a value that is neither
+            subnamespace = subparser.parse_env(defaults=defaults, _skip_validation=True, _nested_parse=True)
         elif defaults:
             subnamespace = subparser.get_defaults(skip_validation=True)