feat(glue-alpha): ensure job parameters consistency

[otaviomacedo]

The CfnJob L1 takes a defaultArguments property, which are the arguments it runs with by default. The various Job L2 constructs build this argument from two sources:

1. A defaultArguments provided directly by the user.
2. Arguments owned by the L2s themselves, but derived from user provided arguments. For instance, if it's a Scala job, it includes --job-language scala; if the user passed continuousLogging.logGroup, it will include--continuous-log-logGroup <group name> and so on.

The main problem, both for usability and security, is that users can pass default arguments that conflict with construct-owned ones. For example, users can create a job with

{
  defaultArguments: { '--continuous-log-logGroup': 'foo' },
  continuousLogging: { logGroup: 'bar' },
}

Currently, defaultArguments silently wins. This can weaken the user's security posture because the L2 constructs have secure defaults, such as continuous logging being enabled. If the user accidentally passes defaultArguments['--enable-continuous-cloudwatch-log']: false, it will silently disable logs for the job.

One possible solution would be to invert the precedence of the arguments, such that the construct owned ones win in case of conflict. But then the problem would flip, and user provided parameter would be silently ignored. Despite being more secure than the current behavior, it is also more confusing.

The solution adopted here is to make them mutually exclusive, so that there is only way to configure an argument: either via the strongly typed API if it exists, or directly via defaultArguments as an escape hatch. If a given argument is present in both, the construct throws an error.

Also, as mentioned, Glue jobs enable continuous CloudWatch logging by default across all job types. But unless a SecurityConfiguration with cloudWatchEncryption is attached, driver/executor stdout and stderr are written to the account-shared default log group (/aws-glue/jobs/logs-v2/) in plaintext. Job logs routinely contain sensitive runtime data (SQL statements, row values, error stack traces), so this is a silent gap for regulated workloads. This PR adds a warning when setting up continuous logging if there is no security configuration set.

Note: the --enable-metrics and --enable-observability-metrics arguments that were being emitted by PythonShellJob were removed. They are no-ops, probably copied from SparkJobProps and RayJobProps. This was confirmed by deploying a job with and without the arguments, and both worked identically. This is a behavior change, in that it modifies the synthesized template, but it has no runtime impact.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

BREAKING CHANGE: Glue job constructs now reject construct-managed and Glue-reserved
arguments passed through defaultArguments. Previously, a managed argument set via
defaultArguments was silently honored in SparkJob and PythonShellJob (customer
value won over the construct default) and silently ignored in RayJob (construct
default won). Both behaviors let a caller bypass the construct's security and
observability defaults with no error. Passing any of the following through
defaultArguments now throws a ValidationError at synthesis time:

• construct-managed arguments — --enable-continuous-cloudwatch-log,
  --continuous-log-logGroup, --continuous-log-logStreamPrefix,
  --continuous-log-conversionPattern, --enable-continuous-log-filter,
  --enable-metrics, --enable-observability-metrics, --enable-spark-ui,
  --spark-event-logs-path, --job-language, --class, --extra-jars,
  --user-jars-first, --extra-py-files, --extra-files, library-set
• Glue-reserved arguments — --debug, --mode, --JOB_NAME

Configure these through their dedicated props instead (continuousLogging,
enableMetrics, enableObservabilityMetrics, sparkUI, className, extraJars,
extraJarsFirst, extraPythonFiles, extraFiles). For example, replace
defaultArguments: { '--enable-continuous-cloudwatch-log': 'false' } with
continuousLogging: { enabled: false }. Arguments without a dedicated prop (e.g.
--enable-glue-datacatalog) are unaffected and remain settable via defaultArguments.

The checkNoReservedArgs(defaultArguments?) method on the Job base class was removed and replaced with protected mergeManagedArguments(managedArguments, defaultArguments?)

[github-actions]

⚠️ This pull request description does not follow the correct template structure.

PRs without a linked issue will receive lower priority for review and merging. Please update the description to follow the PR template and include a line like Closes #123 in the Issue section. If no existing issue matches your change, create one first.

[aws-cdk-automation]

(This review is outdated)

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.