[Bug]: Fix failing integration tests

[mattcasters]

Apache Hop version?

2.19.0-SNAPSHOT

Java version?

OpenJDK 21

Operating system

Linux

What happened?

19 failed tests for a variety of reasons:

https://ci-builds.apache.org/job/Hop/job/Hop-integration-tests/lastCompletedBuild/testReport/

Issue Priority

Priority: 1

Issue Component

Component: Other

[mattcasters]

Plan to fix the 19 failing ITs (#2236)

Daily cycle is ~2h54m (down from ~4h15m) after #7506 (single-JVM suite runner + Surefire Report Output + WorkflowExecutor keeps input fields on the execution-result stream). Most new failures are age 2 since that landed.

Failure groups

Group	Count	Symptom	Likely cause
A. HTTP	11	localhost:80 connection refused	Nested runner: HOSTNAME falls back to parameter default instead of env http
B. Kafka	3	empty / unresolvable bootstrap.servers	Nested runner: BOOTSTRAP_SERVERS not reaching child workflows
C. AES	1	decrypt yields empty plaintext	#7497 re-binds encoder from hop-config (Hop) over docker AES2 env
D. WorkflowExecutor	1 (transforms.0083)	output fields include input f1	#7506 intentionally prepends input fields on execution-result stream
E. Beam	1 (suite_startup)	runner runs as Beam; missing filename	Single-JVM runner incompatible with Beam project run config
F. Env override	1 (parameters…0012)	TEST_0012 stays default	Env vars not winning under nested/single-JVM path
G. Samples	1 (age 41)	chronic 30m suite	Pre-existing; partially addressed by #7508

Approach (prefer fix runner/contracts; keep single-JVM speedup)

1. Confirm with docker ITs (CLIENT_UNZIP=false against the built client):

   CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh http
   CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh kafka
   CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh aes
   CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh transforms
   CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh parameters_and_variables
   CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh beam_directrunner

   Check whether HOP_IT_PER_TEST_JVM=true clears each failure (runner nesting vs product bug).

2. Fix variable/parameter plumbing for A/B/F: extend run-tests.sh / run-workflow.hwf so env hosts and suite params reach nested main-*.hwf (HOSTNAME, BOOTSTRAP_SERVERS, CASTLEMOCK_HOSTNAME, etc.).

3. Update 0083 expected field list for the new WorkflowExecutor schema (D) — do not revert issue #7494 : speed up integration tests with single-JVM runner and Surefire Report Output #7506.

4. Beam (E): force native local for the suite runner, or use per-test JVM only for Beam projects.

5. AES (C): put HOP_PASSWORD_ENCODER_PLUGIN=AES2 + HOP_AES_ENCODER_KEY in the aes project/env config so project enable does not wipe docker env.

6. Samples (G): find first failing sample in the long log; ignore external-only samples or fix product as needed.

Suggested PR split

• PR1: runner plumbing (A/B/F) + 0083 (D)
• PR2: Beam runner isolation (E)
• PR3: AES project encoder config (C)
• PR4: samples (G)

Escape hatch: HOP_IT_PER_TEST_JVM=true only where a suite truly cannot share a JVM (e.g. Beam) — avoid a global flip that would undo the ~1h savings.