Skip to content

Bulk import persistent EMR max capacity is validated even when managed scaling is disabled #6789

@patchwork01

Description

@patchwork01

Description / Background

The property sleeper.bulk.import.persistent.emr.max.capacity is only used when sleeper.bulk.import.persistent.emr.use.managed.scaling is set to true.

If you set sleeper.bulk.import.persistent.emr.min.capacity to 10 or higher, this is refused because it's less than the max capacity, even if you haven't set the max capacity, and managed scaling is not enabled.

Steps to reproduce

  1. Set sleeper.bulk.import.persistent.emr.use.managed.scaling to false
  2. Set sleeper.bulk.import.persistent.emr.min.capacity to 10 or higher
  3. Leave sleeper.bulk.import.persistent.emr.max.capacity unset
  4. See the instance configuration fails to validate

Expected behaviour

The instance configuration should be accepted as valid.

Technical Notes / Implementation Details

This is a problem with the validation comparing the min and max capacity. That should only be checked when managed scaling is enabled, but it's actually always checked.

This is tested in InstancePropertiesTest and implemented in InstanceProperties.validate.

This was caused when the validation was introduced in the following PR:

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions