Is your feature request related to a problem?
When running DSL vs PPL load tests on a managed OpenSearch 3.5 domain, I found that DSL's bottleneck was the entire cluster, while PPL's bottleneck was the sql-worker thread pool.
DSL queries execute on the resizeable search thread pool. When running expensive DSL queries on increasingly high volume data, it dynamically scales up to use more of the cluster's resources in response to the increased load. DSL performance only degraded when the cluster itself was strained from the load.
PPL queries execute on the fixed sql-worker thread pool. When running expensive PPL queries on increasingly high volume data, this thread pool eventually saturated. As a result, PPL tasks waited long times in thread pool queues, and PPL specifically degraded while the rest of the cluster moved on without issue.
What solution would you like?
Change the construction here to use a dynamic executor builder instead of a fixed executor builder:
|
new FixedExecutorBuilder( |
|
settings, |
|
SQL_BACKGROUND_THREAD_POOL_NAME, |
|
settings.getAsInt( |
|
"thread_pool.search.size", OpenSearchExecutors.allocatedProcessors(settings)), |
|
1000, |
|
"thread_pool." + SQL_BACKGROUND_THREAD_POOL_NAME)); |
What alternatives have you considered?
Deliberation may be needed on if this is a desirable change, and consideration of potential consequences for PPL work competing for cluster resources with other tasks.
Do you have any additional context?
N/A
Is your feature request related to a problem?
When running DSL vs PPL load tests on a managed OpenSearch 3.5 domain, I found that DSL's bottleneck was the entire cluster, while PPL's bottleneck was the
sql-workerthread pool.DSL queries execute on the resizeable
searchthread pool. When running expensive DSL queries on increasingly high volume data, it dynamically scales up to use more of the cluster's resources in response to the increased load. DSL performance only degraded when the cluster itself was strained from the load.PPL queries execute on the fixed
sql-workerthread pool. When running expensive PPL queries on increasingly high volume data, this thread pool eventually saturated. As a result, PPL tasks waited long times in thread pool queues, and PPL specifically degraded while the rest of the cluster moved on without issue.What solution would you like?
Change the construction here to use a dynamic executor builder instead of a fixed executor builder:
sql/plugin/src/main/java/org/opensearch/sql/plugin/SQLPlugin.java
Lines 354 to 360 in ee28e3d
What alternatives have you considered?
Deliberation may be needed on if this is a desirable change, and consideration of potential consequences for PPL work competing for cluster resources with other tasks.
Do you have any additional context?
N/A