Skip to content

Fix blocking call warning in Home Assistant by running MQTT connection setup in executor#20

Merged
eman merged 2 commits intomainfrom
blocking
Oct 17, 2025
Merged

Fix blocking call warning in Home Assistant by running MQTT connection setup in executor#20
eman merged 2 commits intomainfrom
blocking

Conversation

@eman
Copy link
Copy Markdown
Owner

@eman eman commented Oct 17, 2025

Problem

Home Assistant was detecting blocking calls to open() inside the event loop when the MQTT client attempted to connect, causing this warning:

WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args (PosixPath('/usr/local/lib/python3.13/site-packages/awsiotsdk-1.26.0.dist-info/METADATA'),) inside the event loop by custom integration 'nwp500' at custom_components/nwp500/coordinator.py, line 143: connected = await self.mqtt_client.connect()

The issue occurs because the AWS IoT SDK performs synchronous file I/O operations when reading metadata files during connection setup in mqtt_connection_builder.websockets_with_default_aws_signing().

Solution

This PR wraps the blocking connection setup operations in an asyncio executor to avoid blocking the main event loop. The changes are:

  1. Wrapped connection setup in executor: The mqtt_connection_builder.websockets_with_default_aws_signing() call and credential provider creation now run in a thread pool via asyncio.run_in_executor()

  2. Moved credential creation into executor: The _create_credentials_provider() call is now inside the executor function to ensure all potentially blocking operations are isolated

  3. Added clear documentation: Comments explain why the executor is needed and what operations are being protected

Changes Made

  • Modified NavienMqttClient.connect() method in src/nwp500/mqtt_client.py
  • No breaking changes - the API remains exactly the same
  • All existing functionality is preserved

Benefits

  • ✅ Resolves Home Assistant blocking call warnings
  • ✅ Maintains full API compatibility (non-breaking change)
  • ✅ Preserves all existing functionality
  • ✅ Thread-safe implementation using asyncio's default executor
  • ✅ Future-proof against other potential blocking operations in AWS IoT SDK

Testing

  • All existing tests pass (29/29)
  • Code passes linting and formatting checks
  • Verified that run_in_executor is properly called during connection setup
  • Confirmed the fix addresses the specific Home Assistant warning scenario

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

References

This resolves Home Assistant warnings about blocking calls to open() inside
the event loop. The AWS IoT SDK performs synchronous file I/O operations
when reading metadata files during connection setup.

Changes:
- Wrap mqtt_connection_builder.websockets_with_default_aws_signing() in executor
- Move credentials provider creation into the executor
- Add documentation explaining why executor is needed

Fixes Home Assistant warning:
'Detected blocking call to open with args (...awsiotsdk...METADATA) inside
the event loop by custom integration'

This is a non-breaking change that maintains full API compatibility.
@eman eman requested a review from Copilot October 17, 2025 20:14
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses a blocking I/O warning in Home Assistant by moving synchronous AWS IoT SDK connection setup off the event loop.

  • Wraps MQTT connection and credential provider creation in a thread executor to avoid blocking.
  • Adds inline comments explaining the rationale for the executor use.

Replace the custom nested _build_connection + run_in_executor pattern with
asyncio.to_thread() for clearer intent and better performance:

- Uses asyncio.to_thread() for credentials provider creation
- Uses asyncio.to_thread() for connection builder call
- Avoids redefining functions on every connect() call
- More explicit about threading blocking operations
- Uses current event loop implicitly
- Cleaner, more readable code

This is the modern Python 3.9+ approach for handling blocking operations
in async code.
@eman
Copy link
Copy Markdown
Owner Author

eman commented Oct 17, 2025

🔄 Improved Implementation

I've updated the implementation based on feedback to use asyncio.to_thread() instead of the custom nested function + run_in_executor pattern. This provides several benefits:

✨ Improvements Made

  1. Cleaner Code: No more nested function definitions
  2. Better Performance: Avoids redefining _build_connection on every call
  3. Clearer Intent: asyncio.to_thread() explicitly shows we're running blocking operations in a thread
  4. Modern Approach: Uses the Python 3.9+ recommended pattern for async/blocking integration
  5. Implicit Loop Usage: Uses the current event loop automatically

📋 Changes

Before:

def _build_connection():
    credentials_provider = self._create_credentials_provider()
    return mqtt_connection_builder.websockets_with_default_aws_signing(...)

self._connection = await self._loop.run_in_executor(None, _build_connection)

After:

credentials_provider = await asyncio.to_thread(self._create_credentials_provider)
self._connection = await asyncio.to_thread(
    mqtt_connection_builder.websockets_with_default_aws_signing,
    endpoint=self.config.endpoint,
    region=self.config.region,
    credentials_provider=credentials_provider,
    # ... other args
)

✅ Testing

  • All existing tests still pass (29/29)
  • Code passes linting and formatting checks
  • Verified asyncio.to_thread() is called correctly for both operations
  • Confirmed the fix still resolves the Home Assistant blocking call warnings

The implementation is now cleaner, more efficient, and follows modern Python async best practices!

@eman eman merged commit 13ce708 into main Oct 17, 2025
10 checks passed
@eman eman deleted the blocking branch October 17, 2025 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants