Skip to content

check_app_updates() takes 2+ seconds every utility loop due to unconditional ast.parse on all .py files #2599

@chase1124

Description

@chase1124

What happened?

Summary

Every utility loop iteration, Dependencies.update() unconditionally calls refresh_dep_graph(), which re-parses all .py
files via ast.parse() / builtins.compile — regardless of whether any file has changed. On a project with 32 .py files
this costs ~2.2s per loop, causing a constant stream of Excessive time spent in utility loop warnings.

Prior report with the same symptom but no diagnosis: #1811

Profiler output (check_app_updates_profile: true)

32674 function calls (30117 primitive calls) in 2.269 seconds

check_app_updates()
└─ check_app_python_files() cumtime=1.919s
└─ dependency_manager.update_python_files() cumtime=1.917s
└─ dependency_manager.refresh_dep_graph() cumtime=1.915s
└─ dependency.get_dependency_graph() cumtime=1.768s
└─ dependency.get_file_deps() × 32 files cumtime=1.870s total
├─ ast.parse() × 32 cumtime=1.320s
└─ builtins.compile × 32 tottime=1.549s ← THE BOTTLENECK

check_app_config_files (Pydantic YAML parsing) took ~0.063s — not significant.

Root cause

In dependency_manager.py, Dependencies.update() looks like this (simplified):

def update(self, new_files):
    self.files.update(new_files)
    # ... bad_files cleanup ...
    self.refresh_dep_graph()  # always runs — even when nothing changed

self.files is a FileCheck object. After self.files.update() is called, FileCheck.there_were_changes is already populatedit
returns True if any file was added, modified, or deleted, and False otherwise. The guard just isn't wired up.

Fix

One line added after self.files.update(new_files):

def update(self, new_files):
    self.files.update(new_files)
    if not self.files.there_were_changes:
        return  # skip the ~2s AST reparse when nothing changed
    # ... bad_files cleanup + refresh_dep_graph only on actual changes ...
    self.refresh_dep_graph()

Verification

Applied this as a monkey-patch in a local AppDaemon app. Result: zero Excessive time in check_app_updates warnings after the
patch initializes. Hot-reload still works correctly at ~1s latency (the guard is there_were_changes, so a real file change
still triggers the full dep graph rebuild). The first 12 loop iterations after startup still run the full parseexpected,
since the patch applies during initialize().

Tested on AppDaemon addon v0.18.3 (AppDaemon 4.5.13).

Environment

- AppDaemon 4.5.13 (addon v0.18.3)
- 32 .py files in app directory
- utility_delay at default (1s)
- Linux / Home Assistant OS

---

### Version

4.5.13

### Installation type

Home Assistant add-on

### Relevant log output

```sh

Relevant code in the app or config file that caused the issue

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    issueSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions