[Bug]: Heap memory leak when enabling MeshMonitor on Heltec V4 (firmware 2.7.15 and 2.7.19)

[Budarov]

Category

WiFi

Hardware

Other

Is this bug report about any UI component firmware like InkHUD or Meshtatic UI (MUI)?

• Meshtastic UI aka MUI colorTFT
• InkHUD ePaper
• OLED slide UI on any display

Firmware Version

2.7.15

Description

When MeshMonitor is enabled, my Heltec V4 device the heap memory usage continuously increases until it reaches the limit, causing the node to crash and restart. This cycle repeats approximately every 1.5 hours. Disabling MeshMonitor stabilizes the heap usage, and the node runs without restarts.

Relevant log output

[github-actions]

@Budarov, Welcome to Meshtastic! 👋

Thanks for opening your first issue. If it's helpful, an easy way
to get logs is the "Open Serial Monitor" button on the Web Flasher.

If you have ideas for features, note that we often debate big ideas
in the discussions tab
first. This tracker tends to be for ideas that have community
consensus and a clear implementation.

We're very active on discord,
especially the #firmware and #alphanauts-testing channels. If you'll
be around for a while, we'd love to see you there!

Welcome to the community! ❤️

[github-actions]

Please provide the relevant log output related to the heap memory usage when enabling MeshMonitor. Additionally, include the steps to reproduce the issue and the device model/variant details.

[caveman99]

@Budarov if you can compile the firmware yourself, you can enable the heap debugger. that should pinpoint where this leak is roughly happening.

[Budarov]

@Budarov if you can compile the firmware yourself, you can enable the heap debugger. that should pinpoint where this leak is roughly happening.

I will try

[Budarov]

I collected some statistics on heap consumption and reboots throughout the day.

[samwiseg0]

@caveman99 I was able to compile the firmware with what I believe is the correct flag for heap debug. For my node, which is an RAK4631 I added this.

build_flags = ${nrf52840_base.build_flags}
  -I variants/nrf52840/rak4631
  -D RAK_4631
  -DMESHTASTIC_USE_EINK_UI=0
  -DRADIOLIB_EXCLUDE_SX128X=1
  -DRADIOLIB_EXCLUDE_SX127X=1
  -DRADIOLIB_EXCLUDE_LR11X0=1
  -D DEBUG_HEAP
  -D USE_LOG_DEBUG
  -D LOG_DEBUG_INC=\"DebugConfiguration.h\"

Is this correct? Is there something specific I should be looking for in the console logs after flashing the device?

Debug images compiled (Based on tag v2.7.19.bb3d6d5)

RAK4631

firmware-rak4631-2.7.19.a092f6b_debug_heap_fixed.zip

Heltec v4

firmware-heltec-v4-2.7.19.a092f6b_debug_heap_fixed.zip

@Budarov you can flash your device with the firmware above. I just added -D DEBUG_HEAP to the platformio.ini file for your hardware. Use at your own risk.

build_flags = 
  ${heltec_v4_base.build_flags} ;-Os
  -D HELTEC_V4_TFT
  -D I2C_SDA=4
  -D I2C_SCL=3
  -D I2C_SDA1=47
  -D I2C_SCL1=48
  -D PIN_BUTTON2=35
  -D PIN_BUZZER=6
  -D USE_PIN_BUZZER=PIN_BUZZER
  -D CONFIG_ARDUHAL_LOG_COLORS
  -D RADIOLIB_DEBUG_SPI=0
  -D RADIOLIB_DEBUG_PROTOCOL=0
  -D RADIOLIB_DEBUG_BASIC=0
  -D RADIOLIB_VERBOSE_ASSERT=0
  -D RADIOLIB_SPI_PARANOID=0
  -D CONFIG_DISABLE_HAL_LOCKS=1
  -D INPUTDRIVER_BUTTON_TYPE=0
  -D HAS_SCREEN=1
  -D HAS_TFT=1
  -D RAM_SIZE=1560
  -D LV_LVGL_H_INCLUDE_SIMPLE
  -D LV_CONF_INCLUDE_SIMPLE
  -D LV_COMP_CONF_INCLUDE_SIMPLE
  -D LV_USE_SYSMON=0
  -D LV_USE_PROFILER=0
  -D LV_USE_PERF_MONITOR=0
  -D LV_USE_MEM_MONITOR=0
  -D LV_USE_LOG=0
  -D LV_BUILD_TEST=0
  -D DEBUG_HEAP
  -D USE_LOG_DEBUG
  -D LOG_DEBUG_INC=\"DebugConfiguration.h\"
...

Edit: I had placed the flags in the incorrect location. If it is successfully enabled it will show the following in the logs.

...
HEAP | 13:48:55 187 [Router] [heap 46092] Allocated static pool item 3 at 0x2000e1c8
INFO  | 13:48:55 187 [Router] [heap 46084] Rebroadcast received message coming from 0
INFO  | 13:48:55 187 [Router] [heap 46072] Packet History - insert: Reusing slot aged 0.024s TRACE MATCHED PACKET
...

[Budarov]

@Budarov you can flash your device with the firmware above. I just added -D DEBUG_HEAP to the platformio.ini file for your hardware. Use at your own risk.
Hi,

I compiled the firmware myself with -D DEBUG_HEAP added in firmware/variants/esp32s3/heltec_v4/platformio.ini and also tested with your pre-built firmware (same behavior).

build_flags = 
  ${heltec_v4_base.build_flags}
  -D HELTEC_V4_OLED
  -D USE_SSD1306 ; Heltec_v4 has an SSD1315 display (compatible with SSD1306 driver)
  -D LED_POWER=35
  -D RESET_OLED=21
  -D I2C_SDA=17
  -D I2C_SCL=18
  -D I2C_SDA1=4
  -D I2C_SCL1=3
  -D DEBUG_HEAP

When heap usage reaches ~85–90%, I try to get more detailed logs by running:
meshtastic --port COM7 --noproto
after the CLI opens the port (literally within 1–2 seconds), the device instantly reboots — no chance to capture any meaningful debug output

[samwiseg0]

@Budarov you can flash your device with the firmware above. I just added -D DEBUG_HEAP to the platformio.ini file for your hardware. Use at your own risk.
Hi,

I compiled the firmware myself with -D DEBUG_HEAP added in firmware/variants/esp32s3/heltec_v4/platformio.ini and also tested with your pre-built firmware (same behavior).

build_flags = 
  ${heltec_v4_base.build_flags}
  -D HELTEC_V4_OLED
  -D USE_SSD1306 ; Heltec_v4 has an SSD1315 display (compatible with SSD1306 driver)
  -D LED_POWER=35
  -D RESET_OLED=21
  -D I2C_SDA=17
  -D I2C_SCL=18
  -D I2C_SDA1=4
  -D I2C_SCL1=3
  -D DEBUG_HEAP

When heap usage reaches ~85–90%, I try to get more detailed logs by running: meshtastic --port COM7 --noproto after the CLI opens the port (literally within 1–2 seconds), the device instantly reboots — no chance to capture any meaningful debug output

Can you try and disconnect/shutdown MeshMonitor before opening the serial connection to the device? Mine does not crash as fast as yours but I do see the heap not releasing especially if I keep MeshMonitor open in a browser.

[samwiseg0]

I was able to capture 2 full runs where my node would crash and reboot after there was no memory left.

Config

# start of Meshtastic configure yaml
channel_url: https://meshtastic.org/e/#REDACTED
config:
  bluetooth:
    enabled: true
    fixedPin: REDACTED
    mode: 1
  device:
    buzzerGpio: 21
    nodeInfoBroadcastSecs: 10800
    tzdef: EST5EDT,M3.2.0/2:00:00,M11.1.0/2:00:00
  display:
    screenOnSecs: 600
    wakeOnTapOrMotion: true
  lora:
    bandwidth: 250
    codingRate: 5
    configOkToMqtt: true
    hopLimit: 4
    region: US
    spreadFactor: 11
    sx126xRxBoostedGain: true
    txEnabled: true
    txPower: 30
    usePreset: true
  mqtt:
    encryptionEnabled: false
  network:
    ethEnabled: true
    ntpServer: meshtastic.pool.ntp.org
  position:
    broadcastSmartMinimumDistance: 100
    broadcastSmartMinimumIntervalSecs: 300
    gpsMode: 1
    gpsUpdateInterval: 1800
    positionBroadcastSecs: 3600
    positionBroadcastSmartEnabled: true
    positionFlags: 43
  power:
    lsSecs: 300
    minWakeSecs: 10
    sdsSecs: 4294967295
    waitBluetoothSecs: 60
  security:
    adminKey:
    - base64:
    - base64:
    - base64:
    privateKey: base64:REDACTED=
    publicKey: base64:REDACTED=
    serialEnabled: true
location:
  alt: REDACTED
  lat: REDACTED
  lon: REDACTED
module_config:
  ambientLighting:
    blue: 26
    current: 10
    green: 143
    red: 117
  cannedMessage:
    enabled: true
  detectionSensor:
    detectionTriggerType: LOGIC_HIGH
    minimumBroadcastSecs: 45
  externalNotification:
    active: true
    alertMessage: true
    alertMessageBuzzer: true
    nagTimeout: 15
    output: 36
    outputMs: 1000
    usePwm: true
  mqtt:
    address: mqtt.REDACTED.net
    enabled: true
    encryptionEnabled: true
    mapReportSettings:
      positionPrecision: 14
      publishIntervalSecs: 3600
      shouldReportLocation: true
    password: large4cats
    root: msh/US
    username: meshdev
  neighborInfo:
    transmitOverLora: true
  serial:
    enabled: true
  telemetry:
    deviceTelemetryEnabled: true
    deviceUpdateInterval: 3600
    environmentDisplayFahrenheit: true
    environmentMeasurementEnabled: true
    environmentUpdateInterval: 3600
owner: REDACTED
owner_short: REDACTED

Logs

meshtastic_debug_heap_MeshMonitor_crash_logs.zip

@Budarov Can you remove the needs-logs label from this issue?