Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak when connecting and disconnecting from WiFi repeatedly #10664

Open
1 task done
lptr opened this issue Nov 29, 2024 · 9 comments
Open
1 task done

Memory leak when connecting and disconnecting from WiFi repeatedly #10664

lptr opened this issue Nov 29, 2024 · 9 comments
Assignees
Labels
Status: Awaiting triage Issue is waiting for triage

Comments

@lptr
Copy link

lptr commented Nov 29, 2024

Board

ESP32-S3

Device Description

I can reproduce this with multiple ESP32 devboards, including an ESP32-S3 devkit-1. It also reproduces under Wokwi, see https://wokwi.com/projects/415900372828990465.

Version

3.0.7, 3.1.0-RC1

Description

When connecting and disconnecting from WiFi repeatedly, a small amount of memory is leaking during each iteration.

Sketch

#include <Arduino.h>
#include <WiFi.h>

void setup() {
  Serial.begin(115200);
}

void loop() {
  while (true) {
    Serial.println("-----------------");
    Serial.println("Total free heap: " + String(ESP.getFreeHeap()));
    Serial.print("Connecting");
    WiFi.begin("Wokwi-GUEST", "", 6);
    while (WiFi.status() != WL_CONNECTED) {
      delay(250);
      Serial.print(".");
    }
    Serial.println(" done");
    delay(1000);

    Serial.print("Disconnecting");
    WiFi.disconnect(true);
    while (WiFi.status() == WL_CONNECTED) {
      delay(250);
      Serial.print(".");
    }
    WiFi.mode(WIFI_OFF);
    Serial.println(" done");
    Serial.println();
    Serial.println();
  }
}

Also see the reproducer at: https://github.com/lptr/esp-arduino-wifi-connect-loop-leak.

It is an ESP-IDF application using Arduino as a component.

It repeatedly connects and disconnects WiFi using WiFi.begin(...) and WiFi.disconnect(true) and WiFi.mode(WIFI_OFF). It measures the total heap and prints per-task allocations after each iteration.

Build with idf.py build and flash it to an ESP32 board, or open the related diagram.json with Wokwi and run the simulation.

Debug Message

Here's a few consecutive iterations' worth of logs from the reproducer:


-----------------
Total free heap: 295716
Task 0x0: Pre-Scheduler al CAP_8BIT: 3688, CAP_32BIT: 0
Task 0x3fcaafc4: main             CAP_8BIT: 15380, CAP_32BIT: 0
Task 0x3fca85d8: ipc0             CAP_8BIT: 14288, CAP_32BIT: 0
Task 0x3fca8c54: ipc1             CAP_8BIT: 24, CAP_32BIT: 0
Task 0x3fcad834: tiT              CAP_8BIT: 96, CAP_32BIT: 0
Task 0x3fcb2420:                  CAP_8BIT: 312, CAP_32BIT: 0
Task 0x3fcae934: sys_evt          CAP_8BIT: 100, CAP_32BIT: 0
Task 0x3fcafab0: arduino_events   CAP_8BIT: 100, CAP_32BIT: 0
Connecting..... done
Disconnecting done
====== Heap Trace: 5 records (256 capacity) ======
   340 bytes (@ 0x3fcb2404, Internal) allocated CPU 0 ccount 0x73cb7e8c caller 0x4037e034:0x4037ec8f:0x4208a66c:0x42067a7d:0x42064536:0x4204a523:0x4204a983:0x4203b152:0x4200be53:0x4200bf41
    16 bytes (@ 0x3fcac8f0, Internal) allocated CPU 1 ccount 0x82bd81f8 caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df0c:0x4201e030:0x4202af55:0x4202b011:0x4037d7c5:
    16 bytes (@ 0x3fcb7824, Internal) allocated CPU 1 ccount 0x82be6358 caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df0c:0x4201e030:0x4202af55:0x4202b011:0x4037d7c5:
    16 bytes (@ 0x3fcac8cc, Internal) allocated CPU 1 ccount 0x82beeacc caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df0c:0x4201e030:0x4202af55:0x4202b011:0x4037d7c5:
    16 bytes (@ 0x3fcb5ff8, Internal) allocated CPU 1 ccount 0x8a6d9da4 caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df39:0x4202b8d5:0x4202d021:0x4202d035:0x42017b3e:0x42017bec
====== Heap Trace Summary ======
Mode: Heap Trace Leaks
404 bytes 'leaked' in trace (5 allocations)
records: 5 (256 capacity, 142 high water mark)
total allocations: 205
total frees: 235
================================


-----------------
Total free heap: 295680
Task 0x0: Pre-Scheduler al CAP_8BIT: 3688, CAP_32BIT: 0
Task 0x3fcaafc4: main             CAP_8BIT: 15380, CAP_32BIT: 0
Task 0x3fca85d8: ipc0             CAP_8BIT: 14288, CAP_32BIT: 0
Task 0x3fca8c54: ipc1             CAP_8BIT: 24, CAP_32BIT: 0
Task 0x3fcad834: tiT              CAP_8BIT: 112, CAP_32BIT: 0
Task 0x3fcb2420:                  CAP_8BIT: 312, CAP_32BIT: 0
Task 0x3fcae934: sys_evt          CAP_8BIT: 100, CAP_32BIT: 0
Task 0x3fcafab0: arduino_events   CAP_8BIT: 100, CAP_32BIT: 0
Connecting..... done
Disconnecting done
====== Heap Trace: 5 records (256 capacity) ======
   340 bytes (@ 0x3fcb601c, Internal) allocated CPU 0 ccount 0x958bccac caller 0x4037e034:0x4037ec8f:0x4208a66c:0x42067a7d:0x42064536:0x4204a523:0x4204a983:0x4203b152:0x4200be53:0x4200bf41
    16 bytes (@ 0x3fcac8f0, Internal) allocated CPU 1 ccount 0xa8e321f8 caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df0c:0x4201e030:0x4202af55:0x4202b011:0x4037d7c5:
    16 bytes (@ 0x3fcb7824, Internal) allocated CPU 1 ccount 0xa8e40358 caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df0c:0x4201e030:0x4202af55:0x4202b011:0x4037d7c5:
    16 bytes (@ 0x3fcac8cc, Internal) allocated CPU 1 ccount 0xa8e48acc caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df0c:0x4201e030:0x4202af55:0x4202b011:0x4037d7c5:
    16 bytes (@ 0x3fcb7ad8, Internal) allocated CPU 1 ccount 0xac2b50a4 caller 0x4202b3f7:0x420174b6:0x42017521:0x4201de7c:0x4201df39:0x4202b8d5:0x4202d021:0x4202d035:0x42017b3e:0x42017bec
====== Heap Trace Summary ======
Mode: Heap Trace Leaks
404 bytes 'leaked' in trace (5 allocations)
records: 5 (256 capacity, 142 high water mark)
total allocations: 208
total frees: 239
================================

Things to notice:

  • Total free heap keeps decreasing,
  • the tiT task's allocation count keeps increasing,
  • the backtraces point to timeouts in the TCP/IP stack being allocated but not freed in some cases.

Here's the resolved backtrace of the many similar allocations reported by the heap leak detector:

mem_malloc
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/core/mem.c:209
do_memp_malloc_pool
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/core/memp.c:254
memp_malloc
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/core/memp.c:350 (discriminator 2)
sys_timeout_abs
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/core/timeouts.c:193
lwip_cyclic_timer
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/core/timeouts.c:265
sys_check_timeouts
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/core/timeouts.c:403
tcpip_timeouts_mbox_fetch
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/api/tcpip.c:111
tcpip_thread
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/lwip/lwip/src/api/tcpip.c:143
vPortTaskWrapper
C:/Tools/Espressif/frameworks/esp-idf-v5.1.4/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:164

I have checked existing issues, online documentation and the Troubleshooting Guide

  • I confirm I have checked existing issues, online documentation and Troubleshooting guide.
@lptr lptr added the Status: Awaiting triage Issue is waiting for triage label Nov 29, 2024
@lptr lptr changed the title Connecting and disconnecting from WiFi repeatedly leaks memory Memory leak when connecting and disconnecting from WiFi repeatedly Nov 29, 2024
@Jason2866
Copy link
Collaborator

Very detailed issue report! Looks like an issue of IDF as reported from the tool heap leak detector you used.

@lptr
Copy link
Author

lptr commented Nov 29, 2024

It has already been checked by the ESP-IDF maintainers in this issue: espressif/esp-idf#8446. The conclusion was that, when using ESP-IDF directly, some memory is retained even after disconnect, but is only a fixed amount that does not grow with subsequent connect/disconnect cycles.

In this Arduino-based reproducer the free heap constantly decreases with each subsequent connect/disconnect cycle.

So my suspicion is that the problem might lie on the arduino-esp32 side.

@lptr
Copy link
Author

lptr commented Nov 29, 2024

Here's an ESP-IDF-only variant. It does not seem to leak any memory: https://wokwi.com/projects/415915821999976449. This seems to confirm that the problem is on the arduino-esp32 side.

@Jason2866
Copy link
Collaborator

Jason2866 commented Nov 29, 2024

Can you test with Arduino 3.1.0rc3? It is the most actual one. Based on IDF 5.3.1
Do you know what Arduino core is your test using on wokwi?
Your latest test example is running with IDF 5.3.1 so it still can be a fault in IDF 5.1.4.
Since Arduino core 3.0.x is based on IDF 5.1.4.

@lptr
Copy link
Author

lptr commented Nov 29, 2024

I tried with Arduino ESP32 3.1.0-RC1 / IDF 5.3.1, and the problem is fully reproducible in the same way.

I'm having some trouble setting up a build with 3.1.0-RC3, because idf.py build tells me arduino-esp32 needs IDF 5.1.4. (Same with 3.1.0-RC2.) I'm not sure why. I'll try to work around this and report back.

@yuvashrikarunakaran
Copy link

yuvashrikarunakaran commented Dec 4, 2024

void *mem_malloc(mem_size_t size) {
    if (size == 0) {
        return NULL;
    }
    // Perform alignment and allocation logic
    void *mem = malloc(size); // Simplified for brevity
    return mem;
}
void *do_memp_malloc_pool(const struct memp_desc *desc) {
    void *memp = mem_malloc(desc->size); // Allocates memory
    if (memp) {
        memset(memp, 0, desc->size);
    }
    return memp;
}
void sys_timeout_abs(u32_t abs_time, sys_timeout_handler handler, void *arg) {
    struct sys_timeo *timeout = (struct sys_timeo *)memp_malloc(MEMP_SYS_TIMEOUT);
    if (timeout == NULL) {
        return;
    }
    timeout->abs_time = abs_time;
    timeout->handler = handler;
    timeout->arg = arg;

    // Add to the linked list of timeouts
    add_to_timeout_list(timeout);
}
void sys_check_timeouts(void) {
    struct sys_timeo *timeout = get_next_timeout();
    while (timeout && is_time_expired(timeout)) {
        timeout->handler(timeout->arg);
        remove_from_timeout_list

@shaojiajun314
Copy link

is it fixed? my program's mem is leak too. Maybe, this bug did.I have not tested

@Nezaemmy
Copy link

Yeatarday My AP had issues connecting and disconnecting. I use esp32 as a homekit; after some time the esp32 was hanging and the lamps turned OFF and ON again. Before, I didn't have the issue with AP working well.
@me-no-dev may be the cause is this bug

@TD-er
Copy link
Contributor

TD-er commented Dec 17, 2024

Can you try again with the current code base?
Yesterday I made a PR to initialize members of NetworkClient.

However I was not sure about whether the destructor of NetworkClient should also call the same as is done in NetworkClient::stop():

  if (clientSocketHandle) {
    clientSocketHandle->close();
  }
  clientSocketHandle = NULL;

Maybe this keeps some clientSocketHandle alive when not explicitly destructed?
Perhaps worth a look to see if this is related to this memory leak as NetworkClientSocketHandle only has a single int as members, thus the sizeof this object wrapped in a shared_ptr could be roughly 16 bytes (pointer, shared count, weak shared count + 4 bytes for the int)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Awaiting triage Issue is waiting for triage
Projects
None yet
Development

No branches or pull requests

7 participants