DuckDB (like most database systems and many applications using memory allocators like jemalloc or mimalloc) doesn't immediately release memory back to the OS after freeing it internally.
Memory allocator strategy - DuckDB uses an allocator that keeps freed memory in a pool for reuse. Returning memory to the OS is expensive (system calls, page table updates), so allocators hold onto it anticipating future allocations.
Thanks for explaining this! I suspected there was some additional context and was digging into it now. Problem is that the memory never seems to be freed, and I could update my issue to show this.
Maintainers have acknowledged problems like this on other issues too.
This is expected behaviour, not a bug.
DuckDB (like most database systems and many applications using memory allocators like jemalloc or mimalloc) doesn't immediately release memory back to the OS after freeing it internally.
Memory allocator strategy - DuckDB uses an allocator that keeps freed memory in a pool for reuse. Returning memory to the OS is expensive (system calls, page table updates), so allocators hold onto it anticipating future allocations.
Also the docs say that dropping tables should free their memory:
"Running DROP TABLE should free the memory used by the table"
Source: https://duckdb.org/docs/stable/sql/statements/drop
Thanks for explaining this! I suspected there was some additional context and was digging into it now. Problem is that the memory never seems to be freed, and I could update my issue to show this.
Maintainers have acknowledged problems like this on other issues too.
But I'll nevertheless read more into this!
tbh i might dig a bit deeper as it reports zero memory use itself.