On Wed, 28 Nov 2018 18:10:36 +0100
Post by Michal SuchánekAccording to top it is claws using memory. It's the top offender. Also
when I run claws it takes about 700M virtual/60M resident. When I
open a small folder this does not change much. When I open inbox this
grows to like 2G VSS/ 1G RSS, and when I open small folder again it
does not get down. Opening inbox does not seem any faster the second
time. So to me it looks like claws is hoarding large amounts of
memory for no good reason (other than freeing it properly sounds like
work).
If only it was that simple. :) Besides Claws Mail data, VSS/RSS also
include any data and code objects from shared libraries (libc, glibc,
gtk+, ...). Those are loaded only when first needed, and not merely on
the application startup. That's why e.g. opening of a first folder
makes the memory footprint grow considerably more than opening of a
second folder, regardless of size of either, because for the second
opening operation, most of what's needed is already loaded in memory.
Post by Michal SuchánekWorse, fork()
has to replicate the page tables for the process and it may run out of
memory for said tables which depend solely on VSS size.
The way I understand it, page tables themselves are not very big,
perhaps 1-2MB of kernel space
("grep VmPTE /proc/$(pidof claws-mail)/status" for exact number at a
given moment). Content of those memory pages is another matter, but
that is not duplicated at the time of fork(). The memory pages are
duplicated and allocated for the child process only when parent of
child writes to them later (copy-on-write).
That's why I find it incredible that on any today's decent desktop
system, things could go as far as fork() reporting ENOMEM.
That said, it is entirely possible that the page tables for our process
grow unnecessarily large, if there is a lot of small memory leaks (or
a few often triggered ones), and there is a lot of small "chunks" of
alloacted memory.
There is a lot of such memory leaks during Claws Mail runtime. Vast
majority of those come from shared libraries the program uses. From
what I remember when running Claws Mail through valgrind's memcheck,
the worst offenders are Glibc, GTK+ and Pango. We cannot do anything
about these leaks on our end, they have to be fixed in code of the
respective libraries.
Of course, Claws Mail itself is not perfect, and also has a few
long-standing tiny memory leaks (there are one or two infuriating leaks
related to toolbar button icons that I just can't figure out even after
hours of looking at the code), but almost all of them only happen once,
during UI initialization on startup, so they should not contribute to
the described problem.
There could also be leaks which are triggered over and over again. Just
a few months ago, we fixed a leak which IIRC leaked a few hundreds
of bytes on every POP3 mail fetch, along with a dozen or so other,
smaller leaks. What can I say, C is hard, and we make mistakes. :)
There are probably other uncaught leaks in the code, perhaps some
triggered only in conditions which I, or whoever bothers running Claws
Mail through valgrind or similar tool, do not happen to have, so they
go undetected, and after weeks of uptime, they inevitably add up.
Regards,
--
Andrej