Feature Requests item #540952, was opened at 2002-04-08 05:01 Message generated for change (Comment added) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=540952&group_id=5470

Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Allan Crooks (amc1) Assigned to: Nobody/Anonymous (nobody) Summary: Memory Usage Reporting

Initial Comment: I would personally like a way in Python to report how many bytes of memory that the interpreter is using (perhaps through the sys module)?

If this sort of mechanism is added, then it may allow SoftReferences (a la Java) to be introduced, which would definitely be useful for memory sensitive caches...

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody) Date: 2004-03-22 10:31

Message: Logged In: NO

What is the minimum worth including?

For instance, objmalloc.c states that it never returns an arena to the operating system, so just (narenas * ARENA_SIZE) is an upper bound on python-managed generic small object memory. Whether that is enough of the used memory (or a tight enough bound, in case of good VM), I can't yet judge.

When I have wanted it, it was to ask either "Do I have to shrink my cache yet?" or "Is this getting slow because I've taken too much memory for the system it is running on". Both probably require more a accurate version. (Walking the individual arenas and pools?)

----------------------------------------------------------------------

Comment By: Jean M. Brouwers (jbrouwers) Date: 2003-07-28 16:57

Message: Logged In: YES user_id=832557

It would be useful to get some figure(s) about the memory usage of the current process.

For example, just the 'high water mark' returned by sbrk(0) on uni* is very helpful in many cases.

I tried the resource.getrusage(resource.RUSAGE_SELF) function but that does return all zero's in slice [2:6] on my uni* system. Not sure whether that is an artifact of the O/S or Python.

----------------------------------------------------------------------

Comment By: Anthony Baxter (anthonybaxter) Date: 2002-04-10 07:03

Message: Logged In: YES user_id=29957

You _can_ do this for most O/Ss, true. The problem is that it's something that's extremely operating system specific, and, worse yet, varies significantly between operating system releases. Have a look at the source for the system utility 'top' one day - most of it is in the 'operating system specific' section, and there's all sorts of horrible horrible hoops you have to jump through for different releases.

In addition, the notion of "how much memory is being used" isn't always that clear in the case of things like shared libraries.

On the plus side, implementing this would move the HP/UX threading problem to #2 on the list of "most annoying operating-system dependent bugs".

----------------------------------------------------------------------

Comment By: Allan Crooks (amc1) Date: 2002-04-09 07:45

Message: Logged In: YES user_id=39733

Thanks for the response.

Rather than having an internal way of figuring out how much memory is used, is there no external way that we can get the OS to return how big the process is?

I know that it is less than elegant, and would have to have different platform-dependent code to do this, but it is still an idea. I know that Jython can quiz the VM to see how much memory is being used, and I'll assume that you can do that with different OS's as well...

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one) Date: 2002-04-08 09:38

Message: Logged In: YES user_id=31435

Allan, this isn't easy, so the cost/benefit ratio is high. For the most part, Python gets memory from the system malloc, and doesn't even try to keep track of it now; nor has it any idea how much overhead (padding, control bytes) the system malloc adds; nor is there a portable interface to C's malloc for finding out such things.

Still, I agree it would be nice to have such things <wink>.

----------------------------------------------------------------------