| From | Sent On | Attachments |
|---|---|---|
| Marilyn Davis | Jan 13, 2005 2:04 am | |
| Danny Yoo | Jan 13, 2005 2:31 am | |
| Danny Yoo | Jan 13, 2005 2:41 am | |
| Marilyn Davis | Jan 13, 2005 3:17 am | |
| Danny Yoo | Jan 13, 2005 6:29 am | |
| Alan Gauld | Jan 13, 2005 10:20 am | |
| Marilyn Davis | Jan 15, 2005 11:19 pm | |
| Marilyn Davis | Jan 16, 2005 3:12 am | |
| Marilyn Davis | Jan 16, 2005 6:47 am | |
| Danny Yoo | Jan 16, 2005 7:40 am | |
| Marilyn Davis | Jan 17, 2005 5:02 am | |
| Danny Yoo | Jan 18, 2005 10:51 am | |
| Danny Yoo | Jan 18, 2005 7:24 pm | |
| Marilyn Davis | Jan 19, 2005 2:32 am | |
| Danny Yoo | Jan 19, 2005 8:12 am | |
| Kent Johnson | Jan 19, 2005 12:35 pm | |
| Marilyn Davis | Jan 19, 2005 8:57 pm | |
| Marilyn Davis | Jan 19, 2005 9:13 pm | |
| Danny Yoo | Jan 19, 2005 9:53 pm | |
| Marilyn Davis | Jan 19, 2005 10:28 pm | |
| Marilyn Davis | Jan 21, 2005 5:05 am |
| Subject: | [Tutor] sockets, files, threads | |
|---|---|---|
| From: | Marilyn Davis (mari...@deliberate.com) | |
| Date: | Jan 19, 2005 9:13:17 pm | |
| List: | org.python.tutor | |
What about when I do an explicit call to a close inside a __del__. Is that a bad idea?
I usually prefer to add a close() method to my objects that releases resources, rather than __del__(), because it's more visible.
OK Danny! I found it! When I was almost asleep last night, I knew what to do and this morning I whittled it down to this situation. Maybe it can be whittled further but I'm holding up our project and I can't take any more time right now.
Note that the two variables TO_WHOM and MAILER, I think, can be safely changed to suit your environment, indeed MAILER='mail' should work for others.
It happens on 2.4 but not on 2.3. Either 2.3 masks my error, or 2.4 has an error?
I wouldn't be surprised if this is responsible for my thread problem but we're not going to find out.
---- #!/usr/bin/env python2.4 '''On my python2.4, this produces: ./test_exim.py close failed: [Errno 9] Bad file descriptor close failed: [Errno 9] Bad file descriptor close failed: [Errno 9] Bad file descriptor
but it runs just fine.
but if I specify python2.3, it runs cleanly.
'''
TO_WHOM = 'marilyn' MAILER = 'exim'
import popen2 import select import os
class Exim: def __init__(self): self.fdin = None self.err_flush = [] self.stdout, self.stdin, self.stderr = popen2.popen3('%s -t' % MAILER) self.fdin = self.stdin.fileno() self.fdout = self.stdout.fileno() self.fderr = self.stderr.fileno() self.outlist = [self.fdout, self.fderr] self.inlist = [self.fdin]
def __del__(self): self.close_up()
def close_up(self): if not self.fdin: return count = 0 in_status = os.close(self.fdin) while 1: sel = select.select(self.outlist, [], [], 1) if sel[0] == []: break else: for fd in sel[0]: r = os.read(fd, 16384) if r: self.err_flush += [r] count = 0 count += 1 if count >= 5: break else: continue break self.err_flush = ''.join(self.err_flush) out_status = os.close(self.fdout) err_status = os.close(self.fderr) self.fdin = None if not in_status: return raise RuntimeError, self.err_flush
def write(self, stuff): while 1: sel = select.select(self.outlist, self.inlist, [], 1) if sel[1] != []: os.write(sel[1][0], stuff) return
################################################################ if __name__ == '__main__': msg = '''To: %s
xx''' % TO_WHOM
p = Exim() p.write(msg) del p
----
On Tue, 18 Jan 2005, Danny Yoo wrote:
[About using the Standard Library]
And since then, please don't shoot me, but I don't immediately trust the modules. I read them and see how many times they loop through the data, and how many copies of the data they put into memory -- and usually decide to write the simple things I need myself, looping zero times and keeping only one block in memory.
Hmm.. . Do you remember which Standard Library modules you were looking at earlier? Perhaps there was some funky stuff happening, in which case we should try to fix it, so that no one else runs into the same problems.
Yes. poplib and socket. But, it's not fair to assume that something funky is happening in there. A poplib should provide a set of tools for making a pop client and I'm not making a pop client, just pumping the message from a pop server through my python/mysql magic and into exim. Similarly with socket. It does buffering and prepares things that I don't need, but probably lots of people do. So when I say I don't "trust" stdlib modules, I mean I don't trust them to be simple enough for my situation.
However, all that said, I do dimly remember that poplib perhaps had some extra processing that maybe is not needed -- but I could be wrong.
In a similar vein, in Spawn.run(), it might be a good idea to explicitely call write_and_close() on our FileSocket instance. For example, we can add a try/finally in the body of the revised start() method:
### def start(self): '''Given the command, provides the function to call.''' global TESTING function = { 'acl_rcpt.py' : calls.acl_rcpt, 'route_mail.py' : calls.route_mail, 'route_errors.py' : calls.route_errors, 'doorman.py' : calls.doorman, 'doorman_errors.py' : calls.doorman_errors, 'is_known_to.py' : is_known_to } if log.level & log.calls: log.it('fd%d: %s: %s', self.descriptor, self.exim_io.call, self.exim_io.collector_name) try: function[self.exim_io.call].main(self.exim_io) finally: self.exim_io.write_and_close() ###
By using try/finally, we can guarantee that finalizers like 'write_and_close()' can be called at the end.
I couldn't tell when write_and_close() was getting called in the original code; I suspected that each of the 'calls' dispatch functions was individually responsible for calling write_and_close(), but I wasn't sure.
Yes, the function[..] is responsible to write_and_close at the proper time in its processing, which varies from function to function. At this point we are very intimate with exim. Right now a failure to write_and_close causes mail to be deferred, which is good, but it does it ungracefully. So maybe I will fiddle there to make it better.
Best of wishes to you!
Thank you. Things have improved.
Are you well yet? A lot of sick people around these days!
M.
--





