1> (len
(with-stream (s (open-file "/usr/share/dict/words"))
(get-lines s)))
** error reading #<file-stream /usr/share/dict/words b7ad7270>: file closed
** during evaluation of form (len (let ((s (open-file "/usr/share/dict/words")))
(unwind-protect
(get-lines s)
(close-stream s))))
** ... an expansion of (len (with-stream
(s (open-file "/usr/share/dict/words"))
(get-lines s)))
** which is located at expr-1:1
The built-in solution is that when you create a lazy list which reads lines from a stream, that lazy list takes care of closing the stream when it is done.
If the lazy list isn't processed to the end, then the stream semantically leaks; it has to be cleaned up by the garbage collector when the lazy list becomes unreachable.
It is possible to address the error issue with reference counting. Suppose that we define a stream with a reference count, such that it has to be closed that many times before the underlying file descriptor is closed.
I programmed a proof of concept of this today. (I ran into a small issue in the language run-time that I fixed; the close-stream function calls the underlying method and then caches the result, preventing the solution from working.)
(defstruct refcount-close stream-wrap
stream
(count 1)
(:method close (me throw-on-error-p)
(put-line `close called on @me`)
(when (plusp me.count)
(if (zerop (dec me.count))
(close-stream me.stream throw-on-error-p)))))
(flow
(with-stream (s (make-struct-delegate-stream
(new refcount-close
count 2
stream (open-file "/usr/share/dict/words"))))
(get-lines s))
len
prinl)
With my small fix in stream.c (already merged, going into Version 292), the output is:
$ ./txr lazy2.tl
close called on #S(refcount-close stream #<file-stream /usr/share/dict/words b7aecee0> count 2)
close called on #S(refcount-close stream #<file-stream /usr/share/dict/words b7aecee0> count 1)
102305
One close comes from the with-stream macro, the other from the lazy list hitting EOF when its length is being calculated.
Without the fix, I don't get the second call; the code works, but the descriptor isn't closed:
$ txr lazy2.tl
close called on #S(refcount-close stream #<file-stream /usr/share/dict/words b7b70f10> count 2)
102305
In the former we see the call to close in strace; in the latter we don't.
Comments
TXR Lisp also fails this test:
The built-in solution is that when you create a lazy list which reads lines from a stream, that lazy list takes care of closing the stream when it is done.If the lazy list isn't processed to the end, then the stream semantically leaks; it has to be cleaned up by the garbage collector when the lazy list becomes unreachable.
We can see with strace that the stream is closed:
It is possible to address the error issue with reference counting. Suppose that we define a stream with a reference count, such that it has to be closed that many times before the underlying file descriptor is closed.I programmed a proof of concept of this today. (I ran into a small issue in the language run-time that I fixed; the close-stream function calls the underlying method and then caches the result, preventing the solution from working.)
With my small fix in stream.c (already merged, going into Version 292), the output is: One close comes from the with-stream macro, the other from the lazy list hitting EOF when its length is being calculated.Without the fix, I don't get the second call; the code works, but the descriptor isn't closed:
In the former we see the call to close in strace; in the latter we don't.