[FIXED] How to solve "OSError: telling position disabled by next() call"

Issue

I am creating a file editing system and would like to make a line based tell() function instead of a byte based one. This function would be used inside of a "with loop" with the open(file) call. This function is part of a class that has:

self.f = open(self.file, 'a+')
# self.file is a string that has the filename in it

The following is the original function (It also has a char setting if you wanted line and byte return):

def tell(self, char=False):
    t, lc = self.f.tell(), 0
    self.f.seek(0)
    for line in self.f:
        if t >= len(line):
            t -= len(line)
            lc += 1
        else:
            break
    if char:
        return lc, t
    return lc

The problem I'm having with this is that this returns an OSError and it has to do with how the system is iterating over the file but I don't understand the issue. Thanks to anyone who can help.

Solution

I have an older version of Python 3, and I'm on Linux instead of a Mac, but I was able to recreate something very close to your error:

IOError: telling position disabled by next() call

An IO error, not an OS error, but otherwise the same. Bizarrely enough, I couldn't cause it using your open('a+', ...), but only when opening the file in read mode: open('r+', ...).

Further muddling things is that the error comes from _io.TextIOWrapper, a class that appears to be defined in Python's _pyio.py file... I stress "appears", because:

1. The TextIOWrapper in that file has attributes like _telling that I can't access on the whatever-it-is object calling itself _io.TextIOWrapper.

2. The TextIOWrapper class in _pyio.py doesn't make any distinction between readable, writable, or random-access files. Either both should work, or both should raise the same IOError.

Regardless, the TextIOWrapper class as described in the _pyio.py file disables the tell method while the iteration is in progress. This seems to be what you're running into (comments are mine):

def __next__(self):
    # Disable the tell method.
    self._telling = False
    line = self.readline()
    if not line:
        # We've reached the end of the file...
        self._snapshot = None
        # ...so restore _telling to whatever it was.
        self._telling = self._seekable
        raise StopIteration
    return line

In your tell method, you almost always break out of the iteration before it reaches the end of the file, leaving _telling disabled (False):

One other way to reset _telling is the flush method, but it also failed if called while the iteration was in progress:

IOError: can't reconstruct logical file position

The way around this, at least on my system, is to call seek(0) on the TextIOWrapper, which restores everything to a known state (and successfully calls flush in the bargain):

def tell(self, char=False):
    t, lc = self.f.tell(), 0
    self.f.seek(0)
    for line in self.f:
        if t >= len(line):
            t -= len(line)
            lc += 1
        else:
            break
    # Reset the file iterator, or later calls to f.tell will
    # raise an IOError or OSError:
    f.seek(0)
    if char:
        return lc, t
    return lc

If that's not the solution for your system, it might at least tell you where to start looking.

PS: You should consider always returning both the line number and the character offset. Functions that can return completely different types are hard to deal with --- it's a lot easier for the caller to just throw away the value her or she doesn't need.

Answered By - Kevin J. Chase

Answer Checked By - Timothy Miller (PHPFixing Admin)