Entry

Is it possible to have a list that automatically extends when I assign to a non-existant element?

Feb 12th, 2007 10:50
stephen brown, Ed Parcell,

You need to derive a class from the list base class (or UserList in
older versions of Python) and override, at least, the __setitem__()
method, which is called when assigning to list elements.  The overriding
method will extend the list for indices out of range, then call the base
class methods to actually modify the underlying list.  Thus
class xList(list):
    def __setitem__(self, idx, val):
        if idx>=len(self):
            self.extend([None]*(idx-len(self)+1))
        list.__setitem__(self, idx, val)
>>> a = xList(['a', 'b', 'c'])
>>> a
['a', 'b', 'c']
>>> a[7] = 'h'
>>> a
['a', 'b', 'c', None, None, None, None, 'h']
>>> 
Note the explicit call to the base classes method, without it we could
never modify the list itself.  Implementing xList.__setslice__() is a
little more complicated and depends on how you want this notation to
behave--in the base list class, it is not an error to use out-of-range
slices indices, i.e.
>>> b = [1, 2, 3]
>>> b[11:12] # not an error
[]
>>> b[11] # error!
Traceback (most recent call last):
  File "<pyshell#45>", line 1, in ?
    b[11] # error!
IndexError: list index out of range
>>> b[11:12] = 'a' # not intuitive behaviour, IMO
>>> b[11:12] == 'a' # not true, even though it was just assigned
False
>>> b
[1, 2, 3, 'a']
>>> 
You'll have to decide how you want it to behave.
The caveat in doing this properly is understanding how the base class'
methods depend on each other.  For instance, overriding __setslice__()
in xList may break the code above, because xList.extend() (which is
inherited from list, so it's the same as xlist.extend()) may depend on
__setslice__().  If xList overrides __setslice__(), but calls extend(),
you can get an infinite loop which isn't obvious to find.
{Example:
class class1:
    def showme(self):
        return 'Fred'
    def __str__(self):
        return self.showme()
class class2(class1):
    def showme(self):
        return self.__str__()
>>> a = class1()
>>> print a
Fred
>>> b = class2()
>>> print b
    .
    . (much red ink)
    .
RuntimeError: maximum recursion depth exceeded
}
Depending on your needs, you may also want to override __getitem__() and
__getslice__() or use a "default" value different from None.  Special
method names for overriding syntax like this are documented in section
3.4 of the Python Reference Manual
http://docs.python.org/ref/specialnames.html
Note that this class has perils--it would be really easy to consume a
huge amount of memory unproductively.  If what you really want is a
sparse array class, that doesn't actually fill all the gaps until
they're used, you'll want a different implementation.  (Hint: Python
dictionaries with integer keys are sparse arrays.)