[FIXED] Why is a list variable sometimes not impacted by changes in function as I thought python3 works on pass by reference with list variables?

Issue

For python3, I originally needed to extract odd and even positions from a list and assign it to new lists, then clear the original list. I thought lists were impacted by a function call through "pass by reference". Testing some scenarios, it works sometime. Could someone please explain how exactly python3 works here?

Case 1: empty list is populated with string as expected.

def func1(_in):
    _in.append('abc')

mylist = list()
print(f"Before:\nmylist = {mylist}")
func1(mylist)
print(f"After:\nmylist = {mylist}")

Output case 1:

Before:
mylist = []
After:
mylist = ['abc']

Case 2: middle list element is replaced with string as expected.

def func2(_in):
    _in[1] = 'abc'

mylist = list(range(3))
print(f"Before:\nmylist = {mylist}")
func2(mylist)
print(f"After:\nmylist = {mylist}")

Output case 2:

Before:
mylist = [0, 1, 2]
After:
mylist = [0, 'abc', 2]

Case 3: why is the list not empty after function call?

def func3(_in):
    _in = list()

mylist = list(range(3))
print(f"Before:\nmylist = {mylist}")
func3(mylist)
print(f"After:\nmylist = {mylist}")

Output case 3:

Before:
mylist = [0, 1, 2]
After:
mylist = [0, 1, 2]

Case 4: working exactly as expected, but note I have returned all three lists from function.

def func4_with_ret(_src, _dest1, _dest2):
    _dest1 = [val for val in _src[0:len(_src):2]]
    _dest2 = [val for val in _src[1:len(_src):2]]
    _src = list()
    return _src, _dest1, _dest2

source = list(range(6))
evens, odds = list(), list()
print(f"Before function call:\nsource = {source}\nevens = {evens}\nodds = {odds}")
source, evens, odds = func4_with_ret(source, evens, odds)
print(f"\nAfter function call:\nsource = {source}\nevens = {evens}\nodds = {odds}")

Output case 4:

Before function call:
source = [0, 1, 2, 3, 4, 5]
evens = []
odds = []

After function call:
source = []
evens = [0, 2, 4]
odds = [1, 3, 5]

Case 5: why no impact on the variables outside the function if I do not explicitly return from function call?

def func5_no_ret(_src, _dest1, _dest2):
    _dest1 = [val for val in _src[0:len(_src):2]]
    _dest2 = [val for val in _src[1:len(_src):2]]
    _src = list()

source = list(range(6))
evens, odds = list(), list()
print(f"Before function call:\nsource = {source}\nevens = {evens}\nodds = {odds}")
func5_no_ret(source, evens, odds)
print(f"\nAfter function call:\nsource = {source}\nevens = {evens}\nodds = {odds}")

Output case 5:

Before function call:
source = [0, 1, 2, 3, 4, 5]
evens = []
odds = []

After function call:
source = [0, 1, 2, 3, 4, 5]
evens = []
odds = []

Thank you.

Solution

Your ultimate problem is confusing (in-place) mutation with rebinding (also referred to somewhat less precisely as "reassignment").

In all the cases where the change isn't visible outside the function, you rebound the name inside the function. When you do:

name = val

it does not matter what used to be in name; it's rebound to val, and the reference to the old object is thrown away. When it's the last reference, this leads to the object being cleaned up; in your case, the argument used to alias an object also bound to a name in the caller, but after rebinding, that aliasing association is lost.

Aside for C/C++ folks: Rebinding is like assigning to a pointer variable, e.g. int *px = pfoo; (initial binding), followed later by px = pbar; (rebinding), where both pfoo and pbar are themselves pointers to int. When the px = pbar; assignment occurs, it doesn't matter that px used to point to the same thing as pfoo, it points to something new now, and following it up with *px = 1; (mutation, not rebinding) only affects whatever pbar points to, leaving the target of pfoo unchanged.

By contrast, mutation doesn't break aliasing associations, so:

name[1] = val

does rebind name[1] itself, but it doesn't rebind name; it continues to refer to the same object as before, it just mutates that object in place, leaving all aliasing intact (so all names aliasing the same object see the result of the change).

For your specific case, you could change the "broken" functions from rebinding to aliasing by changing to slice assignment/deletion or other forms of in-place mutation, e.g.:

def func3(_in):
    # _in = list()  BAD, rebinds
    _in.clear()     # Good, method mutates in place
    del _in[:]      # Good, equivalent to clear
    _in[:] = list() # Acceptable; needlessly creates empty list, but closest to original
                    # code, and has same effect

def func5_no_ret(_src, _dest1, _dest2):
    # BAD, all rebinding to new lists, not changing contents of original lists
    #_dest1 = [val for val in _src[0:len(_src):2]]
    #_dest2 = [val for val in _src[1:len(_src):2]]
    #_src = list()

    # Acceptable (you should just use multiple return values, not modify caller arguments)
    # this isn't C where multiple returns are a PITA
    _dest1[:] = _src[::2]  # Removed slice components where defaults equivalent
    _dest2[:] = _src[1::2] # and dropped pointless listcomp; if _src might not be a list
                           # list(_src[::2]) is still better than no-op listcomp
    _src.clear()

    # Best (though clearing _src is still weird)
    retval = _src[::2], _src[1::2]
    _src.clear()
    return retval

    # Perhaps overly clever to avoid named temporary:
    try:
        return _src[::2], _src[1::2]
    finally:
        _src.clear()

Answered By - ShadowRanger

Answer Checked By - David Goodson (PHPFixing Volunteer)