[FIXED] How to use an assignment expression (:=) with pandas.DataFrame.apply

Issue

I have the following minimal working example (specific to python >= 3.8), which converts a file name string into a full path:

# running this block will produce the expected output
import pandas as pd
from pathlib import Path

def make_path(filename):
    f = filename.split('_')
    return directory / f[-2][:4] / '_'.join(f[:3]) / filename

directory = Path('/ifs/archive/ops/hst/public')

data = {'productFileName': ['hst_15212_ad_wfc3_ir_total_idq2ad_segment-cat.ecsv',
                            'hst_15212_ad_wfc3_ir_total_idq2ad_point-cat.ecsv',
                            'hst_15212_bt_wfc3_ir_total_idq2bt_segment-cat.ecsv',
                            'hst_15212_bt_wfc3_ir_total_idq2bt_point-cat.ecsv',
                            'hst_15212_4g_wfc3_ir_f160w_idq24g_point-cat.ecsv']}
dfx = pd.DataFrame(data)

dfx['filePath'] = dfx.productFileName.apply(make_path)

How can this be done using an assignment expression (:=) inside the .apply(...)?

Something along the lines of the following:

dfx['filePath'] = dfx.productFileName.apply(lambda filename: directory / f[-2][:4] / '_'.join(f[:3]) / filename for (f := filename.split('_')))

which currently results in:

  File "/tmp/ipykernel_3834754/3286169981.py", line 1
    dfx['filePath'] = dfx.productFileName.apply(lambda filename: directory / f[-2][:4] / '_'.join(f[:3]) / filename for (f := filename.split('_')))
                                                                                                                         ^
SyntaxError: cannot assign to named expression

Solution

You should do the assignment at the first usage:

dfx['filePath'] = dfx.productFileName.apply(lambda filename: directory / (f := filename.split('_'))[-2][:4] / '_'.join(f[:3]) / filename)

Answered By - Stef

Answer Checked By - Timothy Miller (PHPFixing Admin)