[FIXED] How to test dataclass that can be initialized with environment variables?

Issue

I have the following dataclass:

import os
import dataclasses

@dataclasses.dataclass
class Example:
    host: str = os.environ.get('SERVICE_HOST', 'localhost')
    port: str = os.environ.get('SERVICE_PORT', 30650)

How do I write a test for this? I tried the following which looks like it should work:

from stackoverflow import Example
import os


def test_example(monkeypatch):
    # GIVEN environment variables are set for host and port
    monkeypatch.setenv('SERVICE_HOST', 'server.example.com')
    monkeypatch.setenv('SERVICE_PORT', '12345')
    #   AND a class instance is initialized without specifying a host or port
    example = Example()
    # THEN the instance should reflect the host and port specified in the environment variables
    assert example.host == 'server.example.com'
    assert example.port == '12345'

but this fails with:

====================================================================== test session starts ======================================================================
platform linux -- Python 3.8.12, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/biogeek/tmp
collected 1 item                                                                                                                                                

test_example.py F                                                                                                                                         [100%]

=========================================================================== FAILURES ============================================================================
_________________________________________________________________________ test_example __________________________________________________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f39de559220>

    def test_example(monkeypatch):
        # GIVEN environment variables are set for host and port
        monkeypatch.setenv('SERVICE_HOST', 'server.example.com')
        monkeypatch.setenv('SERVICE_PORT', '12345')
        #   AND a class instance is initialized without specifying a host or port
        example = Example()
        # THEN the instance should reflect the host and port specified in the environment variables
>       assert example.host == 'server.example.com'
E       AssertionError: assert 'localhost' == 'server.example.com'
E         - server.example.com
E         + localhost

test_example.py:12: AssertionError
==================================================================== short test summary info ====================================================================
FAILED test_example.py::test_example - AssertionError: assert 'localhost' == 'server.example.com'
======================================================================= 1 failed in 0.05s =======================================================================

Solution

Your tests fail because your code loads the environment variables when you import the module. Module-level code is very hard to test, as the os.environ.get() calls to set the default values have already run before your test runs. You'd have to effectively delete your module from the sys.modules module cache, and only import your module after mocking out the os.environ environment variables to test what happens at import time.

You could instead use a dataclass.field() with a default_factory argument; that executes a callable to obtain the default whenever you create an instance of your dataclass:

import os
from dataclasses import dataclass, field
from functools import partial


@dataclass
class Example:
    host: str = field(default_factory=partial(os.environ.get, 'SERVICE_HOST', 'localhost'))
    port: str = field(default_factory=partial(os.environ.get, 'SERVICE_PORT', '30650'))

I used the functools.partial() object to create a callable that'll call os.environ.get() with the given name and default.

Note that I also changed the default value for SERVICE_PORT to a string; after all, the port field is annotated as a str, not an int. :-)

If you must set these defaults from environment variables at import time, then you could have pytest mock out these environment variables in a conftest.py module; these are imported before your tests are imported, so you get a chance to tweak things before your module-under-test is imported. This won't let you run multiple tests with different defaults, however:

# add to conftest.py at the same package level, or higher.

@pytest.fixture(autouse=True, scope="session")
def mock_environment(monkeypatch):
    monkeypatch.setenv('SERVICE_HOST', 'server.example.com')
    monkeypatch.setenv('SERVICE_PORT', '12345')

The above fixture example, when placed in conftest.py, would automatically patch your environment before your tests are loaded, and so before your module is imported, and this patch is automatically undone at the end of the test session.

Answered By - Martijn Pieters

Answer Checked By - Mildred Charles (PHPFixing Admin)