[FIXED] How to get to postgres database at localhost from Django in Docker container

Issue

I have a django in a docker container that must access a postgres database at my localhost. The Dockerfile works fine when accessing the database residing at an external host, but it can't find the database at my host.

It is a well known problem and there is a lot of documentation, but it doesn't work in my case. This question resembles another question but it did not solve my problem. I actually describe the correct solution of this problem as @Zeitounator pointed out, but it still did not work. It was thanks @Zeitounator that I realised two parts of the problem must be solved: the docker side and the PostgreSQL side. I did not find that solution in any of the answers I read. I did however read about the same frustration: getting a solution that did not work.

It all focuses on which internet address I transmit to the HOST key in the database driver dictionary in the django settings.py:

print('***', os.environ['POSTGRES_DB'], os.environ['POSTGRES_USER'],
        os.environ['POSTGRES_HOST'], os.environ['POSTGRES_PORT'])

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2', # os.environ['ENGINE'],
        'NAME': 'demo',
        'USER': os.environ['POSTGRES_USER'],
        'PASSWORD': os.environ['POSTGRES_PASSWORD'],
        'HOST': os.environ['POSTGRES_HOST'],
    }
}

And running the Dockerfile:

docker build -t dd .
docker run --name demo -p 8000:8000 --rm dd

When POSTGRES_HOST points to my external server 192.168.178.100 it works great. When running python manage.py runserver it finds the host and the database. The server starts and waits for commands. When pointing to 127.0.0.1 it fails (which actually is great too: the container really isolates).

django.db.utils.OperationalError: could not connect to server: Connection refused
    Is the server running on host "127.0.0.1" and accepting
    TCP/IP connections on port 5432?

But as I can connect to my server, I should be able to connect to the localhost IP and that fails just as well (I forgot to tell that the database really runs). When I use host.docker.internal it doesn't work either when running:

docker run --name demo -p 8000:8000 --rm --add-host host.docker.internal:host-gateway dd

It replaces host.docker.internal by 172.17.0.1. The only solution that works so far is:

# set POSTGRES_HOST at 127.0.0.1
docker run --name demo -p 8000:8000 --rm --network host dd

but that seems something I don't want. As far as I understand the documentation it makes the full host stack available to the container which defeats the idea of a container in the first place. But second: it doesn't work in docker-compose, though I specify:

extra_hosts:
  - "host.docker.internal:host-gateway"

network_mode: host leads to a compile error, docker-compose refuses to run.

Is there a way to access a service running on my PC from a docker running on my PC as well from Dockerfile as well as docker-compose and can be deployed as well?

using Ubuntu 21.04 (latest version), Docker version 20.10.7, build 20.10.7-0ubuntu5.1, postgresql v14. My Dockerfile:

FROM python:3

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN mkdir /app

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD ["python", "manage.py", "runserver", "127.0.0.1:8000"]

Solution

There are two parts in solving this problem: the docker part and the postgresql part as @Zeitounator pointed out. I was not aware of the postgres part. Thanks to his comment I could resolve this issue. And it works for Dockerfile as well as docker-compose where this Dockerfile is used.

One has to change two postgres configuration files, both are on /etc/postgresql/<version>/main:

postgresql.conf

Change the listen address. Initially it shows:

listen_addresses = 'localhost'      # what IP address(es) to listen on;
                    # comma-separated list of addresses;
                    # defaults to 'localhost'; use '*' for all
                    # (change requires restart)

I changed 'localhost' to '*'. It can be more specific, in this case to the docker address, but as a Proof of Concept it worked.

pg_hba.conf

In my case host.docker.internal resolves to 172.17.0.1. This seems some default docker gateway address as I noticed in most discussion regarding this subject. Add two lines at the very end of the file:

host    all             all              172.17.0.1/0                       md5
host    all             all              ::ffff:ac11:1/0                            md5

Answered By - Arnold

Answer Checked By - Senaida (PHPFixing Volunteer)