[FIXED] How to use the same nginx.conf file for reverse proxy for docker-compose and kubernetes

Issue

in Kube, i have one pod with two containers * container 1: nginx reverse proxy * container 2: myapp

for testing purpose, i also has a docker compose file, include the two services * service 1 nginx reverse proxy * service 2: myapp

the issue is, in docker, the nginx upstream host is in the format of container name. In Kube, it is localhost. Here is a code snipt:

//for docker, nginx.conf
...
   upstream  web{
        server myapp:8080;
    }
....
            proxy_pass         http://web;

//for Kube, nginx.conf


   ...
    upstream  web{
        server localhost:8080;
    }
    ....

            proxy_pass         http://web;
    }

i would like to have one nginx.conf to support both kube and docker-compose. one way i can thinkof is to pass an env run time variable, so i can sed the upstream host in the entrypoint.sh.

are there other ways to accomplish this?

thank you

Solution

You need to do two things.

First, split this up into two pods. (As a general rule, you should have one container in a pod; the typical exceptions to this are things like logging and networking "sidecar" containers that need to share filesystem and network space with the main container.) This probably means taking your existing pod spec (or better deployment spec), taking all of the stuff around the containers: block and making a second copy of it, and putting one container in each.

You need to make sure each of the pods has a distinct label (if you're using a deployment it's the label inside the pod template that matters); this might look something like

metadata:
  name: web
  labels:
    app: web

Second, you need to create a Kubernetes service that points at the "web" pod. This matches on the labels we just set

metadata:
  name: web
spec:
  selector:
    app: web

Now the name of the service will result in a DNS name web.default.svc.cluster.local existing (where "default" is the Kubernetes namespace name). default.svc.cluster.local gets set as a default DNS search domain, so web will resolve to the service will forward to the pod.

The Kubernetes documentation has a more complete example of this sort of thing (using PHP and nginx, but the only code is Kubernetes YAML manifests, so it should be pretty applicable).

Answered By - David Maze

Answer Checked By - David Goodson (PHPFixing Volunteer)