[FIXED] How to use Ingress Nginx Controller to route traffic to private pods Internally

Issue

Problem: I am currently using ingress-nginx in my EKS cluster to route traffic to services that need public access.

My use case: I have services I want to deploy in the same cluster but don't want them to have public access. I only want the pods to communicate will all other services within the cluster. Those pods are meant to be private because they're backend services and only need pod-to-pod communication. How do I modify my ingress resource for this purpose?

Cluster Architecture: All services are in the private subnets of the cluster while the load-balancer is in the public subnets

Additional note: I am using external-dns to dynamically create the subdomains for the hosted zones. The hosted zone is public

Thanks

Below are my service.yml and ingress.yml for public services. I want to modify these files for private services

service.yml

apiVersion: v1
kind: Service
metadata:
  name: myapp
  namespace: myapp 
  annotations:
    external-dns.alpha.kubernetes.io/hostname: myapp.dev.com
spec:
  ports:
    - port: 80
      targetPort: 3000
  selector:
    app: myapp

ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp
  namespace: myapp 
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: "nginx"
  labels:
    app: myapp
spec:
  tls:
  - hosts:
  - myapp.dev.com
  secretName: myapp-staging
  rules:
  - host: myapp.dev.com
    http:
      paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: 'myapp'
              port:
                number: 80

Solution

From this what you have the Ingress already should work and your services are meant to be private(if you set like this in your public cloud cluster), except the Ingress itself. You can update the ConfigMap to use the PROXY protocol so that you can pass proxy information to the Ingress Controller:

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  proxy-protocol: "True"
  real-ip-header: "proxy_protocol"
  set-real-ip-from: "0.0.0.0/0"

And then: kubectl apply -f common/nginx-config.yaml

Now you can deploy any app that you want to have private with the name specified (for example your myapp Service in your yaml file provided.

If you are a new to Kubernetes Networking, then this article would be useful for you or in official Kubernetes documentation

Here you can find other ELB annotations that may be useful for you

Answered By - Bazhikov

Answer Checked By - Marilyn (PHPFixing Volunteer)