[FIXED] How do I communicate with the docker daemon from inside a container using unix sockets?

Issue

A bit of background, I have a docker-compose based application with multiple services. Each service may have n number of instances. My inter-service communication strategy requires the redis service to be aware of the current state of the application, in terms of when there are new instances added, and when instances die or are removed.

From reading multiple blog posts and stack overflow questions, I'm aware that the solution involves communicating with the host docker daemon by binding the unix socket /var/run/docker.sock to a socket in the container, but I am unable to get any traction. Most resources I have come across give a fairly shallow explanation of what is going on, and there is definitely lacking any sort of ELI5 tutorial out there.

Currently, in my docker-compose.yml, I have the following config as part of one of my nodejs based services (no, it's not part of the redis service b/c I am just at a proof of concept stage at the moment)...

volumes:
  - /var/run/docker.sock:/var/run/docker.sock:ro

I've seen this exact snippet dozens of times in other posts and stack overflow questions, but the explanations usually end there.

In my nodejs/express service, I have an endpoint I created for the purpose of testing if my setup is working or not. It is using got by Sindre Sorhus for it's ability to work with unix sockets.

app.get('/dockersocket', async (req, res) => {
  const data = await got('unix:/var/run/docker.sock:/var/run/docker.sock')
  res.status(200).json({ data: data })
})

Needless to say, it does not work in it's current form. When I wrap the snippet above in a try/catch and console.log the error, I receive the output below...

{
  HTTPError: Response code 404(Not Found)
  at EventEmitter.emitter.on(/usr/src / app / node_modules / got / source / as - promise.js: 74: 19)
  at processTicksAndRejections(internal / process / next_tick.js: 81: 5)
  name: 'HTTPError',
  host: null,
  hostname: 'unix',
  method: 'GET',
  path: '/var/run/docker.sock',
  socketPath: '/var/run/docker.sock',
  protocol: 'http:',
  url: 'http://unix/var/run/docker.sock:/var/run/docker.sock',
  gotOptions: {
    path: '/var/run/docker.sock',
    protocol: 'http:',
    slashes: true,
    auth: null,
    host: null,
    port: null,
    hostname: 'unix',
    hash: null,
    search: null,
    query: null,
    pathname: '/var/run/docker.sock:/var/run/docker.sock',
    href: 'http://unix/var/run/docker.sock:/var/run/docker.sock',
    retry: {
      retries: [Function],
      methods: [Set],
      statusCodes: [Set],
      errorCodes: [Set]
    },
    headers: {
      'user-agent': 'got/9.6.0 (https://github.com/sindresorhus/got)',
      'accept-encoding': 'gzip, deflate'
    },
    hooks: {
      beforeRequest: [],
      beforeRedirect: [],
      beforeRetry: [],
      afterResponse: [],
      beforeError: [],
      init: []
    },
    decompress: true,
    throwHttpErrors: true,
    followRedirect: true,
    stream: false,
    form: false,
    json: false,
    cache: false,
    useElectronNet: false,
    socketPath: '/var/run/docker.sock',
    method: 'GET'
  },
  statusCode: 404,
  statusMessage: 'Not Found',
  headers: {
    'content-type': 'application/json',
    date: 'Sun, 31 Mar 2019 01:10:06 GMT',
    'content-length': '29',
    connection: 'close'
  },
  body: '{"message":"page not found"}\n'
}

Solution

The Docker daemon API can be communicated with using HTTP endpoints, and by default listens on a UNIX socket. That means you can communicate with it like any normal HTTP server, with just a bit of extra handling for when it's a socket.

You are getting an error because while you did send a request to the socket, you are requesting the wrong path. The syntax for a request is:

PROTOCOL://unix:SOCKET_PATH:ENDPOINT_PATH

For your code, that means:

const data = await got('unix:/var/run/docker.sock:/var/run/docker.sock')

// protocol      = http (default by library)
// socket path   = /var/run/docker.sock
// endpoint path = /var/run/docker.sock

To fix your issue, you should request a valid Docker Engine API endpoint (documentation for v1.39) as the HTTP path. Example to list containers:

await got('unix:/var/run/docker.sock:/containers/json')

If you have curl handy, you can test this from your shell:

$ curl --unix-socket /var/run/docker.sock http://localhost/containers/json

Answered By - hexacyanide

Answer Checked By - Clifford M. (PHPFixing Volunteer)