[FIXED] How can I store composer cache in a volume during when building a docker?

Issue

I am using this Dockerfile to create an image with my PHP application dependencies:

FROM composer:latest AS composer

COPY ./ ./

VOLUME composer-cache:/tmp/

RUN composer install

FROM php:7.1-fpm

WORKDIR /app

RUN apt-get update && \
    apt-get install -y libpq5 libpq-dev libmemcached11 libmemcached-dev libmagickwand-6.q16-6 libmagickwand-dev unzip --no-install-recommends && \
    docker-php-ext-install pdo pdo_pgsql && \
    pecl install imagick && \
    echo extension=imagick.so >> /usr/local/etc/php/conf.d/imagick.ini && \
    apt-get remove -y libpq-dev libmemcached-dev libmagickwand-dev && \
    apt-get autoremove -y && \
    apt-get autoclean -y

RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"

COPY ./ ./

COPY --from=composer /app/vendor ./vendor

For faster build and reducing network activity I want to store composer's cache in the volume called composer-cache. But composer does not store anything data in that volume.

I checked a official composer Dockerfile and noticed it's exports COMPOSER_HOME environment variable points to /tmp directory. I think is enough to store cache outside from container, but this is not.

On every build composer downloads all dependencies again, and the volume remains empty.

Solution

Your problem is that the directive VOLUME creates a volume mountpoint to be used at runtime.

That volume is actually created when you execute docker run. During build time (docker build), that instruction has no effect.

But you can use new build time enhancements added for this kind of thing.

You need to add the following to your Dockerfile:

# syntax=docker/dockerfile:experimental
FROM composer:latest AS composer

COPY composer.json composer.lock ./

RUN --mount=type=cache,target=/tmp/cache composer install --prefer-dist --no-suggest

### your next build stage...

This will result in speedier install execution after the first time.

To gain download speed (which may make the above kinda relatively moot), you can use Prestssimo. This is a composer plugin that makes that composer download are performed in parallel instead of sequentially.

(Note, with Composer 2 Prestissimo is obsolete, since composer itself performs this type of download optimisation).

You'd end up with

# syntax=docker/dockerfile:experimental
FROM composer:latest AS composer
RUN composer global require hirak/prestissimo

COPY composer.json composer.lock ./

RUN --mount=type=cache,target=/tmp/cache composer install --prefer-dist  --no-suggest

### your next build stage...

You would then call your build process like this:

DOCKER_BUILDKIT=1 docker build .

It is not currently possible to use docker-compose to enable buildkit. But with a bit of foreplanning you can build your images first, and declare those images in your docker-compose files for orchestration.

I find that simply using Prestissimo is enough as not to need additional cache layers, but you'll need to check what's best for your scenario.

Answered By - yivi