[FIXED] How to change a specific variable from True to False in a docker image via new Dockerfile?

Issue

Goal

• I want to change one PIP_USER image variable from True to False
• PIP_USER is not in the Original Dockerfile but it is in the official image's 48th image layer that was built.
• I would like to use the official latest Docker Airflow 2.4.1 image
• I would like to pull than modify the official image via my Dockerfile
• Reason if I can Flip the True to False
  • I can add multiple of my own python virtual environments
  • Install all my python packages to each python virtual environemnt
  • via pip and a requirements.txt
  • I need this because a ExternalPythonOperator feature available since 19 OCT. 2022. = Airflow 2.4.0
  • https://airflow.apache.org/docs/docker-stack/build.html#important-notes-for-the-base-images "Only as of 2.0.1 image the --user flag is turned on by default by setting PIP_USER environment variable to true. This can be disabled by un-setting the variable or by setting it to false. In the 2.0.0 image you had to add the --user flag as pip install --user command."

Situation

• I am using the latest Airflow Docker Image
  • Dockerfile https://hub.docker.com/r/apache/airflow/Dockerfile
  • Image - 48th image layer where I want to do the modification - https://hub.docker.com/layers/apache/airflow/latest/images/sha256-5015db92023bebb1e8518767bfa2e465b2f52270aca6a9cdef85d5d3e216d015?context=explore
• Ubuntu 20.04 LTS
• Python 3.8
• Airflow 2.4.1

OFFICIAL Airflow Docker IMAGE

• to be edited after it gets pulled
• 48th image layer where I want to do the modification - https://hub.docker.com/layers/apache/airflow/latest/images/sha256-5015db92023bebb1e8518767bfa2e465b2f52270aca6a9cdef85d5d3e216d015?context=explore

ENV DUMB_INIT_SETSID=1 PS1=(airflow) AIRFLOW_VERSION=2.4.1 AIRFLOW__CORE__LOAD_EXAMPLES=false 
PIP_USER=true 
PATH=/root/bin:/home/airflow/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

My Dockerfile

that should modify the official image

FROM apache/airflow:2.4.1-python3.8
USER root
RUN python3 -m venv /opt/airflow/venv1

# Install dependencies:
COPY requirements.txt .

#RUN usermod -g 0 root
RUN /opt/airflow/venv1/bin/pip install --user -r requirements.txt
USER airflow

Terminal Command

docker build -t my-image-apache/airflow:2.4.1 .

ERROR Message

Sending build context to Docker daemon  1.902GB
Step 1/4 : FROM apache/airflow:2.4.1-python3.8
 ---> 836b925604e4
Step 2/4 : RUN python3 -m venv /opt/airflow/venv1
 ---> Running in e49018b06862
Removing intermediate container e49018b06862
 ---> 4c98f8cc54a8
Step 3/4 : COPY requirements.txt .
 ---> c0636051a086
Step 4/4 : RUN /opt/airflow/venv1/bin/pip install --user -r requirements.txt
 ---> Running in bb0a4e49d77b
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/opt/airflow/venv1/bin/python3 -m pip install --upgrade pip' command.
The command '/bin/bash -o pipefail -o errexit -o nounset -o nolog -c /opt/airflow/venv1/bin/pip install --user -r requirements.txt' returned a non-zero code: 1

Tried

Dockerfile

FROM apache/airflow:2.4.1-python3.8
env PIP_USER=false
RUN python3 -m venv /opt/airflow/venv1

# Install dependencies:
COPY requirements.txt .

RUN /opt/airflow/venv1/bin/pip install --user -r requirements.txt

Terminal Command

docker build -t my-image-apache/airflow:2.4.1 .

ERROR Message

Sending build context to Docker daemon  1.902GB
Step 1/5 : FROM apache/airflow:2.4.1-python3.8
 ---> 836b925604e4
Step 2/5 : env PIP_USER=false
 ---> Running in 6c840cad848f
Removing intermediate container 6c840cad848f
 ---> b483c5f9f786
Step 3/5 : RUN python3 -m venv /opt/airflow/venv1
 ---> Running in c39cf0c2bb03
Removing intermediate container c39cf0c2bb03
 ---> 2fb03b6a8b20
Step 4/5 : COPY requirements.txt .
 ---> 30a537975b97
Step 5/5 : RUN /opt/airflow/venv1/bin/pip install --user -r requirements.txt
 ---> Running in 68266dfc9d50
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/opt/airflow/venv1/bin/python3 -m pip install --upgrade pip' command.
The command '/bin/bash -o pipefail -o errexit -o nounset -o nolog -c /opt/airflow/venv1/bin/pip install --user -r requirements.txt' returned a non-zero code: 1

Solution

2 Single line answer:

ENV PIP_USER=false
...
RUN /opt/airflow/venv1/bin/pip install -r requirements.txt

In context answer:

--user have to be taken out

FROM apache/airflow:2.4.1-python3.8

env PIP_USER=false

RUN python3 -m venv /opt/airflow/venv1

# Install dependencies:
COPY requirements.txt .

RUN /opt/airflow/venv1/bin/pip install -r requirements.txt

Answered By - sogu

Answer Checked By - Willingham (PHPFixing Volunteer)