[FIXED] How to commit and push job artifact to a specified directory in Gitlab?

Issue

Hi all,

I have a 2 stage pipeline on gitlab-ci.yml file. The first job generates an artifact as an asd.asd file. The second stage uses this artifact. If pipeline starts from the first stage then the second one can use the first ones artifact. But in some cases I run only the second stage without running the first stage. So, I need to commit and push to the master the artifact of last successful running of the first stage. How can I do it in gitlab-ci.yml file?

stages:
  - first
  - second

job1:
  stage: first
  tags: 
    - asdasd
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"
    - XYZ.sh
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA"
    paths: 
     - folder1/asd.asd

#here the asd.asd artifact should be commited into folder1, how?

job2:
  stage: second
  tags: 
    - asdasd
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"
    - run.sh
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA"
    paths: 
     - folder1/asd.elf

Thanks, M.Altay

Solution

So, I need to commit and push to the master the artifact of last successful running of the first stage

In order to solve this let's break it in to components. To get the last successful execution of the stage in your case first, we will use the Gitlab API in combination with jq, to get the last successful pipeline as described in here https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines

GET /projects/:id/pipelines

We will use this API to get the id of the last successful pipeline for a specific project, with the following command

PIPELINE_ID=$(curl -s --header "PRIVATE-TOKEN: <access_token>"  "https://gitlab.com/api/v4/projects/<project_id>/pipelines?status=success" | jq '.[0].id')

Next, we will use the pipeline id to fetch the last successful job, from a specific stage in your case first. To achieve this we will use the Gitlab API https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs

GET /projects/:id/pipelines

Command:

JOB_ID=$(curl -s --header "PRIVATE-TOKEN: <access_token>"  "https://gitlab.com/api/v4/projects/<project_id>/pipelines/$PIPELINE_ID/jobs?scope=success" | jq '.[] | select(.stage=="first") | .id')

Fetct the artifact https://docs.gitlab.com/ee/api/job_artifacts.html#get-job-artifacts

GET /projects/:id/jobs/:job_id/artifacts

wget -U "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.17 (KHTML,like Gecko) Ubuntu/11.04 Chromium/11.0.654.0 Chrome/11.0.654.0 Safari/534.17" --header "PRIVATE-TOKEN: <access_token>" "https://gitlab.com/api/v4/projects/<project_id>/jobs/$JOB_ID/artifacts" -O artifacts.zip
unzip artifacts.zip

Finally commit it with the commit API by passing to content property the path to unzipped file https://docs.gitlab.com/ee/api/commits.html#create-a-commit-with-multiple-files-and-actions

POST /projects/:id/repository/commits

curl -XPOST  --header "PRIVATE-TOKEN: access_token"  https://gitlab.com/api/v4/projects/<project_id>/repository/commits --form "branch=<target_branch>"  --form "commit_message=some commit message" --form "start_branch=master" --form "actions[][action]=update" --form "actions[][file_path]=folder1/asd.asd"  --form "actions[][content]=<path/to/the/unziped/file"

Answered By - Tolis Gerodimos

Answer Checked By - Terry (PHPFixing Volunteer)