How I Cut Our Deploy Time from 45 Minutes to Under 8

The problem was simple: our GitHub Actions pipeline was slow, and every deploy paid for it. We were not dealing with one broken step. We were dealing with a system that kept repeating the same work, waiting on stale jobs, and running checks that did not need to happen for a small change. To speed it up, I had to look at the workflow as a system and fix the bottlenecks that were compounding every day.

We eventually brought deploy time from 45 minutes down to under 8. The win was not magic; it was careful measurement, selective caching, better job boundaries, and a lot of small changes that removed wasted work. No single setting did it. Below is what I changed, in the order it paid off, with the YAML for each step so you can apply it to your own workflows.

Part of my guide to deploying Node.js to production.

TL;DR#

• Dependency caching cut the biggest constant cost in the workflow.
• Docker layer caching removed a lot of repeated image build work.
• Matrix jobs and sharded tests shortened the critical path.
• Path filters prevented irrelevant PRs from triggering the full deploy flow.
• Concurrency groups canceled stale runs and reduced queue waste.
• Reusable workflows and artifact reuse made the pipeline easier to maintain and faster to run.
• AI helped me investigate logs and refactor YAML, but human review still mattered.

The Problem: Why Our Pipeline Took 45 Minutes#

The baseline workflow looked normal at first glance:

• install dependencies
• build the app
• run unit tests
• run integration tests
• build Docker images
• package artifacts
• deploy to staging or production

That structure is not the problem. The problem was that each stage repeated work, the queue filled up with stale runs, and the critical path was too long. We were paying for the same dependency installs, the same Docker layers, and the same validation work across multiple pushes.

The warning signs were obvious:

• every push triggered the full pipeline, even when the change was small
• dependency installs repeated the same work every time
• image builds rebuilt layers that had not changed
• tests ran in one long serial chain
• stale runs kept waiting behind newer ones in the queue

This is where GitHub Actions optimization becomes practical. It is not just about swapping a runner or turning on a random cache. It is about removing the repeated work that slows a team down and makes the pipeline feel unreliable.

I measured the baseline with the workflow run UI, step timings, and log review. The biggest bottlenecks were not surprising:

1. dependency downloads were slow and repeated
2. Docker layer builds were repeated on every branch run
3. the test suite was too long to be the single critical path
4. stale workflows kept consuming queue time

That is the pattern behind most slow CI/CD pipeline bottlenecks: repeated work gets multiplied by branch pushes and PR updates until the critical path becomes a permanent drag on delivery speed.

How to Measure and Find Bottlenecks in GitHub Actions#

The first step is to stop guessing and start measuring. A lot of teams edit YAML before they know which step is actually expensive.

The process that worked for us was simple:

1. open the workflow run summary and look at job timing
2. review the slowest steps in the log viewer
3. split the pipeline into install, build, test, image, artifact, and deploy stages
4. check whether any stage is doing the same work twice
5. compare a few recent runs to find the repeatable bottleneck

After that, the questions were obvious:

• Are we reinstalling the same dependencies every run?
• Are we rebuilding Docker layers that did not change?
• Are the tests too slow because they are all serial?
• Are stale runs stacking up and wasting queue time?
• Are we running the full pipeline for changes that should not trigger it?

Once those answers are clear, the optimization work becomes much more concrete. This is the point where caching and parallelism start to pay off. These are the references I kept open while making the changes below:

• Caching dependencies to speed up workflows (GitHub Docs)
• Using a matrix for your jobs (GitHub Docs)
• Using concurrency (GitHub Docs)
• About GitHub-hosted runners (GitHub Docs)
• Optimizing builds (Docker Docs), especially for BuildKit caching

Dependency Caching with actions/cache#

This was the first lever I reached for. In JavaScript and Node.js repos, dependency installation is often the biggest constant cost. The same package graph is downloaded and resolved over and over, even when the code change is unrelated.

Before, the job looked like this:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test -- --runInBand

The fix was simple and effective:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test -- --runInBand

This is an obvious optimization, but it matters because it removes the same repeated work every single time, and it is one of the fastest wins you will find. For monorepos, I also used cache keys based on the lockfile and package paths so unrelated packages did not invalidate the same cache.

Time saved: roughly 8–15 minutes in our baseline pipeline.

Docker Layer Caching and BuildKit#

The next bottleneck was Docker image builds. We were rebuilding the same container layers on every push, even when only a couple of files changed. That is exactly where Docker layer caching helps.

steps:
  - uses: actions/checkout@v4

  - name: Set up Docker Buildx
    uses: docker/setup-buildx-action@v3

  - name: Build and cache image
    uses: docker/build-push-action@v6
    with:
      context: .
      file: ./Dockerfile
      push: false
      load: true
      cache-from: type=gha
      cache-to: type=gha,mode=max

This matters because image layering is incremental. If the base image, system dependencies, or core packages are unchanged, Docker can reuse them from the cache instead of rebuilding them. That can cut image build time dramatically, especially in services with large dependency trees or heavy build steps. The cache only helps if the Dockerfile is ordered for it: copy the package manifests and install dependencies before copying the source, as in this NestJS multi-stage Dockerfile.

This was a big part of the GitHub Actions optimization work because we were not just trying to save compute. We were trying to reduce the time between a commit and actual deploy feedback. That is why caching is such a strong lever: it reduces waste without sacrificing reliability.

Time saved: around 10–18 minutes on Docker-heavy workflows.

Parallel Jobs and Matrix Builds#

The biggest time reduction came from parallelism.

Our pipeline had a long serial chain: lint, unit tests, integration tests, build, image validation, artifact packaging, and deploy. That meant the slowest single job became the gate for everyone else. Once I split independent checks into separate jobs and used a matrix strategy for test shards, the total time dropped noticeably.

strategy:
  fail-fast: false
  matrix:
    shard: [1, 2, 3, 4]

steps:
  - uses: actions/checkout@v4
  - uses: actions/setup-node@v4
    with:
      node-version: 22
      cache: 'npm'

  - name: Install dependencies
    run: npm ci

  - name: Run shard tests
    run: npm test -- --runInBand --shard=${{ matrix.shard }}/4

This is where a lot of CI/CD pipeline bottlenecks become visible. People often assume the whole workflow is slow because the tests are slow, but the deeper issue is that the test work is serialized instead of distributed. Parallel jobs reduce the critical path and make the whole team faster.

The important nuance is to avoid over-parallelizing. The right goal is to split independent work, not to add a million jobs for the sake of it. We kept a clean critical path and let the independent checks run in parallel.

Time saved: 10–25 minutes depending on the suite size and how much of it could be sharded.

Path Filters and Concurrency Control#

A surprising amount of wasted time came from running the full pipeline for changes that did not affect production code. This included docs-only changes, markdown edits, and unrelated folders in a monorepo.

on:
  pull_request:
    paths:
      - 'src/**'
      - 'package.json'
      - 'package-lock.json'
      - 'Dockerfile'
      - '.github/workflows/**'

Path filters are one of the cheapest wins because they stop irrelevant work from blocking the team. They reduce queue time, make the signal cleaner, and remove friction from code review.

One caveat: if this workflow is a required status check in branch protection, a run skipped by a path filter never reports, and the pull request waits on it forever. In that case, keep the workflow running and skip the expensive jobs inside it instead, for example with a changed-files check as the first job.

We also used concurrency groups to cancel stale runs:

concurrency:
  group: ci-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

This prevented three almost-identical workflows from stacking up when someone pushed several commits in a row. Without this, the team was paying for stale work that no longer mattered. Concurrency control improved both speed and predictability.

Cancel stale runs for CI checks, but not for deploys: a production deploy stopped halfway can leave a server between two versions. For deploy workflows, set cancel-in-progress: false so new runs queue behind the current one, as in my zero-downtime deploy workflow.

Time saved: a few minutes in active development, but a lot of queue noise eliminated.

Reusable Workflows, Composite Actions, and Artifact Reuse#

A lot of pipeline pain is caused by duplication. We had the same install steps, the same lint commands, and the same image build flow repeated in multiple workflows. That created extra maintenance work and more places for drift to happen.

The fix was to move shared logic into reusable workflows and composite actions:

jobs:
  unit:
    uses: ./.github/workflows/reusable-test.yml
    secrets: inherit

This reduced duplication and helped keep the cache strategy consistent. It also made the pipeline easier to review and easier to optimize as the project grew. A smaller, cleaner workflow is easier to trim and easier to trust.

We also reused artifacts instead of rebuilding the same output for every downstream job:

- name: Build app
  run: npm run build

- name: Upload artifact
  uses: actions/upload-artifact@v4
  with:
    name: app-build
    path: dist/

Downstream jobs then download that build instead of running it again:

- name: Download artifact
  uses: actions/download-artifact@v4
  with:
    name: app-build
    path: dist/

That is one of the cleanest GitHub Actions optimization patterns in a pipeline with multiple validation and deploy steps. The idea is simple: build once, reuse the artifact, keep the critical path short.

How I Used AI (GitHub Copilot & LLMs) to Find and Fix Pipeline Bottlenecks#

AI was useful, but it did not replace the actual optimization process. I used GitHub Copilot and LLMs to do three things: analyze workflow logs, suggest better YAML patterns, and triage flaky tests.

The first step was log analysis. I asked AI to scan the workflow summary and point out the slowest steps. It was surprisingly helpful at spotting patterns I might have missed, like repeated dependency install work or Docker layers that were rebuilding without any meaningful change.

The second use case was refactoring YAML. I asked Copilot to suggest matrix strategies, cache patterns, and reusable workflow layouts. It did not always get it right on the first pass, but it cut down the time I spent drafting and iterating. That is where AI in DevOps is strongest: it accelerates the first draft and helps you reason about patterns faster.

The third use case was flaky test triage. AI helped group similar failures and suggest where test selection or sharding could reduce noise. It also helped me think through self-hosted runner trade-offs, larger runner sizing, and the cost of parallel jobs. Those are decisions that depend on real system constraints, not just raw theory.

The honest part is that AI suggestions were not always correct. Some were too abstract, some ignored the actual repo shape, and some missed the fact that a simple cache key fix was more valuable than a fancy runner change. I verified every recommendation against actual timing data. That is the important point: AI can help make the diagnosis faster, but human review still matters.

Results: Before vs. After#

Here is the shape of the change in our pipeline.

These numbers are a real example of why GitHub Actions optimization matters. The pipeline did not become faster because I changed one line. It became faster because I removed repeated work, reduced the critical path, and made the system easier to reason about.

The DORA metrics angle matters here as well. Faster feedback and more reliable deploy flow usually help reduce lead time for changes and make deployment frequency more comfortable for the team. That is not just a technical benefit. It is a team and product benefit.

Mistakes I Made and Lessons Learned#

The first mistake was parallelizing too early. I added more jobs before I had a clear view of the bottleneck, and the result was extra complexity without enough benefit. I learned that measurement comes before optimization.

The second mistake was over-caching. Some cache keys were too broad and reused stale dependencies. That taught me to keep the cache narrow and tie it to actual dependency changes.

The third mistake was trying to cut test coverage rather than split it. If you remove protection, you are not speeding up CI responsibly. Sharding or splitting the suite is the safer move.

The fourth mistake was ignoring the queue. Even when one job was faster, stale workflow runs still filled the queue. Concurrency control fixed that.

The final lesson: start with the bottleneck that repeats the most, then fix the next one. Small wins compound quickly.

A Checklist to Speed Up Your GitHub Actions Pipeline#

• Measure each stage before changing YAML.
• Enable dependency caching with actions/cache or the built-in Node cache.
• Use Docker layer caching and BuildKit for image-heavy jobs.
• Split long suites into matrix or sharded jobs.
• Add path filters so unrelated changes do not trigger the full pipeline.
• Add concurrency groups to cancel stale runs.
• Reuse artifacts instead of rebuilding the same output.
• Extract repeated logic into reusable workflows and composite actions.
• Fix or quarantine flaky tests instead of letting them slow the team down.
• Keep runner choice tied to real workload and cost.
• Use AI for log analysis and workflow drafting, but verify everything manually.

This checklist is practical because it focuses on eliminating wasted work instead of adding complexity for its own sake.

The Future of AI in CI/CD Pipelines#

The future of AI in CI/CD is not an autonomous pipeline that magically ships code. It is better diagnosis, better recommendations, and better developer tooling. We will see more AI-assisted pipeline optimization, more intelligent caching, and more predictive test selection based on historical run data and code churn.

That is a useful direction, but not a replacement for engineering judgment. A good pipeline still requires a human to decide what is safe to automate, what to cache, and which jobs must stay protected by tests. AI can help, but it should not become an excuse to stop understanding the workflow.

FAQ#

How do I speed up my GitHub Actions pipeline?#

Measure first: open the workflow run summary and find the slowest steps. Then remove the biggest repeated costs, usually by caching dependencies, caching Docker layers with BuildKit, splitting long test suites into parallel matrix shards, adding path filters, and cancelling stale runs with a concurrency group.

Why is my GitHub Actions workflow so slow?#

The usual causes are repeated dependency installs, serial test suites, Docker rebuilds of unchanged layers, and stale queued workflows. Each one repeats work that has already been done, and every branch push and pull request update multiplies it.

Does caching really reduce GitHub Actions build time?#

Yes, when it is used correctly. Caching stops the runner from redoing the same install and build work again and again. Key the cache on the lockfile so it is reused until dependencies actually change; keys that are too broad can restore stale dependencies.

Can AI help optimize CI/CD pipelines?#

Yes, especially for log analysis, YAML refactoring, and flaky test triage. It can accelerate discovery, but it still needs human verification.

How can GitHub Copilot help with GitHub Actions workflows?#

It can draft YAML, explain syntax, and suggest patterns like matrix builds and concurrency. It is excellent for the first draft, but the final workflow still needs review.

Are self-hosted runners faster than GitHub-hosted runners?#

Sometimes, but not automatically. They are useful when the workload is heavy, repeated, and predictable enough to justify the maintenance cost.

Wrapping Up#

There is no single magic fix. The fastest GitHub Actions pipelines avoid repeated work, parallelize what is truly independent, and keep the critical path short, without becoming fragile.

If your pipeline is slow, start with measurement and cache reuse. Then split large suites, add path filters, and cancel stale runs before they fill the queue. Those wins compound quickly.

Related reading:

• Deploying Node.js to production: the complete guide, for the whole pipeline end to end
• Docker layer caching in GitHub Actions, for the cache options in depth
• NestJS Dockerfile: multi-stage builds for production, for an image that caches well in CI
• GitHub Actions CI/CD: build, test and deploy to EC2, for releasing the image to AWS
• Zero-downtime deploys for a Node.js API with Docker, Nginx and GitHub Actions, for what happens after the pipeline goes green