Builds in Minutes, Not an Hour: CI/CD for a Streaming Platform 

"Even a small change could mean waiting close to an hour for CI, and then the release could still get stuck because a migration failed. We can't operate like this any longer." - Client representative 

First, we tried the obvious: we added runners and tuned individual jobs. The gain was marginal because the process architecture stayed the same. A large codebase refactoring wasn't an option either, since development had to continue. That left one path: change the delivery mechanics themselves on top of the existing code. So we took on building the CI/CD process from scratch, from commit to production, and focused the work on three areas.

Build Only What's Needed We split the Maven project into smaller parts that can be built independently. The pipeline is generated for each specific change. Dependencies come from the cache, and build outputs are passed between stages without being built again.

Build Once, Deploy Everywhere CI builds the image once, and that same image is promoted across environments. Production receives the same container image that passed checks in the earlier environments.

GitOps, and Migrations Separate from the Application We moved the deployment logic out of CI scripts and into ArgoCD. We moved database migrations into dedicated Kubernetes Jobs and set the "migration first, then service" order with Sync Waves. We make changes like these as part of our Kubernetes consulting work, when a team already has a cluster but releases on it are painful.

CI, attempt #1: optimize Maven jobs and add runners. Almost everyone starts here, and so did we. Things got a bit better, but the problem didn't go away: the pipeline kept rebuilding overlapping components and doing work it could do without. You can't fix an architectural problem with more powerful runners.

CI, attempt #2: split the build and generate the pipeline per change. It worked. The gain came from three things together: splitting the build into smaller units, clear artifact boundaries between them, and a pipeline built according to which parts of the code a change touched. That's what delivered the bulk of the speedup. This kind of process breakdown, rather than tuning individual jobs, is where we start our CI/CD consulting.

CD, attempt #1: retry failed deployments. When a deployment failed because of a migration, it was simply restarted. Sometimes that helped, but the root cause remained: the migration was baked into pod startup. As long as that's the case, every release with a schema change is a gamble.

CD, attempt #2: move migrations out of init containers entirely. It worked. Now a migration is a dedicated Kubernetes Job with its own status and logs. Through Sync Waves, ArgoCD runs it before deploying the service. If the migration fails, the failure is visible immediately and in one place, and the new service version isn't deployed until the problem is fixed. New application pods no longer get stuck on repeated attempts to run the migration in an init container.

The Hardest Part: Changing the Process During Ongoing Releases

The hardest part was reworking a delivery process that had formed around a large Java/Maven codebase without interrupting the releases that were going out at the time.

First, we had to map the dependencies between modules correctly. If you parallelize the build without understanding those relationships, it's easy to end up with inconsistent artifacts: a module gets built against an old version of a dependency, and that only surfaces once it's in an environment.

Second, moving the migrations. The switch from init containers to dedicated Jobs had to preserve the same guarantee: the schema is updated before the new application version starts working with it. A mistake in the order of steps here would mean a broken release, so during the move, our first priority was keeping that sequence safe.

Custom Solutions

Dynamic child pipeline generator. At the start of the pipeline, a dedicated job determines which components a change touched and generates the child pipeline configuration. Only the components and checks needed for that change make it into the child pipeline.

Image promotion automation. After a successful CI run, a script updates the image reference in the environment configuration and points it to the exact immutable image CI built. ArgoCD picks up the change and brings the cluster to the desired state, so a separate build for each environment is no longer needed.

Stack

Language & build: Java, Maven CI/CD: GitLab CI/CD, GitLab dynamic child pipelines, GitLab artifacts Containers & orchestration: Docker, Kubernetes, Helm, container registry GitOps: ArgoCD

Specialized Tools

• GitLab dynamic child pipelines with generated configuration;
• ArgoCD Sync Waves for deployment ordering;
• Kubernetes Jobs for database migrations;
• immutable image propagation across environments;
• Maven dependency caching and reuse of GitLab artifacts between stages.