C# — EF Migrations with Kubernetes Jobs

Originally published on Medium · November 11, 2024

One thing that used to bug me about running EF Core against a database in Kubernetes was migrations — someone always ended up running dotnet ef database update by hand from their laptop, which works fine until it doesn't. Here's the setup I landed on instead: deploy MariaDB in the cluster, bundle EF migrations into a standalone executable, and let a Kubernetes Job apply them automatically. No manual steps, no laptop dependency, and the same process every time.

I'll walk through the whole thing below: standing up MariaDB, building the migration bundle, containerizing it, and wiring up the Job that runs it.

Prerequisites

Before proceeding, ensure you have:

Docker for containerizing and pushing your application.
Kubernetes set up (locally via Minikube or on a cloud provider).
.NET SDK with Entity Framework installed in your project.

Step 1: Deploying MariaDB to Kubernetes

We'll first deploy a MariaDB instance within Kubernetes. This MariaDB instance will act as the target database for our EF migrations.

1.1 Creating the MariaDB Deployment and Service

The deployment will create a MariaDB pod, and the service will expose it within the cluster. Save the following as mariadb-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mariadb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mariadb
  template:
    metadata:
      labels:
        app: mariadb
    spec:
      containers:
        - name: mariadb
          image: mariadb:latest
          env:
            - name: MARIADB_ROOT_PASSWORD
              value: "root-password" # Set a secure root password
            - name: MARIADB_DATABASE
              value: "Blogs" # Database for EF
          ports:
            - containerPort: 3306

---
apiVersion: v1
kind: Service
metadata:
  name: mariadb-server
spec:
  ports:
    - port: 3306
      targetPort: 3306
  selector:
    app: mariadb

Here:

MariaDB root password and database name are defined to align with our EF project.
Service mariadb-server is exposed on port 3306, making it accessible to other pods in the cluster.

Deploy it by running:

kubectl apply -f mariadb-deployment.yaml

Step 2: Creating an EF Migration Bundle Locally

First, we need to check if we can build the EF bundle locally. EF provides the dotnet ef migrations bundle command, which packages migrations into a standalone executable. This enables you to apply migrations without requiring the entire .NET runtime or SDK.

2.1 Generating the Migration Bundle

Navigate to your project directory and use the following command:

dotnet ef migrations bundle -p ./Blogs/Blogs.csproj

This command:

Bundles the EF migrations for the Blogs project.
Outputs an executable (e.g., efbundle.exe on Windows, efbundle on Linux) that can apply migrations when provided with a connection string.

2.2 Testing the Migration Bundle

To confirm the bundle works, run it with a local MariaDB connection:

.\efbundle.exe --connection "Server=localhost;Database=Blogs;User=root;Password=root;"

If successful, the migrations will apply to the specified database.

Step 3: Containerizing the Migration Bundle

Now that we have tested that the bundle is working locally, we will package this logic into a Docker image. For this, we will use the existing ASP.NET project's base Docker image and extend it to add the necessary steps to install the migration tools and generate the migration bundle.

3.1 Dockerfile for the Migration Bundle

In this step, we will leverage the base image from the existing Blogs ASP.NET project and extend it to include the creation of the migration bundle. The Dockerfile for this process looks as follows:

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
USER app
WORKDIR /app
EXPOSE 8080
EXPOSE 8081

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Blogs/Blogs.csproj", "Blogs/"]
RUN dotnet restore "./Blogs/Blogs.csproj"
COPY . .
WORKDIR "/src/Blogs"
RUN dotnet build "./Blogs.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "./Blogs.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

# Build the migration bundle using the dotnet-ef tool
FROM build as migrationbuilder
ENV PATH $PATH:/root/.dotnet/tools
WORKDIR /src
RUN dotnet tool install --global dotnet-ef
RUN dotnet ef migrations bundle -p ./Blogs/Blogs.csproj

# Final image containing the application and the migration bundle
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
COPY --from=migrationbuilder /src/efbundle .
ENTRYPOINT ["dotnet", "Blogs.dll"]

3.2 Building and Pushing the Docker Image

Run the following commands to build and push the image to a container registry:

docker build -t blogs -f ./Blogs/Dockerfile .
docker push blogs:latest

This image is now available for Kubernetes to pull and run.

Step 4: Applying Migrations via a Kubernetes Job

Now that the migration bundle is containerized and stored in a registry, we can create a Kubernetes job to run it. The job will:

Wait until MariaDB is ready.
Apply migrations using the containerized efbundle.

4.1 Defining the EF Migration Job

Create an ef-migration-job.yaml file with the following content:

apiVersion: batch/v1
kind: Job
metadata:
  name: ef-migration-job
spec:
  completions: 1  # The number of successful completions required to consider the job successful.
  parallelism: 1  # Defines how many pods can run in parallel.
  backoffLimit: 2  # Specifies the number of retries before marking the job as failed.
  ttlSecondsAfterFinished: 100  # Defines the time-to-live (TTL) after the job finishes.
  template:
    spec:
      containers:
        - name: migration
          image: cytra/blogs:latest
          command: ["/bin/sh", "-c"]  # The command to run in the container. We're using `sh` with `-c` to execute a shell command.
          args: ["./efbundle"]  # The command to run inside the shell
          env:
            - name: ConnectionStrings__DatabaseConnectionString
              value: "Server=mariadb-server;Database=Blogs;User=root;Password=root-password;"
      restartPolicy: OnFailure

Key point in this job definition:

Migration container runs the bundled migrations using efbundle with a connection string.

4.2 Deploying the Migration Job

Run the following command to deploy the job:

kubectl apply -f ef-migration-job.yaml

4.3 Verifying the Job Execution

Check the logs to ensure the migrations have been applied:

kubectl logs job/ef-migration-job

The logs should indicate the applied migrations with the final message Done.

Wrapping Up

Here's what this setup gets you:

Configured MariaDB as a Kubernetes Deployment and Service, making it accessible to other applications in the cluster.
Created a migration bundle with EF's dotnet ef migrations bundle command.
Containerized the bundle to be run as a job, allowing for flexible and scalable migration processes.
Deployed and verified the migration job in Kubernetes, applying migrations automatically.

This approach is particularly beneficial in cloud-native environments, where automated, containerized processes help ensure database schema consistency across deployments. By combining Kubernetes, Docker, and EF migrations, you now have a streamlined way to manage database changes in production-like environments.