Rubynetes: using OpenAPI to validate Kubernetes configs

Story So Far

In the previous post we saw how we can create a Kubernetes Manifest from a Ruby Hash. However, to check we don’t add incorrect values we have to write detailed tests. Can we automate that in any way? Yes, we can.

Fortunately it is very easy to get the type and layout of the resources Kubernetes uses directly from the Kubernetes server.

Enter OpenAPI

$ kubectl get --raw /openapi/v2

This will obtain the OpenAPI definitions of all the resources installed on the Kubernetes Cluster. All we need to do is translate the OpenAPI definition into a Ruby library we can use.

We can do that with a Makefile

Remember Make?

vendor: kubernetes-client
	bundle install --clean --without development test --deployment

kubernetes-client: kubeapi.json
	docker run --rm -v $(CURDIR):/local \
		-u $(shell id -u $(USER)):$(shell id -g $(USER)) \
		openapitools/openapi-generator-cli generate \
		-g ruby \
		-p "moduleName=Kubernetes" \
		-i /local/$< \
		-o /local/$@

kubeapi.json:
	kubectl get --raw /openapi/v2 > $@

.PHONY: clean
clean:
	rm -rf kubernetes-client kubeapi.json vendor

then tie it all together via the Gemfile

gemspec path: File.expand_path('kubernetes-client', __dir__)

source 'https://rubygems.org'
gem 'activesupport', '~> 6.0'

Build the library with make and we have a typed Ruby library ready to use - precisely tailored to the resources on your Kubernetes cluster.

Use the Generated Library

Now let’s change the create_job_manifest method from last time to use the types from the generated OpenAPI library.

def create_job_manifest(release, config, version, name)
  Kubernetes::IoK8sApiBatchV1Job.new(
    api_version: 'batch/v1',
    kind: 'Job',
    metadata: Kubernetes::IoK8sApimachineryPkgApisMetaV1ObjectMeta.new(
      name: name,
      labels: { build: config[:prefix] }
    ),
    spec: Kubernetes::IoK8sApiBatchV1JobSpec.new(
      ttl_seconds_after_finished: config[:holdTime],
      template: Kubernetes::IoK8sApiCoreV1PodTemplateSpec.new(
        spec: Kubernetes::IoK8sApiCoreV1PodSpec.new(
          restart_policy: 'Never',
          containers: [
            Kubernetes::IoK8sApiCoreV1Container.new(
              name: name,
              image: config[:image],
              resources: Kubernetes::IoK8sApiCoreV1ResourceRequirements.new(
                requests: {
                  memory: config[:requestsMemory],
                  cpu: config[:requestsCpu]
                },
                limits: {
                  memory: config[:limitsMemory],
                  cpu: config[:limitsCpu]
                }
              ),
              args: [
                '--dockerfile=Dockerfile',
                "--context=#{config[:gitRepo]}#refs/heads/release-#{release}",
                "--destination=#{config[:dockerTarget]}:#{version}"
              ],
              volume_mounts: [
                Kubernetes::IoK8sApiCoreV1VolumeMount.new(
                  name: config[:secretName], fred: 'boo', mount_path: '/root'
                )
              ]
            )
          ],
          volumes: [
            Kubernetes::IoK8sApiCoreV1Volume.new(
              name: config[:secretName],
              secret: Kubernetes::IoK8sApiCoreV1SecretVolumeSource.new(
                secret_name: config[:secretName],
                items: [
                  Kubernetes::IoK8sApiCoreV1KeyToPath.new(
                    key: '.dockerconfigjson', path: '.docker/config.json'
                  )
                ]
              )
            )
          ]
        )
      )
    )
  )
end

update the spec to use ‘dot format’ access.

describe '#create_job_manifest' do
  let(:job) {
    load 'create_docker_jobs'
    create_job_manifest('1.16', config, '1.16.1', 'job-1.16')
  }

  it 'has a restart Policy of never' do
    expect(job.spec.template.spec.restart_policy).to eq 'Never'
  end
end

And Test

Run the spec with rspec

$ bundle install --with development
$ bundle exec rspec
F

Failures:

  1) #create_job_manifest has a restart Policy of never
     Failure/Error: load 'create_docker_jobs'
     
     ArgumentError:
       `fred` is not a valid attribute in
       `Kubernetes::IoK8sApiCoreV1VolumeMount`. Please check the name
       to make sure its valid. List of attributes: [:mount_path,
       :mount_propagation, :name, :read_only, :sub_path, :sub_path_expr]
...

Whoops. How did that get in there?

Can We Do More?

Now we have typed manifests that we can manipulate easily within Ruby, but we still have to remember the names of the types. If you can remember that metadata is IoK8sApimachineryPkgApisMetaV1ObjectMeta, you’re doing better than me. Remembering stuff like that is what we have computers for.

Reflect on that, and we’ll see what we can do about it in the next post.

Postscript

Kubernetes knows about the resources it wants, therefore the best way to get the resources correct is to ask Kubernetes what it wants to see.

Fortunately, with the OpenAPI interface and the generators that can use that interface to create code libaries that match our cluster precisely, we can easily incorporate those resource specifications in our Ruby code with ease.

Interested in Managed Kubernetes?

Brightbox have been managing web deployments large and small for over a decade. If you’re interested in the benefits of Kubernetes but want us to handle managing and monitoring it for you, drop us a line.