A couple more uses for generics in Go

As part of the upgrade to version 2 of the Brightbox Go API I’ve been using some of the newer features Go has introduced over the past few years: generators, generics and error wrapping.

Today we’ll be looking at a couple of approaches to using the new Go generic facilities that I’ve found useful within the API.

Go Generics Overview

Go is a function-oriented language, rather than a functional language, and the generics facility within Go reflects that. As it is the shiny new thing, it is easy to overuse it if you’re not careful.

However if used sparingly, then it is very useful indeed. Here are the three generic set operations which work between slices of any comparable type.

// Generic Set Operations
// Set Difference: A - B
func Difference[O comparable](a, b []O) (diff []O) {
    m := make(map[O]bool, len(b))

    for _, item := range b {
        m[item] = true
    }

    for _, item := range a {
        if _, ok := m[item]; !ok {
            diff = append(diff, item)
        }
    }
    return
}

func Intersection[O comparable](a, b []O) (intersect []O) {
    m := make(map[O]bool, len(a))

    for _, item := range a {
        m[item] = true
    }

    for _, item := range b {
        if _, ok := m[item]; ok {
            intersect = append(intersect, item)
        }
    }
    return
}

func Union[O comparable](a, b []O) []O {
    m := make(map[O]bool, len(a))

    for _, item := range a {
        m[item] = true
    }

    for _, item := range b {
        if _, ok := m[item]; !ok {
            a = append(a, item)
        }
    }
    return a
}

By far the most powerful mechanism within Go generics is inference. Thanks to this feature and with clever parameter design you can avoid having to specify generic arguments almost all the time. For example to create the difference of two sets you just call the function with the sets, and Go infers the rest.

result := Difference(requiredVolumeList, currentVolumeList)

Function Constraints

Inference can be combined with another feature of Go - first class functions - to allow generic parameters to be constrained dynamically by a function.

As we saw in the previous example a generic parameter can be given an interface type to restrict the types that are valid with that function. The comparable restriction allows the generic function to use a variable of the generic type as the key in a map.

The other approach is to declare generic parameters as any type, and pass in a function whose signature uses those types. The inference system then dynamically restricts the types to those that work with the function, with the compiler rejecting anything else.

This was very useful with the Brightbox API, where each object has an associated set of options specifically for creating that object. Creating generic functions with a function constraint guarantees each type is associated with the correct option type. Here’s the test function

func testModify[O, I any](
    t *testing.T,
    modify func(*Client, context.Context, I) (*O, error),
    newOptions I,
    jsonPath string,
    verb string,
    expectedPath string,
    expectedBody string,
) *O {
    ts, client, err := SetupConnection(
        &APIMock{
            T:            t,
            ExpectMethod: verb,
            ExpectURL:    "/1.0/" + expectedPath,
            ExpectBody:   expectedBody,
            GiveBody:     readJSON(jsonPath),
        },
    )
    defer ts.Close()
    assert.Assert(t, is.Nil(err), "Connect returned an error")
    instance, err := modify(client, context.Background(), newOptions)
    assert.Assert(t, is.Nil(err))
    assert.Assert(t, instance != nil)
    return instance
}

Now we can pass in the required method as an argument, and the generic parameters are automatically constrained to the correct type

func TestCreateAPIClient(t *testing.T) {
    newResource := APIClientOptions{}
    instance := testModify(
        t,
        (*Client).CreateAPIClient,
        newResource,
        "api_client",
        "POST",
        path.Join("api_clients"),
        "{}",
    )
    assert.Equal(t, instance.ID, "cli-dsse2")
}

func TestUpdateAPIClient(t *testing.T) {
    updatedResource := APIClientOptions{ID: "cli-dsse2"}
    instance := testModify(
        t,
        (*Client).UpdateAPIClient,
        updatedResource,
        "api_client",
        "PUT",
        path.Join("api_clients", updatedResource.ID),
        "{}",
    )
    assert.Equal(t, instance.ID, updatedResource.ID)
}

If we change the type of the options, the compiler complains:

./api_clients_default_test.go:40:3: type ServerOptions of newResource does not match inferred type APIClientOptions for I

Late Creation

Another use of generic parameters is to allow the program to delay creation of a structure to the last moment. In v1 of the API the underlying MakeAPIRequest call created the structure and passed it into the function at the beginning of the request. If the call succeeded any JSON data obtained over HTTP was unmarshaled into it. However if there was any error at any point the structure still existed and ended up as garbage.

func (c *Client) CreateCloudIP(newCloudIP *CloudIPOptions) (*CloudIP, error) {
    cloudip := new(CloudIP)
    _, err := c.MakeAPIRequest("POST", "/1.0/cloud_ips", newCloudIP, &cloudip)

With v2 of the API and generic parameters we can pass in the type of structure we want as a generic argument and create it only when we know we have obtained the JSON data

func (c *Client) CreateCloudIP(ctx context.Context, newCloudIP CloudIPOptions) (*CloudIP, error) {
    return APIPost[CloudIP](ctx, c, CloudIPAPIPath, newCloudIP)
}

which eventually calls

func jsonResponse[O any](res *http.Response, hardcoreDecode bool) (*O, error) {
    if res.StatusCode >= 200 && res.StatusCode <= 299 {
        decode := json.NewDecoder(res.Body)
        if hardcoreDecode {
            decode.DisallowUnknownFields()
        }
        result := new(O)
        err := decode.Decode(result)
        ...

Only once we have confirmed successful receipt of JSON data do we create the return structure.

Bonus Feature - a hint of generic curry

As mentioned at the beginning, Go is a function oriented language rather than a functional one, and therefore the first class function facility isn’t quite as flexible as it could be.

However you can fake it somewhat. The first approach is just to create a function calling the function with some parameters filled in.

// APIDelete makes a DELETE request to the API
//
// relUrl is the relative path of the endpoint to the base URL, e.g. "servers".
func APIDelete[O any](
    ctx context.Context,
    q *Client,
    relUrl string,
) (*O, error) {
    return apiCommand[O](ctx, q, "DELETE", relUrl)
}

and then let the inline optimiser do the dirty work for you.

$ go test -gcflags "-m" 2>&1| more
./client.go:127:22: inlining call to apiCommand[go.shape.struct { "".ResourceRef; ID string; Name string; Description string; Default bool; Fqdn string; CreatedAt *time.Time "json:\"created_at\""; Account *"".Account; FirewallPolicy *"".FirewallPolicy "json:\"firewall_policy\""; Servers []"".Server }_0]

If you need something a little more dynamic, then the Function Factory is your friend

func resourceBrightboxDelete[O any](
    deleter func(*brightbox.Client, context.Context, string) (*O, error),
    objectName string,
) schema.DeleteContextFunc {
    return func(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
        client := meta.(*CompositeClient).APIClient

        log.Printf("[INFO] Deleting %s %s", objectName, d.Id())
        _, err := deleter(client, ctx, d.Id())
        if err != nil {
            return diag.FromErr(err)
        }
        log.Printf("[DEBUG] Deleted cleanly")
        return nil
    }
}

var resourceBrightboxFirewallRuleDelete = resourceBrightboxDelete(
	(*brightbox.Client).DestroyFirewallRule,
	"Firewall Rule",
)

Postscript

Go generics are, on balance, a worthwhile addition to the language that reduces, but doesn’t eliminate, the need for code generators. Go remains a wordy language that seems to require an extraordinary amount of vertical space to do even the most simple of things.

However I hope these few ideas will help to reduce the amount of copypasta in your Go code.

Try Brightbox for free

If you want to play with the Brightbox API, you can sign up in barely a minute and get £50 free credit to give it a go.