Go Vanity Import Paths¶

Let’s dive a bit into Golang’s Vanity Import Paths. This article assumes you have a basic understanding of Go Modules.

We are all aware that Go module path is generally written in the format of <VCS host>/username/path/to/your/mod, example: github.com/yourusername/amazingmodule.

Now, imagine the case where you finished implementing amazingmodule already. You go ahead create a new git tag, then announce that this module is ready for public use.

Everything good so far. Your module users are happy, and you see your module are getting more and more usage. Life is going well until one day, for some reason, you decide to change the VCS you are currently using / change your VCS username.

Either change will result in needing to change your Go module path as well. Changing module path means subsequent releases will be treated as a completely different module, which means all exported fields (variables, functions, etc) released with the new module path will not be compatible with the exported fields released with the old module path. Good luck asking your module users to migrate to use the new module path. The difficulty becomes greater as your module is used transitively.

drawing

Transitive Dependency. Image source: https://xkcd.com/2347/

Decoupling where your actual module lives from module path is something that can be achieved using Vanity Import Paths.

How does Vanity Import work¶

Think of Vanity Import as creating a custom alias for your module path.

To use vanity import, you need to first setup something that we will refer to as vanity server. Vanity server is responsible to map each alias to where the actual code is being hosted.

Now, let’s assume we have vanity server setup at vanityserver.domain. And we have a Go module hosted in github.com/actual/path.

Your go module’s go.mod could look like:

// github.com/actual/path's go.mod

module vanityserver.domain/custom/path

...

You can see that your module path is customized, and decoupled from actual VCS path. Only important thing to note here is that it needs to be prefixed with your vanity server host. As it is decoupled, you are free to change the actual location of your code.

Diagram below illustrates how Go fetches your module using vanity server:

sequenceDiagram participant goget as go get vanityserver.domain/custom/path participant vanityserver.domain participant github.com as github.com/actual/path goget->>vanityserver.domain: try to download module activate vanityserver.domain vanityserver.domain->>goget: returns actual location of the module: github.com/actual/path deactivate vanityserver.domain Note right of goget: actual location of the module is returned as HTML body containing an import-meta header (see below) goget->>github.com: try to download module activate github.com github.com->>goget: module downloaded deactivate github.com

go-import meta header tag¶

go-import meta is structured as:

<meta name="go-import" content="<custom alias> <VCS> <actual mod location>">

Using the example above, your vanity server could return something like this when vanityserver.domain/custom/path is requested:

<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <meta name="go-import" content="vanityserver.domain/custom/path git ssh://git@github.com/actual/path">
    </head>
</html>

What about nested modules?¶

Using the go-import meta structure above:

content="<custom alias> <VCS> <actual mod location>"

<custom alias> and <actual mod location> are 2 things that you need to pay attention for nested modules:

<custom alias> is the alias for the whole project (not a specific nested module). Another way to think about this is, imagine you have a root go.mod, whatever custom path you set for your root go.mod module path must also be the value of <custom alias>
<actual mod location> is the URL that points to the root dir of your project in VCS (not pointing to a specific nested module)

Let’s see an example for better clarity. Assume we host our nested modules in github.com/username/nestedmod, and we want to alias this project as vanityserver.domain/custom/nested

github.com/username/nestedmod
├── modone
│   ├── go.mod // module path: vanityserver.domain/custom/nested/modone
│   └── impl.go
├── modtwo
│   ├── go.mod // module path: vanityserver.domain/custom/nested/modtwo
│   └── impl.go
└── README.md

When we request modone to vanity server, the response that will be returned will still look like:

<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <meta name="go-import" content="vanityserver.domain/custom/nested git ssh://git@github.com/username/nestedmod">
    </head>
</html>

sequenceDiagram participant goget as go get vanityserver.domain/custom/nested/modone participant vanityserver.domain participant github.com as github.com/username/nestedmod goget->>vanityserver.domain: try to download module activate vanityserver.domain vanityserver.domain->>goget: returns actual location of the module: github.com/username/nestedmod deactivate vanityserver.domain goget->>github.com: try to download nested module modone, looks for git tag modone/<semver> activate github.com github.com->>goget: module downloaded deactivate github.com goget-->goget: check whether there's modone/go.mod from the downloaded module activate goget deactivate goget

Vanity Import, hands-on¶

Now that we have an understanding of vanity import path, let’s try to implement one ourself. In the following section, we’ll:

Start up https://github.com/localtunnel/localtunnel. As our vanity server will be started in local machine, we will use the host we get from localtunnel as our vanity server host. Feel free if you prefer to use other tool, such as ngrok.
Create a repository containing multiple modules (root go.mod and nested go.mod). Our module paths will use custom aliases.
Write our vanity server, to map the alias to actual repository location

1. Localtunnel¶

Follow the installation steps here if you haven’t installed yet. Then start localtunnel at port 8000:

lt --port 8000

After you start localtunnel, it should give you a host with random subdomain such as: sweet-views-lick.loca.lt. Note this down, we’ll need it later.

2. Create repository¶

We’ll be hosting the code in a github public repository for this demo (private repo also works, but there are additional configurations. For simplicity we’ll just use public repo).

First of all, let’s write the module codes. Create a project with this structure

├── nestedmod/
│   └── hello/
│       └── hello.go
└── hello/
    └──hello.go

Inside your root dir, init go module using (replace <lthost> with your localtunnel host):

go mod init <lthost>/aliased/module

Then inside nestedmod dir, init go module using (replace <lthost> with your localtunnel host):

go mod init <lthost>/aliased/module/nestedmod

You can fill both hello.go with the following code:

package hello

func Hello() string {
  return "Hello world"
}

The final directory structure will look like:

├── nestedmod/
│   ├── go.mod // module <lthost>/aliased/module/nestedmod
│   └── hello/
│       └──hello.go
├── hello/
│   └── hello.go
└── go.mod // module <lthost>/aliased/module

After that, create a github repo, then push your project into the github repo. Your github repo name can be anything.

Finally, create a tag for both the root module and nested module, then push to your git repo:

git tag v1.0.0
git push origin v1.0.0

git tag nestedmod/v1.0.0
git push origin nestedmod/v1.0.0

3. Starting vanity server¶

I’ve created a simple vanity server here: https://github.com/chfern/golang-simple-vanity-server.

Clone the repository, then open main.go. There, you need to change:

<vanityhost> with your actual localtunnel host, e.g: sweet-views-lick.loca.lt
<modalias> to: aliased/module
<actualurl> with your actual github repo path, e.g: https://github.com/yourusername/repo

Finally, run go run main.go. It will start the vanity server at port 8000

Test everything out¶

Create a new folder, then make it a Go module using go mod init.

Now let’s try fetching your hosted module that is using a custom alias. Try running (replace <lthost> with your localtunnel host):

go get <lthost>/aliased/module/nestedmod
go get <lthost>/aliased/module

It should successfully retrieve the modules.

Congrats, you’ve successfully alias your module path using vanity import. After everything is done, don’t forget to stop the localtunnel.

Transitioning to use Vanity Import Path¶

Not everything is properly planned from start. What if we have an existing module that is not using vanity import path, but then we want to migrate to use vanity import path?

Simply changing the module path will result in the original problem that we’re trying to solve, different module path is treated as different module.

To aid with transitioning, we can utilize Go type alias. Read more about it here: https://github.com/golang/go/issues/18130.

The idea is simple. Create a type alias for every exported field from the old module path to point to the new module path that is using vanity import.

The steps will be something like:

Copy your module code into a new location, then change the module path to use vanity import path
In the old module code, change every exported field to alias to the copied module code that is using vanity import.
Release a new version of your old module code, which contains type aliases.
Your module users can now start migrating to use the new module with vanity import path and it will still be compatible with the old module path.