Using go sub/nested modules together with go-import meta tags for client libraries
For our internal microservices we use a mixture of gRPC and REST services. Originally we had all gRPC protobuf generated code in a internal go-cloudbear library. Eventually this became cumbersome to maintain and version.
To solve this we thought it would be useful to move the (generated) API clients to the microservice repo to which they belong. This allows us the maintain the client library along side the server-side code.
Just putting the client library in a api package and being done with it doesn’t really work though. This doesn’t work for us as every user of the client library would be including all the dependencies of the microservice itself.
Using go nested modules
To solve this we can turn the api package into a go submodule (important to note is that officially they’re called nested modules to avoid confusion with git submodules). We do this by creating a new go.mod file in the package. This didn’t really work as expected and go get kept returning errors like:
go get cbws.xyz/virtual-servers/api@vm: module cbws.xyz/virtual-servers@vm found (v1.2.1-0.20200205200148-6470a35e8c57), but does not contain package cbws.xyz/virtual-servers/api
Using go-import meta tags
To make sure we are not forever tied to gitlab.com we chose to put all our microservices on the cbws.xyz hostname. We did this by including a bit of NGINX configuration that looks for ?go-get=1 and returns a appropriate go-import meta tag which the go tool looks for.
The solution
The go-import NGINX configuration we had wasn’t compatible with go-getting nested modules though. In the end we got it to work by changing the configuration a bit:
location ~ ^/([a-z-]+)(/(.+))?$ {
if ($args = "go-get=1") {
add_header Content-Type text/html;
return 200 '<meta name="go-import" content="$host/$1 git ssh://git@gitlab.com/cloudbear/microservices/$1.git">';
}
}Why this works
This new configuration makes sure go get always point to the root of package, so for example cbws.xyz/microservice and not cbws.xyz/microservice/submodule. It then downloads the entire repository and will only use the code in the nested module.
We can successfully include our nested module in other applications:
get "cbws.xyz/virtual-servers/api": found meta tag get.metaImport{Prefix:"cbws.xyz/virtual-servers", VCS:"git", RepoRoot:"ssh://git@gitlab.com/cloudbear/microservices/virtual-servers.git"} at //cbws.xyz/virtual-servers/api?go-get=1
get "cbws.xyz/virtual-servers": found meta tag get.metaImport{Prefix:"cbws.xyz/virtual-servers", VCS:"git", RepoRoot:"ssh://git@gitlab.com/cloudbear/microservices/virtual-servers.git"} at //cbws.xyz/virtual-servers?go-get=1
get "cbws.xyz/virtual-servers/api": verifying non-authoritative meta tag
go: downloading cbws.xyz/virtual-servers/api v0.0.0-20200205200148-6470a35e8c57
go: cbws.xyz/virtual-servers/api vm => v0.0.0-20200205200148-6470a35e8c57