Table of contents
Recently, the pkg-go team has been quite busy, uploading dozens of Go library packages in order to be able to package gcsfuse (a user-space file system for interacting with Google Cloud Storage) and InfluxDB (an open-source distributed time series database).
Packaging Go library packages (!) is a fairly repetitive process, so before starting my work on the dependencies for gcsfuse, I started writing a tool called dh-make-golang. Just like dh-make itself, the goal is to automatically create (almost) an entire Debian package.
As I worked my way through the dependencies of gcsfuse, I refined how the tool works, and now I believe it’s good enough for a first release.
To demonstrate how the tool works, let’s assume we want to package the Go library github.com/jacobsa/ratelimit:
midna /tmp $ dh-make-golang github.com/jacobsa/ratelimit 2015/07/25 18:25:39 Downloading "github.com/jacobsa/ratelimit/..." 2015/07/25 18:25:53 Determining upstream version number 2015/07/25 18:25:53 Package version is "0.0~git20150723.0.2ca5e0c" 2015/07/25 18:25:53 Determining dependencies 2015/07/25 18:25:55 2015/07/25 18:25:55 Packaging successfully created in /tmp/golang-github-jacobsa-ratelimit 2015/07/25 18:25:55 2015/07/25 18:25:55 Resolve all TODOs in itp-golang-github-jacobsa-ratelimit.txt, then email it out: 2015/07/25 18:25:55 sendmail -t -f < itp-golang-github-jacobsa-ratelimit.txt 2015/07/25 18:25:55 2015/07/25 18:25:55 Resolve all the TODOs in debian/, find them using: 2015/07/25 18:25:55 grep -r TODO debian 2015/07/25 18:25:55 2015/07/25 18:25:55 To build the package, commit the packaging and use gbp buildpackage: 2015/07/25 18:25:55 git add debian && git commit -a -m 'Initial packaging' 2015/07/25 18:25:55 gbp buildpackage --git-pbuilder 2015/07/25 18:25:55 2015/07/25 18:25:55 To create the packaging git repository on salsa, use: 2015/07/25 18:25:55 dh-make-golang create-salsa-project golang-github-jacobsa-ratelimit 2015/07/25 18:25:55 2015/07/25 18:25:55 Once you are happy with your packaging, push it to alioth using: 2015/07/25 18:25:55 git remote set-url origin [email protected]:go-team/packages/golang-github-jacobsa-ratelimit.git 2015/07/25 18:25:55 gbp push
The ITP is often the most labor-intensive part of the packaging process, because any number of auto-detected values might be wrong: the repository owner might not be the “Upstream Author”, the repository might not have a short description, the long description might need some adjustments or the license might not be auto-detected.
midna /tmp $ cat itp-golang-github-jacobsa-ratelimit.txt From: "Michael Stapelberg" <stapelberg AT debian.org> To: [email protected] Subject: ITP: golang-github-jacobsa-ratelimit -- Go package for rate limiting Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Package: wnpp Severity: wishlist Owner: Michael Stapelberg <stapelberg AT debian.org> * Package name : golang-github-jacobsa-ratelimit Version : 0.0~git20150723.0.2ca5e0c-1 Upstream Author : Aaron Jacobs * URL : https://github.com/jacobsa/ratelimit * License : Apache-2.0 Programming Lang: Go Description : Go package for rate limiting GoDoc (https://godoc.org/github.com/jacobsa/ratelimit) . This package contains code for dealing with rate limiting. See the reference (http://godoc.org/github.com/jacobsa/ratelimit) for more info. TODO: perhaps reasoning midna /tmp $
After filling in all the TODOs in the file, let’s mail it out and get a sense of what else still needs to be done:
midna /tmp $ sendmail -t -f < itp-golang-github-jacobsa-ratelimit.txt midna /tmp $ cd golang-github-jacobsa-ratelimit midna /tmp/golang-github-jacobsa-ratelimit master $ grep -r TODO debian debian/changelog: * Initial release (Closes: TODO) midna /tmp/golang-github-jacobsa-ratelimit master $
After filling in these TODOs as well, let’s have a final look at what we’re about to build:
midna /tmp/golang-github-jacobsa-ratelimit master $ head -100 debian/**/* ==> debian/changelog <== golang-github-jacobsa-ratelimit (0.0~git20150723.0.2ca5e0c-1) unstable; urgency=medium * Initial release (Closes: #793646) -- Michael Stapelberg <[email protected]> Sat, 25 Jul 2015 23:26:34 +0200 ==> debian/compat <== 9 ==> debian/control <== Source: golang-github-jacobsa-ratelimit Section: devel Priority: extra Maintainer: pkg-go <[email protected]> Uploaders: Michael Stapelberg <[email protected]> Build-Depends: debhelper (>= 9), dh-golang, golang-go, golang-github-jacobsa-gcloud-dev, golang-github-jacobsa-oglematchers-dev, golang-github-jacobsa-ogletest-dev, golang-github-jacobsa-syncutil-dev, golang-golang-x-net-dev Standards-Version: 3.9.6 Homepage: https://github.com/jacobsa/ratelimit Vcs-Browser: http://anonscm.debian.org/gitweb/?p=pkg-go/packages/golang-github-jacobsa-ratelimit.git;a=summary Vcs-Git: git://anonscm.debian.org/pkg-go/packages/golang-github-jacobsa-ratelimit.git Package: golang-github-jacobsa-ratelimit-dev Architecture: all Depends: ${shlibs:Depends}, ${misc:Depends}, golang-go, golang-github-jacobsa-gcloud-dev, golang-github-jacobsa-oglematchers-dev, golang-github-jacobsa-ogletest-dev, golang-github-jacobsa-syncutil-dev, golang-golang-x-net-dev Built-Using: ${misc:Built-Using} Description: Go package for rate limiting This package contains code for dealing with rate limiting. See the reference (http://godoc.org/github.com/jacobsa/ratelimit) for more info. ==> debian/copyright <== Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ Upstream-Name: ratelimit Source: https://github.com/jacobsa/ratelimit Files: * Copyright: 2015 Aaron Jacobs License: Apache-2.0 Files: debian/* Copyright: 2015 Michael Stapelberg <[email protected]> License: Apache-2.0 Comment: Debian packaging is licensed under the same terms as upstream License: Apache-2.0 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at . http://www.apache.org/licenses/LICENSE-2.0 . Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. . On Debian systems, the complete text of the Apache version 2.0 license can be found in "/usr/share/common-licenses/Apache-2.0". ==> debian/gbp.conf <== [DEFAULT] pristine-tar = True ==> debian/rules <== #!/usr/bin/make -f export DH_GOPKG := github.com/jacobsa/ratelimit %: dh $@ --buildsystem=golang --with=golang ==> debian/source <== head: error reading ‘debian/source’: Is a directory ==> debian/source/format <== 3.0 (quilt) midna /tmp/golang-github-jacobsa-ratelimit master $
Okay, then. Let’s give it a shot and see if it builds:
midna /tmp/golang-github-jacobsa-ratelimit master $ git add debian && git commit -a -m 'Initial packaging' [master 48f4c25] Initial packaging 7 files changed, 75 insertions(+) create mode 100644 debian/changelog create mode 100644 debian/compat create mode 100644 debian/control create mode 100644 debian/copyright create mode 100644 debian/gbp.conf create mode 100755 debian/rules create mode 100644 debian/source/format midna /tmp/golang-github-jacobsa-ratelimit master $ gbp buildpackage --git-pbuilder […] midna /tmp/golang-github-jacobsa-ratelimit master $ lintian ../golang-github-jacobsa-ratelimit_0.0\~git20150723.0.2ca5e0c-1_amd64.changes I: golang-github-jacobsa-ratelimit source: debian-watch-file-is-missing P: golang-github-jacobsa-ratelimit-dev: no-upstream-changelog I: golang-github-jacobsa-ratelimit-dev: extended-description-is-probably-too-short midna /tmp/golang-github-jacobsa-ratelimit master $
This package just built (as it should!), but occasionally one might need to disable a test and file an upstream bug about it. So, let’s push this package to pkg-go and upload it:
midna /tmp/golang-github-jacobsa-ratelimit master $ ssh git.debian.org "/git/pkg-go/setup-repository golang-github-jacobsa-ratelimit 'Packaging for golang-github-jacobsa-ratelimit'" Initialized empty shared Git repository in /srv/git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git/ HEAD is now at ea6b1c5 add mrconfig for dh-make-golang [master c5be5a1] add mrconfig for golang-github-jacobsa-ratelimit 1 file changed, 3 insertions(+) To /git/pkg-go/meta.git ea6b1c5..c5be5a1 master -> master midna /tmp/golang-github-jacobsa-ratelimit master $ git push git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git --tags master pristine-tar upstream Counting objects: 31, done. Delta compression using up to 8 threads. Compressing objects: 100% (25/25), done. Writing objects: 100% (31/31), 18.38 KiB | 0 bytes/s, done. Total 31 (delta 2), reused 0 (delta 0) To git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git * [new branch] master -> master * [new branch] pristine-tar -> pristine-tar * [new branch] upstream -> upstream * [new tag] upstream/0.0_git20150723.0.2ca5e0c -> upstream/0.0_git20150723.0.2ca5e0c midna /tmp/golang-github-jacobsa-ratelimit master $ cd .. midna /tmp $ debsign golang-github-jacobsa-ratelimit_0.0\~git20150723.0.2ca5e0c-1_amd64.changes […] midna /tmp $ dput golang-github-jacobsa-ratelimit_0.0\~git20150723.0.2ca5e0c-1_amd64.changes Uploading golang-github-jacobsa-ratelimit using ftp to ftp-master (host: ftp.upload.debian.org; directory: /pub/UploadQueue/) […] Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1.dsc Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c.orig.tar.bz2 Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1.debian.tar.xz Uploading golang-github-jacobsa-ratelimit-dev_0.0~git20150723.0.2ca5e0c-1_all.deb Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1_amd64.changes midna /tmp $ cd golang-github-jacobsa-ratelimit midna /tmp/golang-github-jacobsa-ratelimit master $ git tag debian/0.0_git20150723.0.2ca5e0c-1 midna /tmp/golang-github-jacobsa-ratelimit master $ git push git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git --tags master pristine-tar upstream Total 0 (delta 0), reused 0 (delta 0) To git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git * [new tag] debian/0.0_git20150723.0.2ca5e0c-1 -> debian/0.0_git20150723.0.2ca5e0c-1 midna /tmp/golang-github-jacobsa-ratelimit master $
Thanks for reading this far, and I hope dh-make-golang makes your life a tiny
bit easier. As dh-make-golang just
entered Debian unstable, you can install it using apt-get install
dh-make-golang. If you have any feedback, I’m eager to hear it.
I run a blog since 2005, spreading knowledge and experience for almost 20 years! :)
If you want to support my work, you can buy me a coffee.
Thank you for your support! ❤️