LegNeato! Christian Legnitto's blog about Mozilla, Apple, technology, and random stuff

GitHub AMQP integration service hook live!

As of last night you can now send AMQP messages to a message broker (like the one running on pulse.mozilla.org) for GitHub pushes and commits!

Here's how to set it up...

First, go to the admin area of one of your repositories:

Next click on "Service Hooks" on the left hand side:

After that, select the AMQP service hook:

Configure the hook to point at your server and it's done!

Once configured, the next time you push there will be messages sent via AMQP to your server from GitHub. Currently, it sends one overall push message containing all changeset info in the push as well as individual changeset messages.

Mesages are sent for the push with the following routing key format:

"github.push.#{owner}.#{repo}.#{ref}"

where:

owner = payload['repository']['owner']['name']
repo  = payload['repository']['name']
ref   = payload['ref_name']

Messages are also sent for each commit in a push, with the following routing key format:

"github.commit.#{owner}.#{repo}.#{ref}.#{author}"

where:

author = commit['author']['email']
(other fields are the same as above)

The message data is sent in JSON format.

Here's an example commit message (dumped from Python):

{u'_meta': {u'exchange': u'org.mozilla.exchange.pulse.test',
 u'routing_key': u'github.commit.LegNeato.bztools.master.clegnitto@mozilla.com'},
 u'payload': {u'author': {u'email': u'clegnitto@mozilla.com',
 u'name': u'Christian Legnitto',
 u'username': u'LegNeato'},
 u'files': {u'added': [],
 u'modified': [u'README.rst'],
 u'removed': []},
 u'id': u'4d69ae955e6f877000ecfe17def333b32973070b',
 u'message': u'Change readme to point to my repo (and a test of AMQP GitHub service hook)',
 u'timestamp': u'2010-11-22T15:16:26-08:00',
 u'url': u'https://github.com/LegNeato/bztools/commit/4d69ae955e6f877000ecfe17def333b32973070b'}}

And here's an example push message (dumped from Python):

{u'_meta': {u'exchange': u'org.mozilla.exchange.pulse.test',
 u'routing_key': u'github.push.LegNeato.bztools.master'},
 u'payload': {u'after': u'0ccf64aa593e96a19529b9c9a3b1e0098c626108',
 u'before': u'9aa20993159d5e714103abc6741b43feb371fc34',
 u'commits': [{u'author': {u'email': u'clegnitto@mozilla.com',
 u'name': u'Christian Legnitto',
 u'username': u'LegNeato'},
 u'files': {u'added': [],
 u'modified': [u'bugzilla/models.py'],
 u'removed': []},
 u'id': u'80539c359d22ca35f61c34edb810bfc9c0bef6a8',
 u'message': u'Add support for keywords',
 u'timestamp': u'2010-11-17T16:14:37-08:00',
 u'url': u'https://github.com/LegNeato/bztools/commit/80539c359d22ca35f61c34edb810bfc9c0bef6a8'},
 {u'author': {u'email': u'clegnitto@mozilla.com',
 u'name': u'Christian Legnitto',
 u'username': u'LegNeato'},
 u'files': {u'added': [],
 u'modified': [u'README.rst'],
 u'removed': []},
 u'id': u'4d69ae955e6f877000ecfe17def333b32973070b',
 u'message': u'Change readme to point to my repo (and a test of AMQP GitHub service hook)',
 u'timestamp': u'2010-11-22T15:16:26-08:00',
 u'url': u'https://github.com/LegNeato/bztools/commit/4d69ae955e6f877000ecfe17def333b32973070b'},
 {u'author': {u'email': u'clegnitto@mozilla.com',
 u'name': u'Christian Legnitto',
 u'username': u'LegNeato'},
 u'files': {u'added': [],
 u'modified': [u'bugzilla/models.py'],
 u'removed': []},
 u'id': u'0ccf64aa593e96a19529b9c9a3b1e0098c626108',
 u'message': u'Add some string representations',
 u'timestamp': u'2010-11-22T18:19:32-08:00',
 u'url': u'https://github.com/LegNeato/bztools/commit/0ccf64aa593e96a19529b9c9a3b1e0098c626108'}],
 u'compare': u'https://github.com/LegNeato/bztools/compare/9aa2099...0ccf64a',
 u'forced': False,
 u'ref': u'refs/heads/master',
 u'ref_name': u'master',
 u'repository': {u'created_at': u'2010/11/15 14:45:56 -0800',
 u'description': u'Models and scripts to access the Bugzilla REST API.',
 u'fork': True,
 u'forks': 0,
 u'has_downloads': True,
 u'has_issues': False,
 u'has_wiki': True,
 u'homepage': u'',
 u'name': u'bztools',
 u'open_issues': 0,
 u'owner': {u'email': u'clegnitto@mozilla.com',
 u'name': u'LegNeato'},
 u'private': False,
 u'pushed_at': u'2010/11/22 19:17:25 -0800',
 u'url': u'https://github.com/LegNeato/bztools',
 u'watchers': 2}}}

Now that this service exists Pulse can get messages about Mozilla checkins for projects hosted on GitHub, making Pulse the one-stop shop for real-time Mozilla data...once the Bugzilla extension, MediaWiki extension, and Mercurial extension are put into production of course.