Here’s a cool visualization of bug activity on bugzilla.mozilla.org over the past 2 days:

[external-vfe name=”bugs”]

And here’s the same activity, zoomed in on the “Core” cluster/branch:

[external-vfe name=”bugs_core”]

I sped up the videos a bit so that every 30 seconds in the video is equal to a day of activity in reality.

Background

I saw David Humphrey’s post using gource to visualize source repositories. I looked at it and thought “That’s neat, it would be cool to have it in real-time on the screens in the Mozilla offices!”

Implementation

Luckily gource supports reading a simple pipe-delimited format from stdin, which makes integration with outside tools trivial. I wrote a 20 line python script that uses my pet-project pulse to pump hg.mozilla.org push events into gource. It wasn’t super exciting though, as there isn’t a ton of pushes happening (even with the try repository).

I thought about it a little bit and realized that pulse also has Bugzilla data flowing through it. I decided it’d be really cool to (ab)use gource to visualize bug activity.

First, I determined the mappings from bug activity to repository activity:

  • File added -> new bug (“bug.new” for the pulse routing key)
  • File modified -> bug changed (“bug.changed.#” for the pulse routing key)
  • File deleted -> closed bug (searching for certain states in “bug.changed” messages)
  • Committer -> user creating or changing the bug
  • File path -> /[Bug’s product]/[Bug’s component]/[Bug’s id]

Constructing the “path” in this way makes bugs cluster in a coherent way. Realtime still wasn’t super exciting (for pulse reasons I will not go into here), so I let the tool run for a couple days, dumped the resulting messages to a file, and pointed gource at it.

If you want to play around with it (gource is interactive), I’ve uploaded the data here. The user/pass is nospam (there are email addresses in the file so I didn’t want to leave it wide open). I used these gource options to make the video:

gource --log-format custom --hide bloom,filenames --user-scale .5 -s 30 /path/to/data.txt

Future plans

I would love for someone to write a visualization tool using canvas or WebGL. I don’t have time to do it with all the Firefox release work. If you want to try to tackle this, I can provide ample help getting it hooked up to pulse’s data stream though.

I also looked a bit at code swarm, as I think it would provide better visuals for bugs. Rather than committers I would focus on bugs, with the change types (cc added, comment added, fields changed, etc) as the different colored dots. If I get time I’ll run the same data through cod swarm and see what looks better. I won’t be able to do realtime with code swarm though, as it uses an XML file format.

As an aside, this is why I am so excited about pulse. Having the data in an easily consumable stream unlocks the potential for tools we haven’t even thought of, generally with minimal development work.

Tagged with:  

7 Responses to Bugzilla activity visualization

  1. Benoit Girard says:

    I would love to see something like this in canvas/WebGL. Is there an easy way to pulse information in the javascript?

  2. reed says:

    What happens if a component or product includes a slash (‘/’) as part of its name? 🙂

  3. Gerv says:

    Awesome 🙂 And thanks for providing .ogvs and .webms 🙂

    Gerv

  4. Wow, that’s really awesome, having things like flicker around realtime in Mozilla offices would really make a nice impression (even if it means that things happen way slower, but maybe we’ll just need some generic visualization of all pulse data (and a really large screen) in the end to make it provide full awesomeness 😉