New tool to generate CSVs from input.mozilla.org data
Our feedback tool (input.mozilla.org) has great data but I found some of the analysis tools lacking. I wrote a new (ugly) python script to generate CSVs from input data. I then feed the CSVs into a spreadsheet program to visualize:
This has already become pretty valuable to me for looking at overall feedback, positive/negative for a particular version or query, etc.
It's a bit rough / hacky, there may be bugs, but you can find it here:
http://hg.mozilla.org/users/clegnitto_mozilla.com/release_tools/file/default/scrape_input.py
The tool requires:
- Python (v2.6+, which is used on Mac 10.6+)
- The module "requests". Install it with 'sudo easy_install requests' or 'sudo pip install requests'
- The module "argparse" (should be in python 2.6+). Install it with 'sudo easy_install argparse' or 'sudo pip install argparse'
The tool currently dumps the CSV to stdout, so you'll most likely want to redirect output to a file.
Input now supports OR queries
When writing the tool I realized I needed to support searching for multiple terms. For example, when trying to get feedback on hangs just searching for "hang" wouldn't give me the complete picture. I needed to search for something more like "hang OR freeze OR responding". I supported this in my tool with queries for each search term and aggregating the results. I quickly realized there were duplicates in the aggregated counts (input doesn't give me unique ids to de-dupe) so I would only get an upper bound.
I hopped into #input on irc.mozilla.org and asked about OR queries. Dave Dash said it'd probably be easy to support, disappeared for a bit, and then BAM, input supported OR queries (using '|' in the web UI, ',' in my tool)! Thanks Dave!
Pivoting on Firefox version (--version)
Details:
- Show input types and ratios
- Allows you to see general "quality"...(praise/issues) is essentially a quality index
- You can restrict the analysis to a particular search if you want by using '--search'
- Beta and alpha versions are folded into the main version number
Example usage:
- python scrape_input.py --version 8.0
- python scrape_input.py --product mobile --version 8.0
- python scrape_input.py --version 7.0 --search memory
- python scrape_input.py --version 6.0 --search hang,freeze,responding
Pivoting on input type (--type)
Details:
- Track input over the life of a product
- Breaks down input by version and overall
- You can restrict the analysis to a particular search if you want by using '--search'
- Beta and alpha versions are folded into the main version number
Example usage:
- python scrape_input.py --product mobile --type issues
- python scrape_input.py --type all
- python scrape_input.py --type issues --search hang,responding,freeze
Manually graphing in Numbers
- Open the CSV file
- Select the first row and make it a header row
- Select the date column and other columns to graph
- Choose "Share X values" after selecting the data in the gear menu
- Click the chart button then scatter chart button
- Open the inspector (View > Show Inspector)
- Click the chart tab
- Click the series subtab
- Data symbol -> None
- Connect points -> Straight
Have fun!
A newbie’s adventure with xulrunner
Fresh off the release of Firefox 3.6.10 and 3.5.13, I was thinking about tools I could write to make releases smoother. I decided to make an update snippet viewer, as even catlee (of RelEng fame) said he would like such a tool. I would be able to stop bugging the RelEng folks for information about what snippets are live where, diff between them, look for builds that don't get snippets offered, etc.
I decided to write it first as a web tool. I then changed my mind and decided on a command line tool in python. I thought about it yet some more and decided to make a graphical client with python and some sort of cross-platform widget toolkit. I then realized that this would be a great little project to learn the Mozilla platform and decided to base it on XUL and Gecko.
These are quick notes of my adventures.
First, I search for xulrunner on Google. I'm not sure if xulrunner is what I should be looking for, but I've read Planet Mozilla long enough that I figure it probably is. If not, it's likely a good place to start.
The first link that comes up is https://developer.mozilla.org/en/XULRunner. That's good, it's on MDN. I read the text, which strikes me as pretty opaque for a newbie. I figure it's ok as this is fairly technical stuff.
I click on the first tutorial. It talks about building or downloading xulrunner. I figure I'll build as I already have all the Firefox trees checked out locally. I look at the build directions and OH MY GOD I am just going to install it.
So, I download and unzip the sdk. The sdk is a bunch of files (headers and such). WTF? I was promised an installer on the previous page! Whoops, it looks like I need the runtime, not the sdk. I download and install the runtime on my Mac via the installer easily. I (naively?) thought I was developing and would need the software development kit to, you know, develop. Hm.
Following along with https://developer.mozilla.org/en/Getting_started_with_XULRunner, it gives me instructions for Windows. Ummm, ok, is it the same for Mac? Do I use ini files or plists? I have a Linux VM I could use instead, why are there no Linux instructions either? I start looking elsewhere for more info.
The getting started page says most info comes from an awesome tutorial at http://web.archive.org/web/20080415114236/http://blogs.acceleration.net/ryan/archive/2005/05/06/1073.aspx. Whoops, the page content doesn't load. Well, that's embarrassing and not at all useful.
I go back to the main page and check out another tutorial link. That's broken as well! I get the sinking feeling I have hit a deserted corner of the web and Mozilla platform. The dead link actually looks to be the same tutorial as above: http://blogs.acceleration.net/ryan/archive/2005/05/06/1073.aspx. A blog post from 2005? Yikes.
The next link in the main wiki page is http://zenit.senecac.on.ca/wiki/index.php/XULRunner_Guide. It looks promising. I go to download the PDF and....parked domain link page. I check the wiki to see if some spammer changed links. Nope, it just looks like someone let the domain expire and the PDF is gone.
So I contemplate stopping at this point but decide to power through with the Windows instructions. I start creating the necessary directory structure. It's tedious so I scroll down to see how much I have to do. Ah, there's sample code to download! That would have been helpful at the top.
So now that I have a project, how do I run this thing on Mac OS X? The wiki is no help. I manage to dig up http://simile.mit.edu/wiki/Crowbar from the list of applications that use xulrunner and notice they have a command to use. Great success! I run the command.
Hmmmm, there's no "Myapp" in the Application directory to launch as Crowbar's directions implied there would be. I sort the Applications directory by mod time see there's a new "Finkle" subdirectory.
Finkle is Einhorn?
Ummm, what? Whatever, I roll with it. I double click the "Myapp" application in the Finkle directory. I get an application with no icon and no window that quits immediately:
"Could not find compatible GRE between version 1.8 and 1.9.0.*".
Ok.....time to look in the sample project. I crack open the ini to see if I can solve the issue. Ah, that's where Finkle came from:
Vendor=Finkle
Lots of Mark Finkle stuff in there. Uhhh, I appreciate the work Mark, but I was expecting a more general "Hello World" project 
I read further down, O HAI!
MinVersion=1.8 ; ; This field is optional. It specifies the maximum Gecko version that this ; application requires. It should be specified if your application uses ; unfrozen interfaces. Specifying 1.8 matches all releases with a version ; prefixed by 1.8 (e.g., 1.8a4, 1.8b, 1.8.2). MaxVersion=1.9.0.*
I am versed enough in Gecko version numbers to know that these are old as dirt™. Others may not be as lucky, but I imagine they could figure it out by the error message. I know I am running 1.9.2.9 (which is the most recent, I just released .10 tonight and xulrunner builds come a bit later) so this is likely my problem. I yank out MaxVersion and re-run:
/Library/Frameworks/XUL.framework/xulrunner-bin --install-app myapp
I launch the app in /Applications/Finkle and get the same issue.
Ruh Roh!
I have no clue how to fix the issue if it isn't caused by those versions! I glance at the application's timestamp and notice it's 10 minutes in the past. WTF, I thought the command above writes the app? It worked the first time at least. I manually delete the Finkle directory and run the above command again.
Take THAT xulrunner!
That worked! Take THAT xulrunner indeed! Interestingly, the launch seemed to crash and then restart--there was a flicker of the window and icon shuffling in the dock. Well it works at least and I can't reproduce the issue on subsequent launches.
Ok, where to next? I head back to the main xulrunner wiki page. The next 2 links don't give any info. Uhhh, ok, I guess I'll look at tips even though I am pretty sure I need more than tips at this point. The tips wiki page looks to have some good info (especially about branding) but I am not at the point in my app where I can use any of them.
I decide to poke around in the sample "Myapp". Nothing really there, darn. Most files don't even have helpful comments to guide me where to play around or add/extend. And then I come across main.js. Woahhhhhh, main.js looks super complicated for this hello world app! Oh, it looks like it's from FF? Why is it in here? Mark, what's going on here? 
At this point I decide that I'm probably on to normal XUL now and should start looking for general platform documentation. I decide I am way too tired to work on this and put it away for later.