Our feedback tool (input.mozilla.org) has great data but I found some of the analysis tools lacking. I wrote a new (ugly) python script to generate CSVs from input data. I then feed the CSVs into a spreadsheet program to visualize:

 

This has already become pretty valuable to me for looking at overall feedback, positive/negative for a particular version or query, etc.

It’s a bit rough / hacky, there may be bugs, but you can find it here:

http://hg.mozilla.org/users/clegnitto_mozilla.com/release_tools/file/default/scrape_input.py

The tool requires:

  • Python (v2.6+, which is used on Mac 10.6+)
  • The module “requests“. Install it with ‘sudo easy_install requests‘ or ‘sudo pip install requests
  • The module “argparse” (should be in python 2.6+). Install it with ‘sudo easy_install argparse‘ or ‘sudo pip install argparse

The tool currently dumps the CSV to stdout, so you’ll most likely want to redirect output to a file.

Input now supports OR queries

When writing the tool I realized I needed to support searching for multiple terms. For example, when trying to get feedback on hangs just searching for “hang” wouldn’t give me the complete picture. I needed to search for something more like “hang OR freeze OR responding”. I supported this in my tool with queries for each search term and aggregating the results.  I quickly realized there were duplicates in the aggregated counts (input doesn’t give me unique ids to de-dupe) so I would only get an upper bound.

I hopped into #input on irc.mozilla.org and asked about OR queries. Dave Dash said it’d probably be easy to support, disappeared for a bit, and then BAM, input supported OR queries (using ‘|’ in the web UI, ‘,’ in my tool)! Thanks Dave!

Pivoting on Firefox version (–version)

Details:

  • Show input types and ratios
  • Allows you to see general “quality”…(praise/issues) is essentially a quality index
  • You can restrict the analysis to a particular search if you want by using ‘–search’
  • Beta and alpha versions are folded into the main version number

Example usage:

  • python scrape_input.py –version 8.0
  • python scrape_input.py –product mobile –version 8.0
  • python scrape_input.py –version 7.0 –search memory
  • python scrape_input.py –version 6.0 –search hang,freeze,responding

Pivoting on input type (–type)

Details:

  • Track input over the life of a product
  • Breaks down input by version and overall
  • You can restrict the analysis to a particular search if you want by using ‘–search’
  • Beta and alpha versions are folded into the main version number

Example usage:

  • python scrape_input.py –product mobile –type issues
  • python scrape_input.py –type all
  • python scrape_input.py  –type issues –search hang,responding,freeze

Manually graphing in Numbers

  1. Open the CSV file
  2. Select the first row and make it a header row
  3. Select the date column and other columns to graph
  4. Choose “Share X values” after selecting the data in the gear menu
  5. Click the chart button then scatter chart button
  6. Open the inspector (View > Show Inspector)
  7. Click the chart tab
  8. Click the series subtab
  9. Data symbol -> None
  10. Connect points -> Straight

Have fun!

Tagged with: