Our feedback tool (input.mozilla.org) has great data but I found some of the analysis tools lacking. I wrote a new (ugly) python script to generate CSVs from input data. I then feed the CSVs into a spreadsheet program to visualize:
This has already become pretty valuable to me for looking at overall feedback, positive/negative for a particular version or query, etc.
It’s a bit rough / hacky, there may be bugs, but you can find it here:
http://hg.mozilla.org/users/clegnitto_mozilla.com/release_tools/file/default/scrape_input.py
The tool requires:
- Python (v2.6+, which is used on Mac 10.6+)
- The module “requests“. Install it with ‘sudo easy_install requests‘ or ‘sudo pip install requests‘
- The module “argparse” (should be in python 2.6+). Install it with ‘sudo easy_install argparse‘ or ‘sudo pip install argparse‘
The tool currently dumps the CSV to stdout, so you’ll most likely want to redirect output to a file.
Input now supports OR queries
When writing the tool I realized I needed to support searching for multiple terms. For example, when trying to get feedback on hangs just searching for “hang” wouldn’t give me the complete picture. I needed to search for something more like “hang OR freeze OR responding”. I supported this in my tool with queries for each search term and aggregating the results. I quickly realized there were duplicates in the aggregated counts (input doesn’t give me unique ids to de-dupe) so I would only get an upper bound.
I hopped into #input on irc.mozilla.org and asked about OR queries. Dave Dash said it’d probably be easy to support, disappeared for a bit, and then BAM, input supported OR queries (using ‘|’ in the web UI, ‘,’ in my tool)! Thanks Dave!
Pivoting on Firefox version (–version)
Details:
- Show input types and ratios
- Allows you to see general “quality”…(praise/issues) is essentially a quality index
- You can restrict the analysis to a particular search if you want by using ‘–search’
- Beta and alpha versions are folded into the main version number
Example usage:
- python scrape_input.py –version 8.0
- python scrape_input.py –product mobile –version 8.0
- python scrape_input.py –version 7.0 –search memory
- python scrape_input.py –version 6.0 –search hang,freeze,responding
Pivoting on input type (–type)
Details:
- Track input over the life of a product
- Breaks down input by version and overall
- You can restrict the analysis to a particular search if you want by using ‘–search’
- Beta and alpha versions are folded into the main version number
Example usage:
- python scrape_input.py –product mobile –type issues
- python scrape_input.py –type all
- python scrape_input.py –type issues –search hang,responding,freeze
Manually graphing in Numbers
- Open the CSV file
- Select the first row and make it a header row
- Select the date column and other columns to graph
- Choose “Share X values” after selecting the data in the gear menu
- Click the chart button then scatter chart button
- Open the inspector (View > Show Inspector)
- Click the chart tab
- Click the series subtab
- Data symbol -> None
- Connect points -> Straight
Have fun!