Firefox release candidate builds available
In case you missed it, late yesterday we declared a release candidate for the next version of Firefox. Please test it!
Blockers, softblockers, betas and you
Summary
- The betaN softblockers will be punted out of Firefox 4 very soon
- Please look over the betaN softblockers by THIS FRIDAY, 9:00 am PST, looking for bugs that should be upgraded to hardblocking status
- If you find a softblocker that needs to make Firefox 4, do the following:
- Remove "[softblocker]" from the whiteboard
- Add justification to the bug
- Toggle the "blocking2.0" flag to "?" (aka nominate it for blocking)
Details
The betaN softblockers are bugs that have been previously identified as:
- really nice to have for Firefox 4 but won't stop us from shipping (the "softblocking" part)
- needing beta coverage to make sure the fix is correct and doesn't cause new issues (the "betaN" part)
Regardless if beta 12 is the last beta, we are rapidly approaching the point where there will be no more betas. Once we freeze for the last beta, non-blocking bugs that require beta coverage are implicitly no longer approved.
We ask that interested developers make sure our soft/hardblocking and beta coverage decisions are sound before we freeze for the final Firefox 4 beta. Additionally, we understand that new information may have come to light (such as reproducibility, impact, etc) and want to make sure we are making decisions with the correct data.
Please take a moment to look over the betaN softblocker list by THIS FRIDAY, 9:00 am PST, looking for bugs that should be upgraded to hardblocking status.
If you find a softblocker that needs to make Firefox 4, do the following:
- Remove "[softblocker]" from the whiteboard
- Add justification to the bug
- Toggle the "blocking2.0" flag to "?" (aka nominate it for blocking)
To be warned, the bar for a hardblocker is extremely high at this point, especially for bugs that have been marked softblocking so long.
After this Friday at 9:00 am PST we reserve the right to remove the blocking status of the betaN softblockers (and thus automatic approval) at any time. Once we do so, approvals will be on a case-by-case basis.
Mozilla-central landings today and the beta 11 plan
As discussed in today's platform meeting and decided by drivers after reviewing current information, this is the plan for beta 11 and how it relates to mozilla-central development work.
Beta 11
We are pushing to get these bugs in by EOD today (PST):
If you are on the hook for any of those (including reviews and feedback) they are your number one priority.
If you are landing bugs NOT in that list, please think about the risk to beta 11. If anything is even slightly risky, please hold off a day until we branch for beta 11 before landing on mozilla-central. We will not close mozilla-central and instead trust developers to act responsibly with risk.
Once we have all / the the majority of the bugs in the query above we will call a build using the last green changeset.
Beta 12
The current plan is to build when the remaining betaN hardblockers are done. This is heavily dependent on fix and blocker creation rate. I will be doing some analysis to give people a possible date range, but as far as development is concerned everything is the same (fix as many betaN hardblockers as quickly as possible).
As usual, feel free to contact me on IRC or via email if there are questions/concerns.
Mining input.mozilla.com for fun and profit
I've been looking at Aakash's awesome Firefox Input site to try to figure out how we are doing with Firefox 4. Unlike others who have sifting through individual reports and writing bugs, I've been trying to look at the data in aggregate. I did some adhoc queries over the lifespan of the Firefox 4 betas and thought the results were interesting. Of course, this is not scientific in any way but I think the insights are still valuable. I think some sort of triggers / alarms doing this analysis automatically should be added to input (bug coming soon).
What prompted me to look specifically at input data was a mention of bug 628872 on Twitter. It seemed like a bug that should block Firefox 4 but I wanted to know the extent of the problem. Rather than try to reproduce I went to input and saw the following:
Though likely not statistically significant, there has definitely been an uptick in negative feedback containing "iplayer". This graph even helps to narrow down a regression timeframe. Recognizing the usefulness of this approach, I did a bunch more queries that came to mind.
I first decided to see if the YouTube player had a similar graph:
From the graph it is easy to tell that YouTube feedback has been relatively consistent, spiking every time we release.
In weekly Mozilla meetings there has been talk about a bug on Hotmail's side causing problems for Firefox 4 beta users. Searching for "hotmail" I got:
Clearly users are feeling the pain and letting us know. Similarly, one of the top issues discussed in support reports has been copy and paste. Searching for "copy paste" gave me:
We're on the case (bug 613915) and need to fix the issue before final.
Next I thought about bug 626016 (which is about Facebook chat) so I searched for "chat":
The two spikes are interesting. My completely uninformed guess about the spike on August 12 is Facebook (or some other chat) going down or a server-side website release that went wrong and was quickly rolled back. The spike on the right is likely bug 626016.
Up next I looked at "netflix":
This is another interesting graph. The left spike was likely due to the known issue of bad user-agent detection on Netflix's side (bug 522957). The increased displeasure on the right is likely due to bug 598406. From that known issue "hulu" had similar sniffing problems which look to have been resolved:
This method can also be used to gauge general user sentiment. I knew the removal of the status bar was contentious, so I pulled up the graph for "status":
Clearly you can see the initial displeasure when the change landed in a release and the resulting dropoff. Of course, there is still a level of sustained feedback which has prompted some additional product changes.
Finally, I searched for "apple" with no bug particularly in mind. I was pretty surprised to find this graph:
The spike seemed to spike and fade too suddenly to be a Firefox issue. I did a quick Google search for "apple october 21" and immediately saw what was going on. On the 23rd Apple reported their earnings. Such an event wouldn't normally impact Firefox in any way, but Apple live-streams their earnings report. Because Apple is heavily invested in H.264 they streamed it using that technology. Firefox doesn't support H.264, Firefox users couldn't access the stream and thus were complaining. The complaints were only relevant while the live stream was relevant and disappeared the next day. Fascinating!
I found this sort of analysis interesting and thought provoking and am glad I have a tool like Firefox Input available to me (and the world!).
Updated Firefox 4 beta plan
This is the updated beta plan as discussed in the Platform meeting earlier today. It is a modification of the previous plan based on current bug counts and fix rates. Please email me if you have any questions or concerns.
The Updated Beta Plan
- If all betaN hardblockers are done by this Friday afternoon PST (2011-01-28) we will build beta 11 at that time. This is essentially staying on the previously posted plan
- If on Friday there are remaining betaN hardblockers we will not build and instead defer the decision until Tuesday (2011-02-01)
- If all betaN hardblockers are done by next Tuesday afternoon PST (2011-02-01) we will build beta 11 at that time. There will be no planned beta 12
- If on Tuesday there are remaining betaN hardblockers we will make a judgement call
- If on Tuesday the remaining betaN hardblockers are on track to land before Friday (2011-02-04), we will not build on Tuesday and instead wait for Friday. There will be no planned beta 12
- If on Tuesday the remaining betaN hardblockers look at risk to land by Friday (2011-02-04) we will create beta 11 on Tuesday and plan on having a beta 12. Beta 12 will not have a fixed schedule and will instead be built when the remaining betaN hardblockers are at zero
Bug Fix Priorities
[risky] betaN hardblockers > betaN hardblockers > final hardblockers > betaN softblockers > final softblockers
Again, feel free to contact me with any questions or concerns.
Firefox 4 beta logistics
This is the current working Firefox 4 beta plan so you can plan accordingly. Feel free to email me directly with any questions.
Summary
- We'll take the most recent green changeset from mozilla-central this Friday, 2011-01-21 at 2:00 pm PST and call that beta 10
- The tree will close for < 15 minutes for this (if at all)
- We'll freeze nightly updates at or around the changeset we branched from (likely only for the weekend)
- Any problems with the beta will be dealt with on the relbranch
- We will release beta 10 as soon as possible during the week of the 24th
- We'll take the most recent green changeset from mozilla-cental on Friday, 2011-01-28 at 2:00 pm PST (or sooner if the remaining betaN hardblockers are finished) and call that beta 11
- The tree will close for < 15 minutes for this (if at all)
- We'll freeze nightly updates at or around the changeset we branched from (likely only for the weekend)
- Any problems with the beta will be dealt with on the relbranch
- We will release beta 11 as soon as possible during the week of the 31st
Details
Both frontend and platform managers have determined that their betaN hardblockers will not all be finished by this Friday. Additionally, greater than 318 bugs have been marked fixed since beta 9. We need beta coverage for most of those fixes but also need beta coverage for the remaining betaN hardblockers. To control risk for the RC the plan is to do two betas (10 and 11) rather than hold beta 10 for all betaN hardblockers. This will give us a week of beta coverage for the 318 bug fixes while still allowing beta coverage for the remaining betaN hardblockers.
Code freeze for beta 10 and 11 will also be a bit different. Previous betas closed the tree to stabilize and reduce beta risk. At this point in development, every day counts--we cannot shut down mozilla-central over the weekend as we have done before. Instead we'll freeze what's in the tree and run with it, dealing with any beta issues on the relbranch. Instead of metering what goes in before a freeze we will be driving to get as many fixes as possible so they can get more beta coverage.
To enable additional testing of the bits we intend to ship to beta while keeping the tree open for development, we will freeze mozilla-central nightlies to the changeset we branched from (likely only for the weekend). Developers can still use hourlies to test and will able to integrate their work with others without waiting a weekend for the tree to reopen.
For those curious, joduinn has provided the the specific plan for the nightly changeset freeze:
- RelEng will create a nightly ~2pm PST Friday, with same changeset as the beta10 builds
- RelEng will create the nightly build fri+sat+sun night as usual, but will force the updates to a hidden location. A human can download+install those nightly builds if they want (for regression ranges / reproduction), but no-one will get updated to these nightlies
- On Monday, release drivers will send the "ok to reenable normal nightly operation". RelEng will revert changes and trigger a mid-day Monday nightly so people on Friday afternoon nightly will directly update to Monday afternoon (skipping the fri/sat/sun night nightlies)
- We will then move snippets generated over weekend back to usual locations, and cleanup
- The change is a small, and we believe safe, change to how updates are created
Explicit Bug/Release Dependencies
- Beta 10: None, time-based release to test what we have done since beta 9
- Beta 11: BetaN hardblockers
- RC: Final hardblockers
I'll keep the various mailing lists and planet updated if anything changes. Again feel free to shoot me an email (or find me on IRC) with any questions you have.
Firefox 4 beta 9 code freeze is TOMORROW
As mentioned in meetings (and planet via the meeting notes), the Firefox 4 beta 9 code freeze is tomorrow, 2011-01-07.
- If you have a risky change, don't land it until we've branched (likely Monday, 2011-01-10)
- If you need as much beta feedback as possible for a particular fix, please get it in for beta 9 if you can
- If you have a question, please ask instead of assuming (LegNeato on irc or clegnitto@mozilla.com)
Basically, think hard about your fixes and decide if you could reasonably justify the schedule slip if they cause problems. If you can't make that decision, feel free to ask me.
I don't anticipate giving the go for builds until Monday, late afternoon PST. In the early afternoon PST I will contact assorted stakeholders and see if there is anything that would hold the release and get an estimated time for the build.
Please let me know if there are any bugs you are worried about not making the freeze (and give justification why they can't wait until the next beta). A word of caution, the bar for blocking the freeze is extremely high.
Thanks!