Releasing code is my thing…I used1 to ship Mac OS X updates at Apple, Firefox updates at Mozilla, theÂ web frontend at Facebook, and manyÂ mobile apps at Facebook. Because of those experiences I’ve seen a lot of what works and what doesn’t, which platforms do things better, and tweaksÂ that could improve the lives of developers shipping software.
Lately I’ve been giving talksÂ about howÂ to shipÂ mobile apps at scale forÂ conferences and startups2. While many of the experiences and challenges for popular apps like Facebook are unique, it is becoming clear that a lot of the experience shipping apps is universal. I decided to write down some of the current frustrations and suggested changes, starting with how I would change the App Store. It got so long that I broke it into a series ofÂ blog posts.
Note: I am sure there are other issues that can be fixed in the app store, particularly related to discovery and monetization. I know nothing about those, as apps I worked onÂ had brand recognition with millions of users and were free.
Many of these suggestions I have already given directly toÂ the TestFlight / iTunes ConnectÂ team3, though some are new. I’ve ordered my suggestions from what I think would be the highest impact to lowest impact. This is the firstÂ suggestion with highest impact.
🚫 Get rid of app review in someÂ situations
App review is the #1 pain I hear from app developersÂ I talk with. Not knowing when your app will ship can be devastatingÂ for your business.
During app review seemingly arbitrary decisions are frequently made, and just because an appÂ passed review last time doesn’t mean it will pass review again–even with no changes.4 Different reviewers have different standards, your app can getÂ caught up inÂ higher-level posturingÂ related to Apple’s long-term business, and worst of all the process is a black box no matter how much Apple documents it.Â While your mobile infrastructureÂ may be perfect and automated, you still throw your app over the fence to a human on Apple’s side. Your reviewer went home because they were feeling sick? Your app ain’t getting reviewed today.5
There is, of course, expedited review for when the sky is falling. The problem is determining when the sky is really falling and if you’ll regret using one on theÂ problem at hand. One unicornÂ I gave a talk at said they often have a room full of people debating if something is “bad enough” to use one of those precious lifelines. This is madness and mustÂ stop.
At Facebook we were not exempt from app review, contrary to popular belief. In fact, I would say widely-installedÂ appsÂ like Facebook are held to even higher standards.6
There should be a larger discussion about what human app review accomplishes in a world where app developers use server-side switches to turn on and off code for A/B testing and feature gating.7 CompaniesÂ like Facebook run thousands of tests every release–no two Facebook apps are alike.8 There is no way a human could test all of them or even a large subset.9 While app reviewers do get a diff of methods and such between app versions, developers can easily obfuscate if they desire.
There is noÂ technical or legal way to stop developers from tweaking code already in the App Store as long as native apps are able to use the internet and webviewsÂ are supported. Enabling new features and substantially altering existing onesÂ is only going to get more prevalentÂ as React NativeÂ use increasesÂ and services like AppHub and CodePush sprout up.Â The usefulness of Apple’s up-front review by humans on code packaged in a zipfile is quickly being eroded.
That being said, Apple will probably never get rid of app review…nor do I think theyÂ should. I think app reviewÂ should be overhauled though:
- Have aÂ verified/top developer program. Make post-1.0 apps10 shipped by these developers auto-approved if Apple’s automated review systems pass (that is, remove humans from the process). ThisÂ switches app review from anÂ adversarialÂ skeptical-by-default model to aÂ trust but verify model.
- App review is mainly there to protect people from getting taken advantage of11, something that doesn’t generally happenÂ with popular/brand name apps and developers12. It is crazy that apps like Facebook and Google Maps go through the same process as Joe Blow’s Fart App–the risk to users is different.
- Most popular appsÂ now ship at least every two weeks and I only see the trend accelerating13. When there is a human in the process it is virtually impossible for Apple to catch up and puts a strain on Apple employees for little end-user benefit. If this change was made app reviewers could focus on more in-depth reviews for 1.0 apps, expedited reviews, and exploratory and/or verification testing from user reports.
- Allow the publicÂ to install apps while they are beingÂ reviewed.
- This would dampen the severity of a major bug, as the app developer could instruct more adventurous people to install an in-review version if they are having issues or curious users could try it if they are experiencing problems.
- This would also help with major app or feature launches. Currently it is a pain coordinating press and ad spend because you don’t know when your app will be approved and available in advance. Â You can of course wait until the app is approved and hold for developer release, but because the App Store’s search index is only updated once a day14 it may not be available for users to find without a direct link. Allowing people to install apps still in review would make the coordination a lot easier and timing less critical.
- This could also help keep app quality high by approximating a “slow rollout” strategy.
- No app review for beta apps.Â App review for beta appsÂ is stupid, regardless of how quickÂ an appÂ is approved.
- Beta users know they are getting something that is potentially harmful to their device and data and can easily fallback to the production version whenever they want. Apple should be focusing their resources / control on production, not beta.
- Because Apple only reviews the initial version of an update,15 once you’ve gotten a beta build approved you can resubmit without review as much as you like.16 The beta review doesn’t actuallyÂ prevent anything from shipping to beta testers and instead merely adds bureaucratic overhead.
- Without this the value of beta as a rapid feedback channel is extremely limited.17
- If this change was made app reviewers could focus on more in-depth reviews for 1.0 apps, expedited reviews, and exploratory and/or verification testing from user reports.
Based on my experience and discussions these changes would help virtually every developer out there–big and small–without giving up control or allowing substandardÂ apps into the AppÂ Store.
Up next are some suggested changes toÂ TestFlight beta testing.
- I am now doing my own stealth startup in this space. ↩
- If you want me to talk at yours, email me at my first name at my last name.com or contact me on Twitter @LegNeato. ↩
- If anyone at Apple needs more context or wants confidential information, people in the Program Office know how to get in touch with me. ↩
- I have so many war storiesÂ I could tell… ↩
- Some apps have dedicated reviewers, others are handled by a team. Every developer is affected by this during the annual holiday shutdown however. ↩
- Any badness affects a large number of users. Apple also knows smaller developers will copy the bigger ones and they don’t want to deal with all the annoying “Facebook did it too” app submissions. I am not sure if it is official policy but Apple often appears to try to treat all developersÂ equally. This means even thoughÂ apps like Facebook are exceptional (integrated in the OS, installed on most phones, drives many App Store installs for other apps, makes a lot of money) they do not really get official exceptions in the review process. ↩
- Hat tip to Ryan Nielsen for pointing out I should mention this. ↩
- This causes a lot of angst for product managers who joinÂ Facebook and are used to a central list of everything going on. ↩
- This is partly why FacebookÂ invests in tooling, beta channels, and slow rollout so heavily. ↩
- New/1.0 apps would still be reviewed. ↩
- Of course there are other strategic reasons for Apple to have an upfront say (note that app quality is NOT one of them), but they’d still have their after-the-fact veto if anything threatened their business. ↩
- Here is where cynicalÂ people will scoff, but it is true. Taking advantage of people’sÂ trust would be bad for business at the very least. Most companies with popular appsÂ like Facebook, Snapchat, Google, Twitter are also audited byÂ government privacy watchdogs and organizations like the FTC. ↩
- There are too many benefits to the business and we’ve been spoiled by the web’s deployment model. ↩
- It is insane thatÂ Facebook and Google can index the internet in basically realtime but Apple doesn’t appearÂ to be able to index their structured data on demand. ↩
- Pointed out by my ex-colleague Colm Doyle on Facebook. ↩
- As longÂ you don’t bump the public build number. ↩
- Facebook usesÂ alpha and beta in Google PlayÂ religiously. TheyÂ haveÂ over 100,000 alpha testers that get the code off master every night and over 2,000,000 beta testers that get the code off a release branch every night. It’s all automatedÂ andÂ the world has not exploded. ↩