Questions for Google AMP
While writing the article for David Walsh’s blog I had posted a version of the article on Medium to have @cramforce, the lead on Google AMP, answer the questions brought up. Here is a link to the article where he posted those answers: bit.ly/1VCLBr0
As developers we know that site speed is extremely important. In recent years, the developer community has created and supported new browser specifications and techniques to enhance site speed. So when I heard that Google was launching the Accelerated Mobile Pages (AMP) project I got really excited and wanted to immediately begin using the standards Google set to make sure that the sites at Aristotle Interactive were performing the best they could.
The developers, designers, and project managers at Aristotle have worked very hard over the last year to create a site speed specification document and performance budget to ensure that every website we build is fast without sacrificing design or functionality. Naturally, we wanted to use AMP to make our clients' sites even faster, but before we started changing code we wanted to do some testing to explore the benefits of using AMP, aside from the SEO boost. The tests raised a question among our team and we wanted to post this question out to the community to start a discussion: Is AMP really needed to increase site speed?
First, let's look at the results of the tests. We used a tool called WebPageTest to check the site speed of every page below. We tested the pages using an iPhone 6 running iOS 9, simulating a 3G connection (1.6 Mbps/768 Kbps 300ms RTT). We tested each page 5 times and have shown the median result in this article. Our performance team at Aristotle uses 3 metrics that have proven to measure site speed and user experience: Time to First Byte, Speed Index (Perceived Performance), and Page Size. We tested pages that were displayed on the AMP demo link that Google provided (http://g.co/amp), AMPs "in the wild" that we found through Google searches, and then a few sites Aristotle has recently created.
Google AMP Demo Results
| Site | Time to First Byte | Speed Index | Page Size | WPT Test |
|---|---|---|---|---|
| NY Times | B | 6.44 secs | 649 kb | link |
| USA Today | B | 7.42 secs | 1,962 kb | link |
| New York Post | A | 6.72 secs | 443 kb | link |
| Mashable | A | 6.43 secs | 593 kb | link |
| The Verge | A | 5.69 secs | 465 kb | link |
"In the Wild" AMP Results
| Site | Time to First Byte | Speed Index | Page Size | WPT Test |
|---|---|---|---|---|
| Buzzfeed | D | 5.53 secs | 3,882 kb | link |
| Wordpress Blog | A | 5.78 secs | 324 kb | link |
| SE Round Table | A | 5.60 secs | 1,007 kb | link |
| BBC | A | 5.58 secs | 389 kb | link |
Aristotle Interactive Results
| Site | Time to First Byte | Speed Index | Page Size | WPT Test |
|---|---|---|---|---|
| Fort Smith * | A | 5.77 secs | 893 kb | link |
| Little Rock * | A | 8.30 secs | 754 kb | link |
| ASPSF | A | 7.24 secs | 411 kb | link |
| Beacon Legal Group * | A | 6.78 secs | 761 kb | link |
* means the redesigned site is nearly launched and a link is not available.
There is a difference between the content on the sites Aristotle has created and the Google AMPs. Our sites are tourism sites while the AMPs are news articles. We expected the tourism sites to be much larger in size and slower in speed considering the amount of images and Javascript used to create interactive experiences but found that wasn't always the case.
Questions
- The biggest question: why are pages such as Buzzfeed and USA Today boosted and preloaded on a user's device, but not pages like Fort Smith and ASPSF (unless they were to use AMP)? AMPs appear to load quickly because there are some assets that are preloaded from the page. Doesn't it make most sense to give an SEO boost pages that inherently meet a standard? We should reward the pages (and publishers) that build site speed into their site as a feature.
- Since there are assets that get preloaded from the Google search results page, is there a cap on how many kilobytes get downloaded from there? Sites like Buzzfeed that appear to abuse visitors' data usage could be a problem if you are preloading assets.
- It appears that some sources are saying that publishers should "have to maintain at least two versions of any article page: The original version of your article page that users will typically see, and the AMP version of that page." Since most SEO sources recommend a responsive site over a separate mobile site and most sites have spent a lot to make that happen, doesn't it seem like taking 2 steps back to have a separate AMP site? This could get complicated when you consider Facebook Instant Articles and other services that want to accomplish the same goal.
- There are a few nice features that AMP offers, such as how images and assets are loaded and displayed, but wouldn't it be better to standardize a way of doing these things rather than doing it a different way for each service?
Services like Google AMP and Facebook Instant Articles appear to be setting themselves up to award publishers that use "proprietary" markup (meaning not standardized markup), rather than awarding publishers who build their sites with performance as the foundation. Is that really the web we want to promote? What are your thoughts on Google AMP and other services like it?
Share your thoughts and get others involved by sharing this post.
About Matt Shull
Data Science Program Manager at Thinkful. Writes a lot about performance, Vue/Nuxt, and IoT. Loves tweeting and retweeting practical development advice.
Thanks for sharing this!
You might already have read about it, Tim Kaldec and Yoav Weiss had similar thoughts about AMP and have started a project for a Content Performance Policy. I tried to summarize all of this here and to bring my own opinion about it if you might be interested: http://blog.dareboost.com/en/2016/02/content-performance-policy-for-a-faster-web/
About your question #2, do you have data about “Sites like Buzzfeed that appear to abuse visitors’ data usage”? I totally missed this issue.
Hey Damien,
Thanks for your comment. I haven’t heard of the Content Performance Policy but just briefly looking at it, it sounds like that’s where we should be heading.
As for question #2, right now it’s just a question. We see from the speed test that it still downloads ~3900kb. My concern with Google prefetching assets or preloading/prerendering pages is that it could get out of hand if those assets are still large. That’s why I’m wondering if there is a limit to the amount of data that will be prefetched.
While writing the article for David Walsh’s blog I had posted a version of the article on Medium to have @cramforce, the lead on Google AMP, answer the questions brought up. Here is a link to the article where he posted those answers: https://medium.com/@mattshull/questions-for-google-amp-a6c1963b13cd#.1oex8smrh
It’s a pretty straightforward arrangement. If you want to benefit from a CDN for the sake of delivering your web pages to your users quickly, you need to either pay or jump through your vendor’s hoops.
The part that smells is that Google prioritises AMP articles in Google Search. It’s extremely good idea to prioritise speedy articles, but not such a good idea to only prioritise articles using AMP.
AMP may be very good at what it does, but if somebody creates a performant website using a CDN offered by another vendor, it should also be prioritised as a speedy article. Google can naturally argue that its Search is its product and it has every right to prefer users of its other products, but it’s still a cop out.
A good example of how speedy web pages should be treated in Google SERP is that of businesses and Google Maps. Businesses are free to set themselves up with Google MyBusiness and can jump through the hoops of address verification to get themselves on Google Maps. However, well known establishments are also on maps without having been registered (indicated by such available actions as “Claim this as your business” or something to that effect).
Likewise, all speedy pages should be featured regardless of whether they jump through Google’s hoops if they meet the same criteria of high performance, mobile friendliness and so on. Google AMP must of course justify its existence, since CDN hosting is not free and it’s quite different to using something like Google’s Closure JS library catalogue. Google AMP could obviously become a paid service, but that’s not very friendly either.
It is in Google’s best interests to figure out a way to deliver advertising which doesn’t piss off users to the point of using ad-blocking browser extensions/apps, and AMP represents a way to do that. An incentive all around is to mitigate the threat to ad revenue posed by ad-blockers.
Personally, I think advertising is a shitty phenomena, but there’s no way to pay for any of this without it. People aren’t going to pay an annual “internet license” as they do for the BBC, and even if they did, it wouldn’t come close to paying for the costs of the modern web infrastructure.
Ultimately, I believe that the benefits of proprietary markup are justified if it enforces responsible/unobtrusive advertising, but at the same time, this doesn’t make sense as this should be the global default. No one should require a vendor JS library/delivery channel to deliver an unobtrusive advertising experience. There should be limits on what advertisements can do (advertisements should not have any JS execution involved) and best practices for loading advertisements asynchronously should be followed.
As long as AMP represents an open source means to offer advertising in an enforced way, I do not think it represents an evil. The only questionable factor is, as I have said, the preferential treatment to other performant web pages, which is tantamount to forcing users to use Internet Explorer by default.
Fortunely Matt Shull, WordPress have AMP plugin that very easy to use, but comment form disappear maybe a low experience for user cant comment.