PoP Blog

on 4 Jan, 12:55

PoP won Smashing Magazine’s Front-End Performance Challenge! Here is the winning submission

PoP won Smashing Magazine’s Front-End Performance Challenge! Here is the winning submission
Blog

A few weeks ago Smashing Magazine launched its Front-End Performance Challenge, for which contestants were asked to improve their websites to perform blazingly fast. And PoP won the competition!

Below is the winning submission. Enjoy!

The Front-End Performance Challenge: Submission by Leonardo Losoviz

Hi Smashing Magazine team! Here my submission for your challenge. Before talking about the implementations, I will explain below the context of my application.

Final results: as you will see in the tests’ section, the website loading speed is still not the greatest (I’m working on it every day, it’s a constant work in progress), however in relative terms, as in what it takes to load given the complexity/amount of requirements of the application, and what gains have been produced so far, I believe that it is quite remarkable. I hope you are also giving weight to the delta of loading time (how much it was optimized), and not just the final loading time number.

Measuring the improvements: you have requested a printout of WebPageTest/Lighthouse from before/after doing the optimizations. Indeed, I am providing you with something better: a way to measure the improvement obtained by different techniques, baked into the application itself, and configurable through a URL, so that anyone can reproduce the tests and see the gain produced by each optimization item on its own, and then combined with all the other optmizations. I will explain further in the tests’ section about this. Unluckily, as a side effect of adding this feature, Lighthouse somehow stopped reporting accurately: for instance, it says the application has no Service Workers, when it sure does. As such, I’ll limit myself to sending only WebPageTest reports.

Usability of the application: all my optimizations have been implemented for PoP, which is a framework to build websites (the PoP website itself was built using PoP, and I will mention these other PoP websites for doing the tests: https://www.mesym.com and https://agendaurbana.org). That means that all the optimization items I describe below come out-of-the-box for anyone implementing their websites using PoP. And it is open source, all code is available in this GitHub repository.

Complexity of the application: a PoP website ships with tons of features to become a social network, such as: infinite scrolling, likes, following users, subscription to topics of interest, real-time notifications, #hashtags, @mentions, creation of content from the front-end, sending messages to users, dynamic comments, up/down voting system, interactive maps, powerful search, page tabs, embeds, providing an API to share data, and others. In addition, different PoP websites can interact with each other, becoming a decentralized social network (please see https://sukipop.com as an example).

Custom-made implementations: PoP is built on top of WordPress. That means that many innovative optimization techniques available to Javascript frameworks, such as code-splitting using Webpack or Service Workers through sw-precache, are out of reach (at least so without a big workaround). As such, all the optimization techniques described below are all DIY, designed and implemented from scratch.

And with this, I detail the optimizations below 🙂

Optimizations

Low hanging fruits

  • HTTP/2
  • GZip compression
  • Minification of assets
  • CDN for assets
  • High Cache-Control max-age for static assets
  • DNS prefetching
  • Async load of all external domain scripts (eg: Google Maps API key, Google Analytics)
  • Load libraries from own domain, instead of from their own domain/CDN (eg: Bootstrap)

Removal of the front-end WordPress bloat

The application is a WordPress website, but only on the back-end. On the front-end, it uses a custom-made Javascript framework to render content. All WordPress plugins cannot directly output their content or load their assets, but they must integrate with the framework, thus streamlining and unifying code and assets, and avoiding general bloat.

Provision of an API to get the data for any page on the website

Every page in a PoP website is a self-consumed API:

  • Configuration data and database data are decoupled
  • Adding parameter output=”json” to any page’s URL switches from website to API, returning both configuration and database that for that page
  • Further adding parameter module=”data” returns only the database data for that page.

Eg: a webpage, its API with all data, its API with only DB data.

The self-consumed, highly-decoupled API architecture enables most of the optimizations explained through the article.

Reduction of the output size

Because of the decoupling, we can mangle the configuration and keep the database data intact. Eg: mangled API data, non-mangled API data.

Database data is produced in relational format, allowing to send every piece of information only once. Eg: if 2 posts have the same author, all author information, such as name, email, etc, is sent only once. In addition, the data is also queried only once from the database, all of it at the end of the request, making the application more performant on the server.

Viewing the response’s JSON code in Chrome’s Console. Pay attention how under key ‘database’ the object data is relational, and objects from different domains have also been fetched: the post’s author (with id 851) and the post’s location (with id 23494); and under key ‘dataset’ it indicates the result object ids for each block (posts with ids 23787, 23784, 23644 for the main block)

Output can be suited to the device: when a user clicks on a link, extra information is added to the request before sending it through AJAX, and hidden to the user. Through this extra information (such as device, viewport size, etc) we can implement adaptive design for mobile. This way, we can avoid sending back code that would otherwise be hidden. And because the extra information is in the requested URL, it’s easily cacheable, without having to inspect the headers. Eg: when clicking in this link, it fetches this URL.

Configuration data can be extracted from the HTML output into a JS file: instead of delivering the same configuration code time and again from the server, to serve similar pages, it can be saved as a .js file, which can be cached on the client. This process, which takes place on runtime when first accessing a page, uploads the newly-created .js file to an AWS S3 bucket and serves it through a CDN. Eg: extracting the configuration for this homepage, produces this .js file, which is 459 KB!

Aggressive caching

5 layers of caching are implemented in between the server and the client:

1. Page-caching in the server: PoP relies on plugin WP Super Cache to cache pages in the server. As an added feature, the caching can take place even if the user is logged-in. This is accomplished by lazy-loading the user state: first return the output stripped of all user state (such as what posts were “liked” by the logged-in user), and immediately after load a 2nd request in the background to fetch this information. As a consequence, most pages on the website can always be served immediately. Only exception are those pages which are all about user state, such as /my-posts/ or /edit-my-profile/, in which case the first request already returns all data, including the user state (these pages cannot be cached anyway).

2. Configuration-caching in the server: because the configuration is decoupled from the database data, it can be cached independently. Moreover, the configuration does not depend on the URL, but on the page-type: two events (/events/1/ and /events/2/) share the same configuration. Then, server processing is reduced: the first time an event is loaded, the configuration will remain cached for whenever loading any other event; only database data needs to be fetched. The configuration cache is a .json file, which can be stored in the server’s disk or, even better, in a memory caching system such as Memcached or Redis.

3. Content CDN: requests to fetch content are routed through a CDN, making dynamic content become static, since the CDN will return the cached version of that content and not need reach the server. The application accomplishes this by, on runtime, adding a versioning parameter with a timestamp as value to the requested URL, reflecting the freshness of content in the website. This technique allows the website, which is by nature dynamic since it is built on top of WordPress, to obtain the benefits of a static site generator: content is served very fast, from a CDN located near the user. Eg: when clicking on this link, the application fetches this URL (notice that the domain has changed from www.mesym.com to content.mesym.com, which points to a CDN). (I wrote an article for Smashing Magazine on how to implement it, it is currently in the queue to be published.)

4. Content and assets caching in the browser through Service Workers: the application is a PWA, implementing Offline first capabilities based on the Cache then network caching strategy. All needed assets are pre-cached when installing the Service Workers script (around 1.5 MB of files). Content is cached when accessed, so the website can be browsed offline. And an application shell handles 404 requests and offers the user to open all previously-cached pages. As a bonus, the Service Workers script helps avoid unneeded round-trips from known redirects: loading https://www.mesym.com will redirect to https://www.mesym.com/en/, and loading https://www.mesym.com/en/posts will redirect to https://www.mesym.com/en/posts/, then the Service Workers script already re-writes these requests to their intended URL before these are sent out to the network. (I wrote this article on Smashing Magazine on how to implement Service Workers for a WordPress SPA.)

5. Application view caching through LocalStorage: upon loading, the application loads many of its eventually-needed, most-important-to-render-quickly views through additional requests, executed in the background, and stored in LocalStorage, so that next time the user comes to the website the views are already loaded.

Lazy-loading of data, lazy-execution of JS code, deferring execution of JS code

Application views are lazy-loaded. They are accessed through their own URL, so that if the user clicks on a link to access a view, and the view has not been loaded from the server yet, it still works (in that case, the view will be fetched from the server in that moment.)

Content that is not immediately visible, either because it falls below the fold, or it is hidden under a component such as a collapse, is lazy-loaded. Eg: the comments in the feed here.

Javascript functions applied to hidden DOM elements, such as in a collapse, are lazy-executed when they become visible. Eg: opening any of the collapses (“Events Calendar”, “Projects”, “Featured Articles”) in the top section here will execute the Javascript code in that moment (the calendar/map/carousel lazy-executes their JS code, fetching their required data from the server, and drawing the view only in that moment).

Components that are repeated across pages are lazy-loaded, as to load their data only for the first page, after which it will be loaded from the Service Workers cache. This way, it removes unneeded processing from the server. Eg: The “Trending” and “Events” widgets on the right column here appear in all pages, they are lazy-loaded.

Javascript code is included/executed only after the HTML code (when doing server-side rendering, see below), to speed up the first meaningful paint.

Application rendering

The application offers several ways to render the HTML, which are all used for different cases:

1. Client-side rendering: the application is a Single-Page Application, rendering its content dynamically in the client using Handlebars Javascript templates. When loading, the application already has all the data it needs to be rendered (unlike appshells, which need to fetch data from the server upon loading), avoiding that initial round-trip; this is accomplished thanks to the website being a self-consumed API, so that the initial request already knows what data it will need and cand send it together with the initial response. Being dynamic, the application can optimize how bulk content is retrieved and consumed: it allows the user to click on several links at the same time, and all those pages will open concurrently and remain open until explicitly closed (the user can switch back and forth between them through page tabs).

2. Application Shell (or appshell): even though client-side rendering with no appshell is faster to show the page content, the appshell can be used in particular situations. Eg: to implement Offline first with Service Workers, the application loads an appshell when the user is offline; this appshell will both show the message “You are offline”, and offer the user to visit already-visited, still-cached pages. It optimizes for offline use.

3. Server-side rendering: the application allows to render the initial HTML content directly on the server. It is isomorphic: the same Handlebars Javascript templates to generate the views in the front-end are used to produce the HTML in the server, through LightnCandy. The end result is a faster first meaningful paint.

Efficient loading of assets and execution of JS code

The application implements innovative techniques for dynamically loading assets:

Code splitting for both JS and CSS assets: similar to Webpack (even though not nearly as fancy), the application allows to create a mapping of all relationship between routes and their assets, and serve only the required assets when visiting a page and nothing else. Quite conveniently, there is no need to create a configuration file to generate the mapping: existing code is the only one source of truth, there is no duplication whatsoever. The application’s configuration already indicates what Javascript methods are to be executed on dynamically added elements to the DOM, and what CSS files are needed for each component (this is explained below in item “Links in body”); then, a build tool iterates over all possible application routes and extracts the configuration from each, adds all dependencies by statically analizing all Javascript source files (by running a script based on jParser and jTokenizer to parse Javascript in PHP), and generates the mapping file, in Javascript format, detailing all the assets required for all routes in the website. This file is loaded on the client, so that when clicking on a link, it can already start loading all required assets immediately (instead of having to wait until the response is back to know what assets must be loaded), and in combination with Service Workers, all assets will be loaded by the time the response is back from the server. Since the size of this file with the mapping for the whole application is around 140 KB, it is loaded using “defer”. During the uncanny valley period, if a user clicks on any link before this file is loaded, a message “The application is loading, please wait to click” is shown to the user. Eg: this is a generated mapping file. (I am currently writing an article for Smashing Magazine, possibly titled “Code Splitting DIY”, explaining how it was done.)

Bundling all required assets together: code splitting allows the application to know what assets are required for the page; now it must load them. The application offers three different ways to load them: 1. load all the required individual resources straight; 2. load a unique bundle, called a “bundlegroup”, with all of them bundled in; 3. load a series of bundles, of 4 resources each (the number is configurable). Even though all files from the 3 options can be cached on Service Workers on demand, only option #1 allows to add the resources to the SW pre-cache: the generated bundles/bundlegroups combinations, for all routes, is quite heavy: while originally all resources together are 1.5 MB, all bundles together are 14.3 MB, and all bundlegroups together are 147.7 MB! (The build tool produced around 1000 bundles at 14 KB each average, and 390 bundlegroups at 378 KB each average.) As such, loading a bundlegroup may be more suitable for serving clients without support for HTTP/2, and loading bundles serve as a compromise: more files to load, but more easily cacheable than a bundlegroup (only a bundle item must be invalidated if the code changes, not the whole.)

Progressive booting: Javascript methods can be set as either “critical” or “non-critical” in the configuration. Then, when initializing the application, critical methods are executed immediately, while non-critical ones are executed only after the page has loaded. More importantly, the Javascript files containing non-critical methods are loaded as “defer”, so that the Javascript engine does not spend time parsing these initially, reducing the time to interactive. By default, all Javascript methods are set as non-critical, in order to force the developer to determine what methods are really needed immediately.

Links in body: CSS files are included in the HTML code not in the header, but immediately before an HTML component needs those styles. The architecture of a PoP application is perfectly suited to this technique, since a PoP website is simply a collection of HTML components wrapping each other in forever-nested levels, in a huge LEGO-like building process. It is Brad Frost’s Atomic Design by definition. Components are either an atomic functionality (eg: a link), or a composition of other components. Each square in the image below is a component:

Architecture of a PoP website: components wrapping each other.

In this application, each component can include its required CSS file right before it needs it, and only for the first time a component is printed (the same component can be printed multiple times, as in a feed of posts). Additionally, instead of linking to the file, the full CSS styles can be printed inline.

Decentralized architecture to serve repeating content

The PoP architecture is decentralized, allowing for several websites to share content one with each other. If a particular content needs to appear across several websites (eg: a video service, or an ad-network), it can then be served from only one domain and shared with all other websites, maximizing the likelihood of it being cached for a user visiting two or more of these websites. Eg: this calendar and the calendar in the homepage here are aggregating the same events from other websites.

Tests

Running the tests in the application

I have made available a special URL parameter config in the application, so that anyone can reproduce the tests and see the before/after from applying each optimization, and also for discovering what combination of optimizations produces the most performant version. It can be added to any of these two websites: https://www.mesym.com (hosted in Singapore) and https://agendaurbana.org (hosted in US North Virginia).

To modify the configuration of the website, simply add parameter config listing down all those properties to be true (comma-separated); all properties not on config will have value false (Eg: https://www.mesym.com/en/?config=localstorage,sw,externalcdn,serverside-rendering)

Property Description
localstorage Cache application views in the client through LocalStorage
sw Enable Service Workers to cache (and pre-cache) files on the client
externalcdn Access 3rd party libraries from their own CDN
runtime-js Extract code (needed to generate dynamic views) from the server HTML response into Javascript files
serverside-rendering When loading the website, have the server already produce the HTML code; otherwise, HTML is rendered on the client-side, using Javascript templates
appshell Initially load an Application Shell, which fetches the content upon loading
Loading resources – choose one of the following:
1. app-bundle Load all resources needed for the entire application as a single application bundle (as in the good old days of HTTP/1.1)
2. code-splitting Load only those resources needed by the page, on demand
If doing “server-side rendering”:
scripts-end Re-organize all JS scripts to be included/executed only after the HTML output
If doing “code-splitting”:
progressive-booting Load and execute critical JS code immediately, and non-critical only after the page is loaded
If doing “code-splitting” – Choose one of the following, about how resources are initially loaded:
1. load-resources Load the required resources for that page straight
2. load-bundles Load all required resources for that page through a series of bundled files, each of them comprising up to 4 resources
3. load-bundlegroups Load all required resources for that page through a unique bundlegroup file
If doing “serverside-rendering”, “code-splitting” and “load-resources” – Choose one of the following, about how styles are loaded:
1. resources-header Load the styles in the header
2. resources-body Include the styles in the body as links, each of them right before it is needed by some HTML component
3. resources-body-inline Inline the styles in the body, each of them right before it is needed by some HTML component

Test results

The following URLs represent the page with/out each optimization. Copy/paste each of them on WebPageTest to see the improvement achieved, or simply click on the link below each table to access the corresponding results.

Caching Application views in LocalStorage

Not enabled: https://agendaurbana.org/es/?config=sw,code-splitting,load-resources,serverside-rendering,scripts-end

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Enabled: https://agendaurbana.org/es/?config=sw,code-splitting,load-resources,serverside-rendering,scripts-end,localstorage

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Caching assets through Service Workers (SW)

(Please notice: in order to test the “no SW” option in a browser and the SW had already been registered, then we must unregister it: 1st browse the URL, then “Unregister” the SW in the browser (eg: in Firefox: Tools => Web Developer => Service Workers and then click on “unregister” under the domain), and then call the URL again.)

Not enabled: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Enabled: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end,sw

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Accessing 3rd party libraries

From our website’s CDN: https://agendaurbana.org/es/?config=localstorage,sw,code-splitting,load-resources,serverside-rendering,scripts-end

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

From their own domain/CDN: https://agendaurbana.org/es/?config=localstorage,sw,code-splitting,load-resources,serverside-rendering,scripts-end,externalcdn

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Extraction of code from the server HTML response

Keep it inside the HTML response: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Extract it into JS files: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end,runtime-js

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Server/client-side rendering, Application Shell

Client-side rendering: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Server-side rendering: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Application Shell (through client-side rendering): https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,appshell

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Application Shell (through server-side rendering): https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end,appshell

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Loading resources

Single application-wide bundle file: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,app-bundle

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Code-splitting (loading 1 “bundlegroup”): https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-bundlegroups

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

(If doing “server-side rendering”)

Including/executing JS code…

As it comes: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Only after all HTML output: https://agendaurbana.org/es/?config=localstorage,code-splitting,load-resources,serverside-rendering,scripts-end

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

(If doing “code-splitting”)

Progressive Booting

Not enabled: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-resources

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Enabled: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-resources,progressive-booting

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

(If doing “code-splitting”)

Initially loading…

Individual resources: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-resources

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

List of bundles (of up-to-4-resources each): https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-bundles

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

1 bundlegroup: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-bundlegroups

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

(If doing “serverside-rendering”, “code-splitting” and “load-resources”)

Loading styles…

In the header: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-resources,resources-header

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Link in the body: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-resources,resources-body

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Inline in the body: https://agendaurbana.org/es/?config=localstorage,serverside-rendering,scripts-end,code-splitting,load-resources,resources-body-inline

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Analysis and Conclusions

Given all the individual tests, we can make the following observations:

localstorage Adding LocalStorage shaves a few hundred milliseconds
sw Having support for Service Workers made the application take longer to load. However, this extra time (needed to download all required assets from the website for all routes, in total around 1.5 MB of files) will be saved later on when accessing the website time and again. So SW should still be enabled, in spite of this negative test result.
externalcdn Loading external resources from their own CDN shaves a few hundred ms
runtime-js Running the page only a few times shows similar loading times. However, I expected that, on loading the application time and again, saving the (when uncompressed) 374 KB of code to be sent with the output should be faster.
serverside-rendering Server-side rendering improves significantly the loading time over client-side rendering
appshell Loading an Application Shell takes longer than loading the content straight
scripts-end Adding Javascript after the HTML output makes the first meaningful paint happen significantly earlier (1.384s vs 2.546s and 1.709s vs 3.385s), but less clear results for the time to interactive (7.613s vs 9.942s and 14.094s vs 13.765s)
progressive-booting Progressive Booting improved the loading time, particularly in mobile
Loading resources:
1. app-bundle vs 2. code-splitting Code splitting is slightly faster, shaving a few hundred milliseconds off
How resources are initially loaded:
1. load-resources vs 2. load-bundles vs 3. load-bundlegroups The “bundlegroup” options is the fastest way (even while assets are served from a CDN with HTTP/2), followed by the bundles, and finally the individual resources. The results are consistent across the different devices.
How styles are load:
1. resources-header vs 2. resources-body vs 3. resources-body-inline Adding the links in the body or inlined is faster than adding all links in the header

With all the test data, we obtain which are the most performant options:

  • localstorage
  • externalcdn
  • serverside-rendering
  • scripts-end
  • progressive-booting
  • code-splitting loading “bundlegroup”
  • resources-body-inline

Now we can calculate what were the gains, by comparing the delta between the least and most performant versions:

Least performant version: https://agendaurbana.org/es/?config=sw,appshell,app-bundle,resources-header

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

Most performant version: https://agendaurbana.org/es/?config=localstorage,externalcdn,serverside-rendering,scripts-end,progressive-booting,code-splitting,load-bundlegroups,resources-body-inline

Results WebPageTest.
Results for “Dulles, VA USA (Desktop, Android, iOS)”
Results WebPageTest.
Results for “Dulles, VA USA (Moto G — gen 4)”

From the least performant to the most performant versions, we have achieved the following improvements (calculated as time difference):

  • Load time: 3 seconds in desktop, 5 seconds in mobile
  • First meaningful paint: 500 ms in desktop, -500 ms in mobile (it actually got worse)
  • Time to interactive: 2 seconds in desktop, 3 seconds in mobile

And the first meaningful paint happens below the 3 seconds threshold, for both desktop and mobile.

on 12 Dec, 02:11

Coming soon: GutenPoP, the integration of WP Gutenberg with PoP

Coming soon: GutenPoP, the integration of WP Gutenberg with PoP
Blog

WordPress Gutenberg is looking incredibly impressive, as it can be appreciated in the demo provided during WordCamp US 2017. Gutenberg will change drastically how WordPress sites are built with its intuitive, yet extremely powerful, user interface. While currently Gutenberg allows to only create content (replacing the content editor), in the future it is expected to be able to construct the whole site. In this post, Morten Rand-Hendriksen explains the new approach quite clearly: “Today we think of creating posts and pages. In the WordPress of Tomorrow, we will be making blocks and placing them in views.” This is how the WordPress of tomorrow will look like:

gutenberg-blocks

Blocks. Inside the views. Remarkably, that is what PoP is all about! Now, look how blocks are already filling the view in the PoP of today:

blocks

Also remarkably, these components have also been called blocks in PoP, putting in evidence that the Gutenberg concept is the same PoP uses. Whereas the WordPress community has been working on adding the block-concept to WordPress since 11 months ago, PoP has been based on this concept since its inception, already 4.5 years ago!

PoP, however, takes the block concept deeper down: the smallest units of components in a PoP view are not blocks, but modules, which are either an atomic representation of a functionality (eg: something as basic as a link), or a composition of other modules. It looks like this:

modules

In PoP, everything is a module. Blocks are also modules, with the particularity that they have their own state, and they know from what URL to fetch their data. A block can be as simple as containing 1 inner module, or extremely complex. Moreover, blocks can be nested, wrapping other blocks, creating incredibly powerful components, yet made-up of re-using independent, autonomous ones. It looks like this:

blockgroups

In this image above, the block with text “Welcome to PoP Demo!” is wrapping another block, which adds the gray background styling, and which itself is wrapping 2 other blocks: a log-in menu, and a newsletter form. These 2 blocks are completely autonomous, and they live in the website in different other views too. When nesting, blocks can alter the attributes of the inner blocks, thanks to PoP’s top-down approach for setting properties.

What’s coming next

Given all the similarities between Gutenberg and PoP, including how they are both based on block populating the view, it is extremely compelling to provide an integration between PoP and Gutenberg. PoP could perfectly augment the capabilities of a Gutenberg-based website by providing its own blocks as Gutenberg blocks.

In the following months, we will work on these features:

  • Site-builder integration: use Gutenberg to insert PoP blocks into the view of a GutenPoP website
  • Front-end integration: add Gutenberg to the front-end of a PoP website, for users to be able to create content (posts, comments) using the Gutenberg interface

We are extremely excited with this development, we can see how the combination of Gutenberg and PoP will enable the creation of highly-dynamic websites, yet in a purely drag-and-drop, users-can-do-it-from-the-frontend manner.

on 12 Dec, 02:10

PoP now treats Javascript as progressive enhancement

PoP now treats Javascript as progressive enhancement
Blog

PoP now treats Javascript as a progressive enhancement, meaning that browsing a PoP website from a device with no support for Javascript will still work. This feature is available since adding server-side rendering to PoP. To check it out, simply disable Javascript from your browser, and refresh this current website.

You can also force the website to remove all Javascript output from the page. This could be useful in several scenarios, such as testing the overhead from adding Javascript to a site. (For simple blogs for emerging markets, a very basic HTML-only website might work better). For this, simply add parameter config with value disable-js to the site URL. To test it, please check it out on any of these websites: https://www.mesym.com/en/?config=disable-js and https://agendaurbana.org/es/?config=disable-js.

Currently, around 80% of all functionality can be accessed without Javascript (check below to see what is still missing), including:

  • All content in the website is browsable
  • The “Load more” button works, it simply refreshes the page to show more content instead of doing infinite-scroll
  • Search works
  • All interaction with the website works: users can log-in, add a new post, follow other users, etc
  • Modal windows, such as opening a Google Map, work; instead of opening in a modal, they open in a new tab
  • etc

Some examples of functionalities with(out) Javascript support:

Adding comments:

adding-comment-js

Adding a comment with Javascript enabled: it opens an inline window.

adding-comment-no-js

Adding a comment with Javascript disabled: instead of opening an inline window, it opens on a new browser tab.

Searching:

search-js

Searching with Javascript enabled: a typeahead offers results immediately.

search-no-js

Searching with Javascript disabled: it simply performs the search, refreshing the page.

Work still to be done

There are still plenty of kinks to be ironed out for this functionality to work 100%, such as:

  • Adding a comment uses TinyMCE, which relies on Javascript; it should just use a standard textarea in this case
  • Events calendar is still printed using Javascript
  • Google Maps does not print a fallback image
  • Comments rely on Javascript to be added to the DOM
  • Plenty of modules make sense only when Javascript is enabled (eg: filter posts by section). These should either be hidden, or, even better, not added to the configuration in first place if Javascript is not supported
  • Bootstrap components are still rendered, such as collapses, carousels, etc

As usual, enjoy!

on 3 Dec, 15:53

New feature: Link in body

New feature: Link in body
Blog

By adding support for on-demand loading of CSS resources through code-splitting, we have been able to implement a very handy new feature: Link in body. This technique allows for a sequential render of the page. There is no need to explicitly extract all CSS styles which go “above the fold”, since now they will be output just before the first instance of the corresponding module. It not only adds convenience for the developer, but it also boosts optimization: because the browser will initially process those CSS styles which are visible on the screen, the first meaningful paint will take place earlier.

The architecture of a PoP application is perfectly suited to this technique, since a PoP website is simply a collection of modules wrapping each other in forever-nested levels, in a huge LEGO-like building process. Components are either an atomic functionality (eg: a link), or a composition of other components. Each square in the image below is a component:

modules
Architecture of a PoP website: modules wrapping each other.

Additionally, instead of linking to the file, the full CSS styles can be printed inline.

As usual, enjoy!

on 3 Dec, 15:39

Added support for code-splitting of CSS resources

Added support for code-splitting of CSS resources
Blog

A while ago we implemented a nifty feature: code-splitting. Code-splitting allows to load only those assets which are required for the requested page, and nothing else. However, back then the implementation only worked for Javascript assets.

We are now thrilled to announce support for CSS assets too. CSS assets can now be defined on a module-by-module basis, so that when a module is to be loaded, its needed CSS assets will also be loaded, and only then. The mapping from module to CSS assets is decoupled: modules must not necessarily know what CSS files they need to load, since they are the functionality of the application, and CSS implements the look-and-feel. Then, the implementation is simply a hook, which allows the theme to add CSS files for each module.

function get_template_resources($template_id, $atts) {

  // Allow the theme to hook in the needed CSS resources. It could be based on the $template_id, or its template source, then pass both these values
  return array_values(array_unique(apply_filters(
    'GD_Template_ProcessorBase:template-resources',
    array(),
    $template_id,
    $this->get_template_source($template_id, $atts),
    $atts,
    $this
  )));
}

Defining the hook in file wp-content/plugins/pop-frontendengine/kernel/processors/pop-processor.php

class PopThemeWassup_AAL_ResourceLoaderProcessor_Hooks {

  function __construct() {

    add_filter(
      'GD_Template_ProcessorBase:template-resources',
      array($this, 'get_template_css_resources'),
      10,
      5
    );
  }

  function get_template_css_resources($resources, $template_id, $template_source, $atts, $processor) {

    switch ($template_source) {
      
      case GD_TEMPLATESOURCE_LAYOUT_PREVIEWNOTIFICATION:

        $resources[] = POP_RESOURCELOADER_CSS_NOTIFICATIONLAYOUT;
        break;
    }

    return $resources;
  }
}
new PopThemeWassup_AAL_ResourceLoaderProcessor_Hooks();

Implementing the hook to add styles to a module

As usual, enjoy!

Load more

Sign up to our newsletter:

Welcome to the PoP framework!
Break the information monopoly

the PoP framework is open source software which aims to decentralize the content flow and break the information monopoly from large internet corporations. Read more.