PoP is an open source framework for building progressive web apps, decentralized websites and social networks.

PoP combines WordPress and Handlebars into an MVC architecture:

Show me how it works

Development features

The PoP engine automatically provides the website of its own API, just by adding output=json to any page’s URL.

Key benefits:

Documentation-less API: the API needs no documentation, because simply removing parameter “output=json” from its URL renders the website, displaying the information for GET requests and forms with their required fields for POST operations.

Simplicity: the API URL can be calculated directly from the website URL.

SEO friendly: the API URL keeps WordPress’ permalink structure and, hence, the slug’s high ranking.

API-ready: the API can also be consumed by third-party systems, mobile phone apps, etc, without further work needed by the developer.

Because they comprehend the same JSON structure, PoP websites can natively interact with each other, fetching and processing data from each other in real time, for both GET and POST requests.

The calendar below is fetching and displaying events from many different websites. Clicking on an external link inside the calendar opens the page in this same website.

Applications:

Decentralization: the application’s data sources can be split into several websites/domains

Aggregation: content from a variety of PoP websites can be aggregated and displayed together

Component microservices: components/libraries can be made available to developers as SaaS

PoP is based on the concept of “modules”, which are either an atomic functionality, or a composition of other modules. Building a PoP application is akin to playing with LEGO: stacking modules on top of each other or wrapping each other, ultimately creating a more complex structure:


modules

Modules can be grouped into higher-order entities called “blocks”. Blocks have additional properties of their own: they know from what URL to fetch their data, and they provide a place where modules can store their state, among others. Blocks are autonomous components, with a specific purpose or logic, which make sense of the collection of modules within them:


blocks

Blocks can be nested: any block can be wrapped by another block, thus becoming a “blockGroup”. BlockGroups change the properties and behaviour of the inner blocks, and synchronize the state of all wrapped blocks among each other:


blockgroups

All blocks and blockgroups are placed in the view in physical/functional sections on the webpage, which are called “pageSections”. A pageSection can also alter the properties of the blocks placed within it:


pagesections

Finally, all pageSections are combined all together into one single “topLevel”, which is similar in concept to WordPress’ hierarchy template (Single, Home, Archive, Author, etc):


toplevel

In PoP, everything is a module: topLevels, pageSections and block/blockGroups are modules too, just with additional properties. Modules are thus categorized in a stricly top-down architecture:

  • 1 topLevel contains N pageSections,
  • 1 pageSection contains N blocks/blockGroups,
  • 1 blockGroup contains N blocks/blockGroups,
  • 1 block contains N modules,
  • 1 module contains N modules, ad infinitum.

Every module knows what inner modules they contain, down the line to the last module, and can change those modules’ properties; however, inner modules do not know who is including them, and do not need to care either. When initializing properties, parent modules take precedence over inner modules: when a module initializes itself, if any one property has already been set by a parent module, then that value will be kept. This way, PoP allows for dependency-injection, where each module can set its default attributes, yet allow for properties to be injected externally in advance (possibly coming from a configuration file).

class GD_EM_Template_Processor_ScrollMapBlocks extends GD_Template_Processor_BlocksBase {

  function init_atts($template_id, &$atts) {

    // $template_id = current module
    $this->add_att($template_id, $atts, 'direction', 'horizontal');

    // 'map-layout' = submodule
    $this->add_att('map-layout', $atts, 'direction', 'horizontal');
    
    return parent::init_atts($template_id, $atts);
  }
}





		

		

Most content in the website, by nature “dynamic” or mutable, is made “static” or immutable by serving it through a Content Delivery Network (CDN), thus making the best of both worlds: content is dynamically generated through WordPress, but it can be cached as coming from a static site generator.

This feature has the following advantages:

Cheaper hosting: websites with high traffic can reduce their hosting costs, since content is mainly served from the CDN instead of the server.

Faster: CDN data-centers are located near the user, thus greatly reducing latency and boosting the overall website speed.

WordPress is still there: content is still created dynamically, through WordPress. The static version of the website is generated on runtime.






		

		

Through the implementation of service workers, PoP provides offline first capabilities, with the following benefits:

  • the website functions offline
  • performance increases by reducing network requests
  • pages load immediately (from the second access onwards)

Service workers are not yet implemented in all modern browsers, however both Chrome and Firefox (the two most popular browsers, with a combined market share of more than 70%) do support it.






		

		

PoP provides optimal rendering capabilities, allowing to render the HTML in the backend when loading the page first time, yet rendering all code in the frontend through javascript from the 2nd page-load onwards.

It is isomorphic: the code used to render the HTML in both front and back-end is the same: precompiled Handlebars javascript templates, rendered in the front-end using the Handlebars runtime library, and in the back-end using LightnCandy.

Advantages:

SEO: all search engines can perfectly parse the site’s HTML.

Integration with technologies and systems: server-side HTML rendering is needed for implementing new technologies such as AMP and microformats, and for broad system interoperability.

Speed: the first page is loaded much faster, since it doesn’t need to wait to load javascript templates to render it. Starting on the 2nd page onwards, javascript takes over, rendering faster and making the website dynamic.

Re-usability: developers implement the code only once and use it everywhere: front and back-end, emails, etc.

Clearly-Decoupled Responsibilities

Each module is composed of clearcut, non-overlapping concerns, allowing a team of several people, each with a specific responsibility (back and front-end development, design, etc), to work on the same component simultaneously:

Back-end logic through PHP

Views through Handlebars templates

Front-end logic through Javascript

CSS styles






		

		

5 layers of caching are implemented in between the server and the client:

1. Page-caching in the server: PoP caches pages in the server, even when the user is logged-in.

2. Configuration-caching in the server: because the configuration is decoupled from the database data, it can be cached independently. Moreover, the configuration does not depend on the URL, but on the page-type. Then, server processing is reduced: the first time a post is loaded, the configuration will remain cached for whenever loading any other similar post; only database data needs to be fetched.

3. Content CDN: requests to fetch content are routed through a Content Delivery Network, making dynamic content become static, since the CDN will return the cached version of that content and not need reach the server.

4. Content and assets caching in the browser through Service Workers: all needed assets are pre-cached when installing the Service Workers script, browsable even if the user is offline.

5. Application view caching through LocalStorage: upon loading, the application loads many of its eventually-needed, most-important-to-render-quickly views through additional requests, executed in the background, and stored in LocalStorage, so that next time the user comes to the website the views are already loaded.






		

		

The application implements innovative techniques for dynamically loading assets:

Code splitting for both JS and CSS assets: similar to Webpack (even though not nearly as fancy), the application allows to create a mapping of all relationship between routes and their assets, and serve only the required assets when visiting a page and nothing else. Quite conveniently, there is no need to create a configuration file to generate the mapping: existing code is the only one source of truth.

Bundling all required assets together: the application offers three different ways to load the assets, which can be used whenever convenient:

  1. load all the required individual resources straight
  2. load a unique bundle, called a “bundlegroup”, with all the needed assets bundled in
  3. load a series of bundles, of 4 resources each (the number is configurable)

Progressive booting: Javascript methods can be set as either “critical” or “non-critical” in the configuration. Then, when initializing the application, critical methods are executed immediately, while non-critical ones are executed only after the page has loaded. More importantly, the Javascript files containing non-critical methods are loaded as “defer”, so that the Javascript engine does not spend time parsing these initially, reducing the time to interactive.

Links in body: CSS files are included in the HTML code not in the header, but immediately before an HTML component needs those styles.

PoP has support for A/B Testing natively, through the modification of one or two URL parameters which change the look and feel of a webpage:

  • thememode
  • themestyle

Whereas thememode is used for rendering a webpage for a completely different use case (such as embed, print, or mobile), themestyle is used to tweak the presentation styles. Some examples:

Homepage:

Homepage for printing purposes, appending thememode=print to the URL:

Homepage with a wider presentation, appending themestyle=expansive to the URL:

Once the website is initially loaded with a given value for any of these two parameters, the framework in the front-end will always append these values to all subsequent requests. In the image below, themestyle=expansive is added to all requested URL, either as a link clicked upon by the user, or dynamic content fetched by some component:

Then, a PoP site can be customized using these parameters, and users can be served either version. Because this information will always be in the requested URL, then we can analyze and measure the impacts of each design very easily. Finally, to append one or another parameter value to the URL, the URL is intercepted in the client through Service Workers, adding a parameter or the other following the weight given to each.

All content in the website, either general content open to the public, as much as personal content that only logged in users can see, can also be sent through email, with no extra effort or duplicated work, because the email html code can be produced from the same source as the website pages.

There is no need even to convert CSS to inline styles for the emails, since this is already taken care of automatically using library PHP CSS Parser (there is a limitation though: it only works for single-level classenames (eg: .btn); there is no support for other forms in the css, such as concatenated classnames (eg: .btn.btn-info), nested classes (such as .btn .btn-info), or html elements (eg: input[type="submit"]).

This way, it is now possible to create email digests with any content shown in the website, such as:

  • Personal daily notifications
  • Daily digests of new posts
  • Daily digests of upcoming events
  • Reminder emails to users attending an event
  • Feedback emails to review a product
  • Others

PoP queries the database and recreates the data structure in the JSON code sent to the front-end, keeping the relationships among all different objects (eg: if two different posts share the same author, the author’s data is output only once, and the posts reference the author through the id).

The produced JSON object has key ‘database’, containing the data for all objects queried in the request; key ‘dataset’, with the ids of the result objects for each block; and key ‘settings’=>‘db-keys’, with the top-most object domain (“posts”, “users”, “locations”, etc) for each block.

json-structure

Viewing the response’s JSON code in Chrome’s Console. Pay attention how under key ‘database’ the object data is relational, and objects from different domains have also been fetched: the post’s author (with id 851) and the post’s location (with id 23494); and under key ‘dataset’ it indicates the result object ids for each block (posts with ids 23787, 23784, 23644 for the main block)

PoP integrates with WordPress mainly in the back-end, executing WordPress functions to fetch data. Implementing a PoP website requires the implementation of these objects to communicate with the WordPress API to fetch the data. PoP already ships with implementations for the most basic object types (posts, users, tags, comments) and some custom post types (events, locations).

Basic example

Below is a description of setting up some required components: a “block”, a “dataloader”, some module defining “data-fields”, and a “fieldProcessor”.


A “block” is a special module in PoP which defines the configuration for fetching data from WordPress, such as what “dataloader” will be invoked, how to filter the query, the data’s source URL, among others:

class GD_Template_Processor_SectionBlocks extends GD_Template_Processor_BlocksBase {

  function get_dataloader() {

    return GD_DATALOADER_POSTLIST;
  }

  function get_dataload_source() {

    global $author;
    return get_author_posts_url($author);
  }

  function get_dataload_query_args() {

    global $author;
    return array(
      'author' => $author,
      'cat' => 55,
    );
  }
}

// Initialization
new GD_Template_Processor_SectionBlocks();

A “dataloader” executes WordPress functions to fetch data from the database, allowing to set the args in the query to filter the results.

define ('GD_DATALOADER_POSTLIST', 'post-list');

class GD_DataLoader_PostList extends GD_DataLoader_PostListBase {

  function get_name() {
    
    return GD_DATALOADER_POSTLIST;
  }
  
  function get_query($vars = array()) {
  
    $query = array();
    if ($cat = $vars['cat']) $query['cat'] = $cat;
    if ($author = $vars['author']) $query['author'] = $author;

    return $query;
  }
  
  function execute_query($query) {

    return get_posts($query);
  }
}

// Initialization
new GD_DataLoader_PostList();

What data will be retrieved from the queried object is defined in all modules, through specifying what “data-fields” they need.

class GD_Template_Processor_PostLayouts {

  function get_data_fields() {

    return array('title', 'url', 'comments-count');
  }
}

// Initialization
new GD_Template_Processor_PostLayouts();

The “data-fields” are processed and resolved through an object called a “fieldProcessor”.

define ('GD_DATALOAD_FIELDPROCESSOR_POSTS', 'posts');

class GD_DataLoad_FieldProcessor_Posts extends GD_DataLoad_FieldProcessor {

  function get_name() {
  
    return GD_DATALOAD_FIELDPROCESSOR_POSTS;
  }
  
  function get_value($resultitem, $field) {
  
    $post = $resultitem;

    switch ($field) {

      case 'title' :
        return get_the_title($post->ID);

      case 'url' :
        return get_permalink($post->ID);
    
      case 'post-type' :
        return $post->post_type;

      case 'comments-count' :
        return get_comments_number($post->ID);

      case 'cats' :
        return wp_get_post_categories($post->ID, array('fields' => 'ids'));

      case 'tags' :
        return wp_get_post_tags($post->ID, array('fields' => 'ids'));
    }

    return null;
  }
}

// Initialize
new GD_DataLoad_FieldProcessor_Posts();

In summary, adding the required objects to implement a WordPress website using PoP is a straightforward process, and the code already ships with plenty of implementations.

The decentralization features of PoP allow to create an application architecture in which different components are being served from different subdomains:

site-wireframe

To implement it, simply set the external domain source on the block. This configuration value can be defined directly in the block, or it can be injected externally, in which case the block is unaware where it is fetching its data from.

class GD_Template_Processor_SectionBlocks extends GD_Template_Processor_BlocksBase {

  function get_dataloadsource_domain($template_id, $atts) {

    switch ($template_id) {
      
      case GD_TEMPLATE_BLOCK_AUTHORHIGHLIGHTS_SCROLL_LIST:
        
        return 'https://demo.getpop.org';
    }

    return parent::get_dataloadsource_domain($template_id, $atts);
  }
}
class GetPoP_Template_Processor_CustomBlockGroups extends GD_Template_Processor_ListBlockGroupsBase {

  function init_atts_blockgroup_block($blockgroup, $blockgroup_block, &$blockgroup_block_atts, $blockgroup_atts) {

    if ($blockgroup == GD_TEMPLATE_BLOCKGROUP_AUTHORHIGHLIGHTS_SCROLL && $blockgroup_block == GD_TEMPLATE_BLOCK_AUTHORHIGHLIGHTS_SCROLL_LIST) {

      $this->add_att($blockgroup_block, $blockgroup_block_atts, 'dataloadsource-domain', 'https://demo.getpop.org');
    }

    return parent::init_atts_blockgroup_block($blockgroup, $blockgroup_block, $blockgroup_block_atts, $blockgroup_atts);
  }
}
Open all
PoP treats javascript as a progressive enhancement, so that browsing a PoP site with javascript not working (possibly for some network error) will still serve the intended content or provide the expected functionality.
All pages have been stripped of all user information related to them (eg: has the user recommended it?). All such user information is loaded on a subsequent, immediate request.

As a consequence, requesting the page produces the same response for all users, allowing most pages in the website to be cached, independently if the user is logged in or not; only those pages showing user-specific information, such as My Content, are non-cacheable.
Selected content, such as comments added to a post, can be loaded only after the page has loaded, allowing for a faster initial page load.
The software pre-loads specified pages, running in the background, so they are already loaded by the time the user clicks on the corresponding link. Eg: when clicking on the following links they will open immediately, since they were preloaded: Add Post, Contact us. Once these pages have loaded, if available, their code is stored in the browser using Local Storage.
Clicks can be intercepted in the front-end, and choose to do something instead of sending the request to the server. Eg: adding a post can open in a new page, or be intercepted and open immediately in a floating window.
When the user clicks on a link, PoP intercepts the request to submit it using AJAX. PoP can add extra parameters to the URL, which are not in the original link (and so hidden to the user), such as: language, theme, timestamp, etc.

Sign up to our newsletter:

Welcome to the PoP framework!
Break the information monopoly

the PoP framework is open source software which aims to decentralize the content flow and break the information monopoly from large internet corporations. Read more.