A new PoP is coming soon, with a revamped architecture and plenty of exciting new features

Even though released as open source 2 years ago, PoP still doesn’t have installation scripts, making unfamiliar developers unable to install it (to this day, it has barely received 50+ stars in GitHub.) 8 months ago we finally started working on the scripts. In the process, we discovered several issues which had to be fixed too. Given the ambition of what the new PoP will be able to offer, and the amount of code developed during the last 5+ years which needs refactoring, after 8 months of work we still haven’t finished. If everything goes fine, the new PoP will be released around the end of 2018 or beginning of 2019.

After PoP is newly released, it will live up to the expectations of being called a framework: until now, PoP was pretty much limited to building community-focused social networks like MESYM.com, but through the new PoP it will be possible to build any type of website: static, dynamic, with/out users, with/out Single Page Application, with/out Server-Side Rendering, etc

There are 2 big announcements:

  1. PoP is being re-written, based on the following foundations:
    • Everything is a module
    • The module is its own API
    • Reactivity
  2. The codebase has been modularized and made CMS-agnostic, and plenty of new features have been developed

Announcement #1 is quite big on its own, so it will be dealt with in its own blog post. Below we deal with Announcement #2, describing some of the new features/enhancements produced during the past 8 months. These have not been uploaded to the Github repo yet, because the code in still under heavy development (we expect to do it around the end of 2018 or beginning of 2019).

Split the codebase up into countless plugins, allowing to progressively enable features

In the past, PoP was a huge monolithic codebase, in which you either got everything (a site such as MESYM.com) or nothing, since code was dependant in such a way that deactivating a feature may break everything else.

We have progressively fixed this by identifying pieces of code, analyzing their functionality, and placing them in a specific plugin. To this date, we have produced more than 50 plugins, and since we haven’t finished, it is still increasing.

Some plugins we have created deal with functionality, such as the following:

  • pop-userlogin: Allows users to log-in to the site from the front-end
  • pop-userplatform: Allows users to create an account on the site
  • pop-socialnetwork: It transforms the site into a social network, adding features such as liking posts, following users, and others

Some other plugins deal with the framework itself, such as the following:

  • pop-spa: Transforms the site into a Single Page Application
  • pop-ssr: Enables Server Side Rendering on the site
  • pop-resourceloader: Enables code-splitting

Until now, all code had been housed under a unique Github repo, but from now on every plugin will have a repo of its own. Having their own repository, each plugin will have its own documentation, and its own methodology for installation (some plugins may depend on Composer, others on NPM, others on GitHub subprojects, etc…)

Introduced dependencies among plugins

Plugins will be activated only if their dependencies are all active. Then, in the hierarchy from plugins described above, we can declare:

  • Activate pop-socialnetwork only if pop-userplatform is active
  • Activate pop-userplatform only if pop-userlogin is active

This way, the application will always work, such as avoiding compilation errors if a required PHP class is not loaded anymore.

Split the application into 4 layers: API, Frontend, Backend, and Implementations

Each plugin is divided into up-to-4 separate yet interconnected subplugins:

  • API: it deals with data-loading/posting for each module
  • Frontend: JS and CSS resources, Handlebars JS templates, general functionality when there is a browser client.
  • Backend: abstract classes establishing the configuration for a module
  • Implementations: actual module implementations, filling the configuration values as suitable

This architecture makes a module itself be modularized. Such fine-grained modular architecture allows for scalability/extensibility: developers can add their own custom implementations, replace the frontend JS/CSS files, establish a different configuration for a module, all without affecting the other portions of code.

In addition, if the Frontend layer is disabled, then the site barely returns the data in JSON, hence effectively becoming an application server. If we have all the layers enabled, the site will look like normal:

However, when plugin pop-engine-frontend is disabled, and we reload the page, we get this response instead:

Which viewed through a JSON formatter looks like this:

 

This is basically the JSON response containing all the data needed to render the page. Since disabling the frontend layer in the application, PoP assumes that only the data is needed and nothing else.

Re-organized the architecture to support other Content Management Systems

Because most of the code for PoP is plain PHP, it can perfectly be made CMS-agnostic, so that PoP could also be powered by other PHP-based Content Management Systems, such as Joomla or Drupal.

For this purpose, we implemented several interfaces (such as PoP_CMS_FunctionAPI, PoP_CMS_ObjectPropertyResolver and others), which list those functions that will be needed by PoP, expecting the implementing CMS to provide an implementation for them.

Currently, this feature is still under heavy development, with the function signatures being those used by WordPress (eg: functions names get_posts, get_users, etc, and also expecting the same parameters as by the WordPress functions) so it is not expected for other CMSs to be plugged in and work immediately; plenty of work is still required. However, the architecture has already support for the extending into other CMSs, so it must just be implemented.

Introduced interfaces to make functionality be plugin-agnostic.

Many modules in PoP rely on 3rd-party plugins for their functionality, and if that 3rd party plugin was not installed, then the component would not function. For instance, the events calendar in MESYM.com relies on plugin Events Manager for fetching the events; if Events Manager is not installed, then the website cannot display the PoP events calendar module.

We have now overcome this restriction by introducing interfaces to connect the 3rd party plugins with the PoP modules. For instance, for the events calendar mentioned above, we have now added interface PoP_Events_API listing down all the functions required by the calendar functionality, so that Events Manager just provides an implementation of this interface. This way, other plugins can also be used as functionality providers.

Implemented infinite data-loading

Similar to GraphQL, the site allows to fetch relational data, involving many relationships among object types and as complex as needed, such as:

Fetch all posts. For each post:
    Fetch the author. For each author:
        Fetch the posts the user has "liked". For each post:
            Fetch all its comments. For each comment:
                Fetch the comment author. For each author:
                    Fetch all the events which this user is attending. For each event: 
                        Fetch its location. For each location: ...

Which translates into object types being loaded like this:

post => user => post => comment => user => event => location => ...

PoP will fetch the data from the database, and replicate the relational structure of the data into the resulting JSON object:

database: {
    posts: {
        1: {author: 1, title: "This is a post", ...},
        2: {author: 2, ...},
    },
    users: {
        1: {name: "Leo", ...},
        2: {name: "Pedro", ...},
        ...
    },
    comments: {
        1: {author: 1, postId: 1, content: "This is a comment by Leo on post 'This is a post'", ...}
    },
    events: {...},
    locations: {...},
}

The data for each object is fetched only once from the database, and it is printed only once on the output (i.e. it is not nested), making it very efficient. Adding additional levels doesn’t incrase the complexity of the query operation (as in GraphQL), while being fully cacheable on the server side (unlike GraphQL.)

Allowed data to be marked as dynamic, so it will be sent to the frontend even when doing SSR

For optimization reasons, when doing Server Side Rendering (SSR) we don’t send the JSON object with all the database data, which may be quite big. However, this carried the issue that some data is actually dynamic, i.e. it will be used on the client to dynamically create views. Hence, for these situations, we had to send the whole database.

Now, through the newly added function get_dynamic_data_settings, it is possible to mark what data-fields are dynamic, and in that case only these are also sent to the frontend, and all static data, which is already rendered in the HTML generated in the server through SSR, is discarded, effectively sending less data down the wire.

Many more

  • Used PHP traits to re-use code, between classes in plugins PoP Engine and PoP Frontend Engine
  • Converted categories with particular functionality/inputs into custom post types (eg: Highlight custom post type)
  • Made blocks self-aware if they require lazy-loading based on the checkpoint status of their corresponding page, so that lazy-loading can be done automatically instead of through configuration
  • Streamlined code by grouping functionality (eg: function pop.DynamicRender.render), created interfaces (eg: FormComponent)
  • Split all Handlebars helpers into multiple files, as to only load what is needed and improve modularity
  • Moved all PoP JS functions under global variable pop, so as not to pollute the global scope anymore
  • Re-wrote plenty of modules from scratch
  • And many others

Sign up to our newsletter:

Welcome to the PoP framework!
Break the information monopoly

the PoP framework is open source software which aims to decentralize the content flow and break the information monopoly from large internet corporations. Read more.