John Swimmer Recommendations

on 10 Aug, 17:36

PoP can finally be installed!

PoP can finally be installed!
Blog

Alright, that took some time in coming, but we can finally announce it: PoP can finally be installed through Composer! Sucess!!!!!

So for first time in its history, PoP can be easily installed, without resorting to hacks: simply execute a script, wait for a few minutes, and voilĂ . Currently, only the PoP API for WordPress is finished, and in a few months we will unlock further functionality: creating the whole site through PoP, and for other CMSs in addition to WordPress. Yay!

Installation instructions

Via Composer and WP-CLI (both are assumed to be installed globally):

  1. Create the WordPress database and user
  2. Configure WordPress through environment variables:

Copy the code below to an editor, replace all values (such as {YOUR_SITE_DB_NAME}) with your own values, and then either paste it on the terminal to execute, or save it in file “~/.bash_profile” and then execute source ~/.bash_profile.

export DB_NAME={YOUR_SITE_DB_NAME} #eg: database
export DB_USER={YOUR_SITE_DB_USER} #eg: admin
export DB_PASSWORD={YOUR_SITE_DB_PASSWORD} #eg: sADF!kl9diq@#Sjfk
export DB_HOST={YOUR_SITE_DB_HOST} #eg: 127.0.0.1
export SITE_URL_WITHOUT_HTTP={YOUR_SITE_URL_WITHOUT_HTTP} #eg: localhost
export SITE_URL_WITH_HTTP={YOUR_SITE_URL_WITH_HTTP} #eg: http://localhost
export SITE_NAME="{YOUR_SITE_NAME}" #eg: "My awesome website"
export ADMIN_USER={ADMIN_USER} #eg: admin
export ADMIN_PASSWORD={ADMIN_PASSWORD} #eg: JKo$@sfjASD00w
export ADMIN_EMAIL={ADMIN_EMAIL} #eg: pedro@example.com

To set the SALT keys there are two alternatives:

I. Set random values through environment variable SHUFFLE_SALT_KEYS:

export SHUFFLE_SALT_KEYS=true

II. Set the corresponding values directly:

# Obtain random values from https://api.wordpress.org/secret-key/1.1/salt
export AUTH_KEY={YOUR_AUTH_KEY}
export SECURE_AUTH_KEY={YOUR_SECURE_AUTH_KEY}
export LOGGED_IN_KEY={YOUR_LOGGED_IN_KEY}
export NONCE_KEY={YOUR_NONCE_KEY}
export AUTH_SALT={YOUR_AUTH_SALT}
export SECURE_AUTH_SALT={YOUR_SECURE_AUTH_SALT}
export LOGGED_IN_SALT={YOUR_LOGGED_IN_SALT}
export NONCE_SALT={YOUR_NONCE_SALT}
  1. Bootstrap a new project from this repo:
composer create-project leoloso/pop-api-wp
  1. Wait for a few minutes ☕️😁
  2. ✅ The site is installed under:
    • 👉WordPress site: {YOUR_SITE_URL_WITH_HTTP}
    • 👉WordPress admin: {YOUR_SITE_URL_WITH_HTTP}/wp/wp-admin/
    • 👉PoP API: {YOUR_SITE_URL_WITH_HTTP}/posts/?action=api&datastructure=rest (REST for posts)

 

Enjoy!

on 4 Apr, 22:48

PoP now automatically generates the GraphQL-like query to fetch data from an external site

PoP now automatically generates the GraphQL-like query to fetch data from an external site
Blog

When fetching data for a component, PoP allows to point to an external domain. Until now, the external site had to have the same components installed as the origin site, to guarantee that the response would be what was expected by the origin site. Undoubtedly, this was less than ideal.

This is solved now, and very elegantly: whenever a component fetches data from an external site, it automatically generates the URL to query, including all the fields that need be retrieved. Similar to GraphQL, this request will fetch only the required data, and because it is automatically generated by the application, there are no possibilities of manual errors from creating the query, and if any field is updated or a new one introduced or removed, the query is automatically upgraded too. The automatically generated query looks like this (assuming, in this case, that nextapi.getpop.org is the external site).

Through this feature, PoP sites will be able to share data effortlessly. The developer will not even need to code a query to fetch data, but simply arranging all modules inside each other will do. PoP takes care of the rest.

As usual, enjoy!

on 1 Mar, 00:36

PoP is being migrated into components

PoP is being migrated into components
Blog

PoP has historically been called a “framework”. The reason for this is that, having started as a project for WordPress, its architecture expected to manage the whole website. Then, if you wanted to run PoP, it had to be a PoP website. It was an all or nothing situation.

However, this situation will soon be drastically improved: We are currently migrating PoP from a monorepo into a set of components, each of them hosted under its own repo, and all of them managed and tied together through Composer. The components can be installed independently of each other and, quite importantly, they will be CMS-agnostic. Having an architecture based on components, PoP is not expecting to take over the site any longer: components be added into any website to improve its capabilities, providing specific features (such as the component-based API)

Our aim is to release the components providing an implementation for WordPress and, in the future, provide an implementation for October CMS (given the availability of time and man power for the task). We expect to start releasing the first components around May 2019.

 

on 12 Dec, 18:40

The new PoP API can now be installed!

The new PoP API can now be installed!
Blog

We have started releasing the new PoP. This is the first time ever that PoP can be installed without fear of it breaking up: originally the codebase was not split into plugins, so if certain plugins were not activated, everything broke, greatly impairing the chance of developers being able to use it (for which the project barely has 50+ stars on GitHub). After 10 months of heavy work and tears, it started being ready. Hurray!

To download the new PoP API, simply head to the repository on GitHub and follow the instructions.

The overall migration is not finished though: only the API layer is ready, the rendering layers (for both client-side and server-side) are still under development, so they will be released in stages as soon as they are finished. We expect everything to be up by 2nd quarter 2019.

This is a major release: The new PoP API is fully based on components (we believe this is a primer among APIs, we’ve never seen or heard of one before), and will be able to offer unique capabilities, making it extremely easy to build any kind of website, from simple sites to decentralized social networks.

We will keep blogging about the new capabilities alongside each new release. Enjoy!

 

on 28 Oct, 16:43

A new PoP is coming soon, with a revamped architecture and plenty of exciting new features

A new PoP is coming soon, with a revamped architecture and plenty of exciting new features
Blog

Even though released as open source 2 years ago, PoP still doesn’t have installation scripts, making unfamiliar developers unable to install it (to this day, it has barely received 50+ stars in GitHub.) 8 months ago we finally started working on the scripts. In the process, we discovered several issues which had to be fixed too. Given the ambition of what the new PoP will be able to offer, and the amount of code developed during the last 5+ years which needs refactoring, after 8 months of work we still haven’t finished. If everything goes fine, the new PoP will be released around the end of 2018 or beginning of 2019.

After PoP is newly released, it will live up to the expectations of being called a framework: until now, PoP was pretty much limited to building community-focused social networks like MESYM.com, but through the new PoP it will be possible to build any type of website: static, dynamic, with/out users, with/out Single Page Application, with/out Server-Side Rendering, etc

There are 2 big announcements:

  1. PoP is being re-written, based on the following foundations:
    • Everything is a module
    • The module is its own API
    • Reactivity
  2. The codebase has been modularized and made CMS-agnostic, and plenty of new features have been developed

Announcement #1 is quite big on its own, so it will be dealt with in its own blog post. Below we deal with Announcement #2, describing some of the new features/enhancements produced during the past 8 months. These have not been uploaded to the Github repo yet, because the code in still under heavy development (we expect to do it around the end of 2018 or beginning of 2019).

Split the codebase up into countless plugins, allowing to progressively enable features

In the past, PoP was a huge monolithic codebase, in which you either got everything (a site such as MESYM.com) or nothing, since code was dependant in such a way that deactivating a feature may break everything else.

We have progressively fixed this by identifying pieces of code, analyzing their functionality, and placing them in a specific plugin. To this date, we have produced more than 50 plugins, and since we haven’t finished, it is still increasing.

Some plugins we have created deal with functionality, such as the following:

  • pop-userlogin: Allows users to log-in to the site from the front-end
  • pop-userplatform: Allows users to create an account on the site
  • pop-socialnetwork: It transforms the site into a social network, adding features such as liking posts, following users, and others

Some other plugins deal with the framework itself, such as the following:

  • pop-spa: Transforms the site into a Single Page Application
  • pop-ssr: Enables Server Side Rendering on the site
  • pop-resourceloader: Enables code-splitting

Until now, all code had been housed under a unique Github repo, but from now on every plugin will have a repo of its own. Having their own repository, each plugin will have its own documentation, and its own methodology for installation (some plugins may depend on Composer, others on NPM, others on GitHub subprojects, etc…)

Introduced dependencies among plugins

Plugins will be activated only if their dependencies are all active. Then, in the hierarchy from plugins described above, we can declare:

  • Activate pop-socialnetwork only if pop-userplatform is active
  • Activate pop-userplatform only if pop-userlogin is active

This way, the application will always work, such as avoiding compilation errors if a required PHP class is not loaded anymore.

Split the application into 4 layers: API, Frontend, Backend, and Implementations

Each plugin is divided into up-to-4 separate yet interconnected subplugins:

  • API: it deals with data-loading/posting for each module
  • Frontend: JS and CSS resources, Handlebars JS templates, general functionality when there is a browser client.
  • Backend: abstract classes establishing the configuration for a module
  • Implementations: actual module implementations, filling the configuration values as suitable

This architecture makes a module itself be modularized. Such fine-grained modular architecture allows for scalability/extensibility: developers can add their own custom implementations, replace the frontend JS/CSS files, establish a different configuration for a module, all without affecting the other portions of code.

In addition, if the Frontend layer is disabled, then the site barely returns the data in JSON, hence effectively becoming an application server. If we have all the layers enabled, the site will look like normal:

However, when plugin pop-engine-frontend is disabled, and we reload the page, we get this response instead:

Which viewed through a JSON formatter looks like this:

 

This is basically the JSON response containing all the data needed to render the page. Since disabling the frontend layer in the application, PoP assumes that only the data is needed and nothing else.

Re-organized the architecture to support other Content Management Systems

Because most of the code for PoP is plain PHP, it can perfectly be made CMS-agnostic, so that PoP could also be powered by other PHP-based Content Management Systems, such as Joomla or Drupal.

For this purpose, we implemented several interfaces (such as PoP_CMS_FunctionAPI, PoP_CMS_ObjectPropertyResolver and others), which list those functions that will be needed by PoP, expecting the implementing CMS to provide an implementation for them.

Currently, this feature is still under heavy development, with the function signatures being those used by WordPress (eg: functions names get_posts, get_users, etc, and also expecting the same parameters as by the WordPress functions) so it is not expected for other CMSs to be plugged in and work immediately; plenty of work is still required. However, the architecture has already support for the extending into other CMSs, so it must just be implemented.

Introduced interfaces to make functionality be plugin-agnostic.

Many modules in PoP rely on 3rd-party plugins for their functionality, and if that 3rd party plugin was not installed, then the component would not function. For instance, the events calendar in MESYM.com relies on plugin Events Manager for fetching the events; if Events Manager is not installed, then the website cannot display the PoP events calendar module.

We have now overcome this restriction by introducing interfaces to connect the 3rd party plugins with the PoP modules. For instance, for the events calendar mentioned above, we have now added interface PoP_Events_API listing down all the functions required by the calendar functionality, so that Events Manager just provides an implementation of this interface. This way, other plugins can also be used as functionality providers.

Implemented infinite data-loading

Similar to GraphQL, the site allows to fetch relational data, involving many relationships among object types and as complex as needed, such as:

Fetch all posts. For each post:
    Fetch the author. For each author:
        Fetch the posts the user has "liked". For each post:
            Fetch all its comments. For each comment:
                Fetch the comment author. For each author:
                    Fetch all the events which this user is attending. For each event: 
                        Fetch its location. For each location: ...

Which translates into object types being loaded like this:

post => user => post => comment => user => event => location => ...

PoP will fetch the data from the database, and replicate the relational structure of the data into the resulting JSON object:

database: {
    posts: {
        1: {author: 1, title: "This is a post", ...},
        2: {author: 2, ...},
    },
    users: {
        1: {name: "Leo", ...},
        2: {name: "Pedro", ...},
        ...
    },
    comments: {
        1: {author: 1, postId: 1, content: "This is a comment by Leo on post 'This is a post'", ...}
    },
    events: {...},
    locations: {...},
}

The data for each object is fetched only once from the database, and it is printed only once on the output (i.e. it is not nested), making it very efficient. Adding additional levels doesn’t incrase the complexity of the query operation (as in GraphQL), while being fully cacheable on the server side (unlike GraphQL.)

Allowed data to be marked as dynamic, so it will be sent to the frontend even when doing SSR

For optimization reasons, when doing Server Side Rendering (SSR) we don’t send the JSON object with all the database data, which may be quite big. However, this carried the issue that some data is actually dynamic, i.e. it will be used on the client to dynamically create views. Hence, for these situations, we had to send the whole database.

Now, through the newly added function get_dynamic_data_settings, it is possible to mark what data-fields are dynamic, and in that case only these are also sent to the frontend, and all static data, which is already rendered in the HTML generated in the server through SSR, is discarded, effectively sending less data down the wire.

Many more

  • Used PHP traits to re-use code, between classes in plugins PoP Engine and PoP Frontend Engine
  • Converted categories with particular functionality/inputs into custom post types (eg: Highlight custom post type)
  • Made blocks self-aware if they require lazy-loading based on the checkpoint status of their corresponding page, so that lazy-loading can be done automatically instead of through configuration
  • Streamlined code by grouping functionality (eg: function pop.DynamicRender.render), created interfaces (eg: FormComponent)
  • Split all Handlebars helpers into multiple files, as to only load what is needed and improve modularity
  • Moved all PoP JS functions under global variable pop, so as not to pollute the global scope anymore
  • Re-wrote plenty of modules from scratch
  • And many others

Sign up to our newsletter:

Welcome to the PoP framework!
Break the information monopoly

the PoP framework is open source software which aims to decentralize the content flow and break the information monopoly from large internet corporations. Read more.