The PoP team is very proud to announce the launch or Verticals, our own venture to provide decentralized social networks, and clusters of niche websites, to anyone:

We decided to call this venture Verticals, because it specializes on providing vertical portals. Our focus is on empowering communities, providing them the tools needed to be autonomous online (i.e. not dependant on any mainstream platform), and enabling them to connect to all related communities. Our objective is to foster the decentralization of the Internet.

So, what is Verticals about?

Launch your vertical portal, and combine the effort of your community

A vertical portal is an online community in which those people looking for a certain topic can meet up online. It is a unique access point to a certain industry, market niche, or area of interest.

Verticals creates decentralized vertical portals — information is accessed from a central point, which gets all data from a cluster of niche websites. The niche websites are autonomous: they can be accessed on their own and under their own URL, be owned by different people, and be attached to/de-attached from the vertical portal at any moment.

The PoP framework’s slogan is “Break the information monopoly”. Today, we are finally able to showcase our vision of what can be achieved through PoP, with the ultimate goal of decentralizing access to information on the Internet.

We have launched SukiPoP, a concept website demonstrating what we call a decentralized social network:

All content in SukiPoP is fetched, in real time, from several other websites/social networks, namely:

• MESYM, a social network on environmental issues from Malaysia
• Agenda Urbana, a social network on political activism from Buenos Aires, Argentina
• SDG-SSE, a content showcasing information related to the Sustainable Development Goals and the Social and Solidarity Economy

(These websites have been chosen randomly to be part of SukiPoP, with the only objective of demonstrating the concept; there is no common topic among them)

When visiting SukiPoP’s homepage, we are presented with a calendar of events (must click on the title to open the calendar). It can be seen how all events from all different websites are all displayed all together, in one single place, with different background colours to differentiate all websites:

Similarly, when scrolling down, we have the content feed, which is gathering content from all websites. To show it clearly, we visualize it as the list format and showing 3 items:

And when clicking on any external link, it opens the external page directly within SukiPoP:

The content aggregation takes place all throughout the website:

• when searching for information (for both content and users), the search will be performed on all websites, and the results will be displayed in one single place
• when viewing a hashtag’s page, it will show all content with that hashtag from all websites
• all events are shown on one single events map
• etc

The decentralization also works for posting content

Fetching content on real time is not the only feature: whenever we interact with the content coming from any website, such as adding a comment, recommending a post, or following a user, the operation will be directly executed on the original website (that is, either MESYM, Agenda Urbana or SDG-SSE), and not on the aggregator (in this case, SukiPoP).

However, the user experience is always the same, and the user can be made unaware to be interacting with external content. For instance, clicking on the “Add comment” link on a post from an external website, it opens the “Add comment” window as it would normally:

After filling in the comment, clicking on “Submit” will save the comment on the original website (in this case, MESYM). If the user is not logged into that website, a message will appear prompting the user to log in to MESYM, all within SukiPoP:

In other words, SukiPoP handles the user session for all of its aggregated websites, and makes sure users are logged in to the appropriate website whenever they need to post content there.

Even more, uploading files will also be done on the original website. After logging in, in that same “Add comment” window, we click on the “Add media” button, which will allow to insert images from, and upload images to, the original website (always MESYM):

And when clicking on the external post’s author “Send message” link, we can send a message to a user from MESYM all from within SukiPoP:

The end goal is to allow any user from any website, to be able to interact with all content and users from other websites, without having to be part of a centralized platform.

What comes next

SukiPoP is a work in progress, we are already working on incorporating the following elements:

• An overview of all external websites, having a unique style to represent them throughout the aggregator (always same background colours for all content from each website), and clearly visible to the user
• The possibility to filter content from specific websites (clicking on a checkbox next to the website name, it will show/hide all its corresponding content all throughout the website)
• Addition of a marketplace to each website, to create a decentralized marketplace (users will be able to buy products from different websites/shops, all at the same time, from a central place)
• Addition of categories, to classify all aggregated websites under common topics (eg: category ‘sports’ will show content from websites ‘football.com’, ‘tennis.com’ and ‘basketball.com’)
• The possibility to select, on runtime, from which subset of websites to show content from (eg: for a decentralized marketplace, select those websites whose shops are located near the user)
• The possibility to launch customizable social networks within SukiPoP for any visiting user; these social networks are to be hosted under SukiPoP’s cloud infrastructure, and feed its content directly to SukiPoP, in addition to being an autonomous website all by itself

As always, enjoy!

The great potential of PoP is materializing: the ability to fetch data from different domains, and combine it all inside a single component (such as a calendar, a map, a list of posts, a news feed or a marketplace), is almost ready!

If you check the homepage of this website, and click on item “Decentralized”, you will see an events calendar, fetching data and displaying the corresponding events from several different websites:

All of those events, whose background-colors are different, are being fetched from different domains, all in real time. Moreover, clicking on any event, either local or from an external domain, it will always open the event locally, and not in a new tab.

For instance, when doing right click => “Open page in a new tab” on any external link, such as event World Rangers Day 2017 (from website MESYM), the new tab opens the external website:

However, if we just click on it, as usual, then lo and behold!, it opens locally:

While the link is pointing to https://www.mesym.com/en/events/world-rangers-day-2017/, the browser URL bar is updated as https://getpop.org/en/external/?url=https://www.mesym.com/en/events/world-rangers-day-2017/.

It works for posting content too!

And this works not only for GET request, but also for POST operations. So we can interact with the source website, all from within the aggregator website. Actually, anything that can be done there, such as following users, recommending content, posting events, adding comments, etc, can also be done through the aggregator website, and it’s all transparent to the user. Even media files can be uploaded to, and accessed from, the external domains.

All interactions always present the same interface to the user, either performed locally or on an external domain. For instance, adding a comment to an event, either local or from another website, will open the same small window on the bottom-right of the page:

More developments are to be finished soon

There are still some things left to do, to complete the implementation of this feature:

• Cache external data with Service Workers, following the Offline First strategy
• Route external data through their own Content CDN
• Implement discoverability and autonomy, so that domains can announce that they are alive and want to join an aggregator website, and make their configuration information publicly available

We expect to have these features developed by the end of August 2017.

As always, enjoy!

Our latest implemented feature is a great one: all content in the website, either general content open to the public, as much as personal content that only logged in users can see, can also be sent through email, with no extra effort or duplicated work, because the email html code can be produced from the same source as the website pages. This is indeed code reusability to the extreme: creating a webpage can be used in both front and back-end (through server-side rendered html), and now also in emails sent to the users.

Even more, there is no need even to convert CSS to inline styles for the emails, since this is already taken care of automatically using library PHP CSS Parser (there is a limitation though: it only works for single-level classenames (eg: .btn); there is no support for other forms in the css, such as concatenated classnames (eg: .btn.btn-info), nested classes (such as .btn.btn-info), or html elements (eg: input[type="submit"]).

Digest with personal notifications, sent daily to the users

This way, it is now possible to create email digests with any content shown in the website, such as:

• Personal daily notifications
• Daily digests of new posts
• Daily digests of upcoming events
• Reminder emails to the users attending an event, on the day of the event (“Attend event” functionality must still be implemented #7)
• Feedback emails to review the quality of the service sent to a user buying a product, after it has been delivered (the market place must still be implemented #14)
• Others

All implemented new code can be found under wp-content/plugins/pop-frontendengine/library/css-to-style-conversion and wp-content/plugins/poptheme-wassup-automatedemails.

As always, enjoy!

There are several coding principles guiding the development of all code in PoP:

1. Reuse and leverage

All code we produce must be reusable, as much as possible, all through out the application. In addition, we use existing 3rd party (open source) libraries to provide the functionalities we need, and integrate them to the framework. Since the PoP framework has been done, all from the beginning, by just one person, reusing code from both internal and external sources been of extreme importance.

WordPress code reuse

WordPress currently powers 28% of all websites in the world, and their community of developers is huge. It seems that, for every new technology out there, there will be a plug-in implementing it in no time. By using WordPress in the back-end as a service, PoP leverages the power of the WordPress community, integrating their plugins into the framework.

Isomorphic rendering

Isomorphism means that developers can implement the code only once and use it everywhere. Those single pieces of code are Handlebars javascript templates, from which PoP is able to create the HTML for:

• front-end rendering, powering the Single-Page Application architecture
• server-side rendering (SSR), producing HTML on the server for the initial website load
• email digests and newsletters

Atomicity/Reusability of modules

A PoP module is either an atomic/reusable functionality, a composition of other modules, or a combination of the two. This way, modules can build upon each other, create complex layouts and structures from very tiny and simple components, as if playing with LEGO.

This scheme allows for implementing Separation of Concerns, in which a module tackles a specific problem and nothing else, and the collaboration among developers, in which several developers can be working alongside each other on the same module, without conflict.

Microservices

PoP’s decentralized architecture enables to create applications which get their data from different smaller websites, emulating microservices. This way, content that needs to be repeated in several websites can be extracted into an external entity and feed its data to everyone who needs it.

For instance, if you have 3 websites with ads, and all 3 need to show the same ad, instead of creating the same post with that ad image in all 3 websites, it is possible to create a 4th website, placed under ads.mydomain.com, create the ad just there, just once, and inject its data (in this case, the ad) into all 3 other websites.

2. Decouple

All code in PoP is decoupled as much as possible. We try to not depend on any specific software solution or vendor, and the implementation of the base code decouples the business logic, the presentation, accessing the database, and the configuration into different layers, which do not overlap into each other.

In addition, modules are composed in a top-down manner, and configured through dependency-injection, so that a developer overseeing the construction of the application at the top can change the behaviour of the different components, which are being built, at the same time, by other developers.

No dependency on WordPress (in theory)

Even though PoP currently is based on WordPress, this dependency was built loosely in the architecture, so that a different software system could replace WordPress if needed. Because PoP is built using PHP, other CMS solutions also based on PHP, such as Joomla or Drupal, could be plugged in without much trouble (this is in theory; in practice, there is still plenty of work to do to achieve an absolute decoupling, but the overall architecture has already been designed in this fashion and for this purpose).

Decoupling of business/presentation/database/configuration layers

All following elements are decoupled one from each other, and as such can be developed, concurrently, by different developers with different skills:

• View design (through Handlebars javascript templates) and configuration (through PHP objects)
• Javascript functions
• Styles
• Access to the database

Dependency injection

All modules can declare what properties can be configured, and allow their parent modules to modify this configuration in a top-down manner, in which upper modules take precedence over lower modules. This way, a developer overseeing the construction of the application can modify the behaviour of the application accordingly, and needs not bother the team building the components. This scheme effectively adds a layer of decoupling even among the development team.

3. Cache

PoP makes heavy use of caching, wherever possible. It implements several layers of caching:

Server cache (through WP Super Cache)

This is the standard cache: whenever a URL is requested, it caches the produced HTML in the server, so that the next request to the same URL needs not generate the response again.

Configuration cache

Because the configuration is decoupled from everything else, such as the object data, then it can be cached. Differently from the server cache, the configuration cache does not rely on the URL, but on the type of page that is requested, which can be applied more often.

For instance, if two event pages are requested, even though they have different object data (i.e. their title, content, date, etc), their configuration (i.e. the classnames, color, fonts, etc) will still be the same. Then, requesting the first event will generate the page as usual, but requesting the second one will only need to get the data from the database and nothing else, since the configuration can be reused after it was generated and cached for the first event. This way, the second request will be processed faster.

Content CDN + Asset CDN

A CDN is a network of servers which is located near the user, so requesting resources from it is generally faster than requesting them from the server. All static assets from the application (such as images, .js and .css files, PDFs, etc) can be normally cached in CDN, and PoP is no exception. However there is nothing extraordinary here.

There is magic, though, concerning the Content CDN: dynamic content, such as a post data, can also be cached in a CDN on its way back to the server. PoP indeed makes the best from both worlds: dynamic content generated in a CMS, such as WordPress, which behaves as if coming from a static site generator.

Service Workers

Finally, PoP implements the offline first strategy for caching all accessed content in a browser cache, so that it can be retrieved faster, and even when the user is offline. Through the use of the Cache then network strategy, PoP will serve content in no time, directly from the browser cache, but still notify the user if/when there is more up-to-date content.

Recently there has been a discussion among the WordPress development community on what javascript framework should be chosen to be added to WordPress in the near future, with React and Vue as the most likely candidates. In that same blog post, a comment was highlighting that the discussion is not focusing on how good each framework is at implementing those functionalities which sit at the very core of WordPress, such as: hooking, localization and dynamic activation/deactivation of modules.

Concepts alike to these functionalities have been implemented for the PoP javascript engine. In this blog post we will detail how our solutions have been designed and implemented.

Dynamic activation/deactivation of modules

An object called JSLibraryManager allows to register javascript function through objects. This object has function register, where JS objects can register themselves for each function they are implementing, and function execute which triggers the execution of the specified function in all objects which registered themselves as implementing it.

(function($){
JSLibraryManager = {

  // All the registered JS libraries
  libraries: [],

  // All the methods that each library can handle. Each method can be served by different libraries
  // This way each library can also serve common methods, like 'destroy', 'getState', 'clearState', etc
  methods: {},

  register : function(library, methods, highPriority, override) {

    var t = this;

    t.libraries.push(library);
    $.each(methods, function(index, method) {

      // override: allows for any library to override others
      if (!t.methods[method] || override) {
        t.methods[method] = [];
      }

      if (highPriority) {
        t.methods[method].unshift(library);
      }
      else {
        t.methods[method].push(library);
      }
    })
  },

  execute : function(method, args) {

    var t = this;

    var ret = {};
    var libraries = t.methods[method];
    if (libraries) {
      $.each(libraries, function(index, library) {

        ret['l'+index] = library[method](args);
      });
    }

    return ret;
  }
};
})(jQuery);

A javascript object then registers all its public functions under JSLibraryManager:

(function($){
popBootstrap = {

  tooltip : function(args) {
    
    var t = this;
    var pageSection = args.pageSection, targets = args.targets;

    jQuery(document).ready( function($) {
      
      targets.each(function() {

        var tooltip = $(this);
        var options = {
          placement : tooltip.data('tooltip-placement') || "top",
          title: tooltip.attr('title'),
        };

        tooltip.tooltip(options);
      });    
    });    
  },

  popover : function(args) {

    var t = this;
    var pageSection = args.pageSection, block = args.block, targets = args.targets;
    jQuery(document).ready( function($) {
      
      var options = {
        placement: 'auto',
        delay: {
          hide: 100
        },
      };    
      targets.each(function() {

        var popover = $(this);
        var popoverTarget = $(popover.data('popover-target')).first();
        var content = popoverTarget.html();
        var popoverOptions = $.extend({}, options, {content: content});
        popover.popover(popoverOptions)
      });  
    });  
  },  
};
})(jQuery);

//-------------------------------------------------
// Register all functions
//-------------------------------------------------
JSLibraryManager.register(popBootstrap, ['tooltip', 'popover']);

Finally, in the PHP object we specify what javascript functions must be executed on the newly created elements on the DOM, for each module:

class PoP_Template_Processor_PopoverLayoutsBase extends GD_Template_ProcessorBase {

  ...

  function get_block_jsmethod($template_id, $atts) {

    $ret = parent::get_block_jsmethod($template_id, $atts);  
    $this->add_jsmethod($ret, 'popover');    
    return $ret;
  }
}

The PoP engine in the frontend will execute the specified javascript functions for each module, by calling the method execute from  JSLibraryManager, which will invoke all registered functions under that name. If no javascript object has registered as implementing it, then nothing will happen.

When activating a WordPress plugin, it will load its .js files containing javascript objects registering themselves; when deactivating, the .js files will never be loaded, and so its javascript objects will never be registered. Hence, activating/deactivating plugins will indeed do the same at the frontend side.

Hooking

Adding actions and filters to javascript is a continuation of the strategy defined above. We can define any hook as a function, to be invoked under some name, under which can register those javascript objects implementing the hook.

Let’s see an example. Say that we need to modify some value, but we don’t know or care about who will modify it, we can simply call function execute from JSLibraryManager, passing the name of the hook and an object containing all values to modify alongside other needed values. Because javascript passes objects by reference, the modified properties of the object can be read again after the completion of the execution.

function fetchURL(url, target) {
  
  ...

  // Allow plug-ins to modify the url
  var args = {
    url: url,
    target: target
  };
  JSLibraryManager.execute('hookFetchURL', args);
  url = args.url;

  fetch(url);
}

Finally, we register the functions implementing the hooks, by registering an object and its functions under JSLibraryManager.

(function($){
HookObject = {

  hookFetchURL : function(args) {
  
    var t = this;
    args.url = t.add_query_arg(args.url, 'timestamp', Date.now());
  },

  add_query_arg : function(key, value, url) {

    url += (url.split('?')[1] ? '&':'?') + key + '=' + value;
    return url;
  }
};
})(jQuery);

//-------------------------------------------------
// Register all functions
//-------------------------------------------------
JSLibraryManager.register(HookObject, ['hookFetchURL']);

Localization

PoP does not attempt to provide localization features in the front-end. Instead, all values are already localized from the back-end, and sent as configuration values in the JSON code provided by the API.

In other words, we do not create a javascript template with this logic:

Your event is on {{localize "13/08/2017"}}

Instead, our Handlebars javascript template is like this:

{{titles.event-date}}

And then we generate the values on the back-end as it is usually done with WordPress, filling it in the configuration of a module, done in its corresponding PHP object:

class PoP_Template_Processor_EventLayoutsBase extends GD_Template_ProcessorBase {

  function get_template_configuration($template_id, $atts) {
  
    $ret = parent::get_template_configuration($template_id, $atts);

    $ret['titles'] = array(
      'event-date' => sprintf(
        __('Your event is on %s'),
        date_i18n(get_option('date_format'), strtotime('13-08-2017'))
      )
    );

    return $ret;
  }
  ...
}

Finally, the PoP engine in the front-end will merge the configuration with the javascript template, and produce the desired output.