Quickstart¶

Using pop to create a plugin oriented project is easy. This tutorial will help you learn how pop works and how to make a project. Once you understand how to build some basic tools in pop you can use the pop-create tool to make setting up new projects easy.

Note

A repo with the full working example can be found here.

Getting Started¶

Start by installing pop and making a new directory for your project:

$ pip3 install pop
$ mkdir poppy

Now outside of the newly created directory, create a simple python script called run.py to create the hub and start the plugin system.

Note

Normally a python project uses setuptools and a setup.py file. Because this tutorial is about pop we skip this part and use the run.py script. This can make development easier because you can run your application directly from your checkout. pop can be suplimented with a program called pop-create that makes these files and a stock setup.py file for you for development convenience! But this tutorial is about learning, so save the pop-create program for when you better know pop!

The hub is the root of the namespace that pop operates on. Don’t worry, it is not that complicated! Think of the hub like a big self variable that is shared across your entire application. The hub allows you to save data that is relative to your plugins while still allowing that data to be shared safely across the application.

# run.py

import pop.hub

# Create the hub
hub = pop.hub.Hub()
# Load up your first plugin subsystem called "plugins"
hub.pop.sub.add(dyne_name="poppy")
# Call the cli function that initializes the app
hub.poppy.init.cli()

This script has created your hub and loaded up your first subsystem, or sub. The dyne_name option tells pop where to load up the python package that contains the plugins. poppy/conf.py has a dictionary called DYNES that maps the source code files for plugin subsystems to directories that contain the subs.

Note

To learn more about the hub take a look at our doc on the hub and how to use it: hub_overview

Now lets create the python package and make it start to work! Make a new directory called poppy as the base python package and then another for your plugins.

$ mkdir -p poppy/poppy

Now that you are in the new poppy directory create the new plugin subsystem’s initializer. Create a file called poppy/poppy/init.py and give it an __init__ function. Like a class you can initialize a new plugin subsystem, or a new module in that function.

# poppy/poppy/init.py


def __init__(hub):
    print("Hello World!!")

Note

Your first sub has been created! To learn more about making subs check the doc here: Plugin Subsystems Overview

Now that you have a plugin with an initializer you can run it! Go back to the same directory as the run.py file and execute it.

$ python3 run.py

With a project up and running you can now add more plugins, more code and more subsystems!

Note

When you make a new sub, that sub follows a pattern. Patterns are an important part of Plugin Oriented Programming. Get to know the basics first! But then spend a few minutes learning about patterns here: Subsystem Patterns. Just so you know, the pattern you just started is called the spine pattern.

Adding Configuration Data¶

Now that you have the basic structure of your application you can easily add configuration data to your project.

Loading configuration data into a project looks easy at first but quickly becomes difficult. To solve this issue pop comes with a system to make configuration loading easy.

When loading configuration data, the data can come from many sources, the command line, environment variables, windows registry, configuration files, etc. But certain sources should overwrite other sources; config files overwrite defaults, environment variables overwrite config files and cli overwrites all. Also, you end up defining default configuration values and parameters in multiple places to enable supporting multiple mediums for configuration input. Finally, you only want to have to document your configuration options in one place.

The config system in pop solves this issue by making a single location where you can define your configuration data. You can also merge the configuration data from multiple pop projects, just like you can add other pop projects’ plugin subsystems to your project’s hub!

Note

That’s right! I just said that you can merge entire applications together onto one hub and bring in all the configuration data too! To learn more about this, take a look at the doc on merging applications: Application Merging Overview

Using the config system, is easy! Create a file called poppy/conf.py and populate it with your configuration data.

# poppy/conf.py

CLI_CONFIG = {
    "addr": {
        "options": ["-a"],
        "default": "127.0.0.1",
        "help": "The address to present the rpc server on",
    },
    "port": {
        "options": ["-p"],
        "default": 8888,
        "help": "The port to bind to",
    },
}

CONFIG = {}

Now lets change the __init__ function in poppy/poppy/init.py to load up the project’s config!

# poppy/poppy/init.py


def init(hub):
    hub.pop.config.load(["poppy"], cli="poppy")
    print(hub.OPT.poppy.addr)
    print(hub.OPT.poppy.port)

Now the configuration data has been loaded, if you run run.py with –help you will see all of your configuration options available. The configuration options will now be made available on the hub under the OPT dict and under the name of the imported project.

This allows for configuration data to be loaded from multiple projects and still cleanly namespaced. So the values of our configurations will be available on the hub:

hub.OPT.poppy.addr
hub.OPT.poppy.port

Now you can use the default IP address and port, or you can pass in different values when you start up the (to be developed) server.

$ python3 ./run.py
127.0.0.1
8888

$ python3 ./run.py --addr 0.0.0.0 --port 8080
0.0.0.0
8080

Note

The config system is very powerful and expansive, take a look at the docs on the conf system to get to know more of the available options and features. It is made to solve many problems that occur when loading configuration data: Configuration Reading The Integrate System

Adding More Plugin Subsystems¶

Next lets create a new plugin subsystem. This makes a new namespace on the hub and allows us to create a pattern in pop So there are a few more new terms to learn!

A plugin subsystem is typically referred to as a sub. This is a namespace on the hub that defines the new set of plugins. Using these namespaces on the hub allows you to set variables on the hub that are defined as to how they should be used based on where they exist. Data on the hub should only be written by relative plugins, but can be read globally.

Note

Remember how I mentioned patterns before? If you are curious, the sub we are making now follows the router pattern. Subsystem Patterns

When you create a new sub it should follow a pattern. These patterns define how the sub interacts with your application. We will start by making a simple pattern called the library pattern. This pattern means that modules have functions that are generally available.

When the hub is created it comes with a sub called pop. The pop sub comes with the functions we need to add our own hub. Now you can execute hub.pop.sub.add to add a new plugin subsystem. Remove the previous print statements, and add the subsystem:

# poppy/poppy/init.py


def init(hub):
    hub.pop.config.load(["poppy"], cli="poppy")
    hub.pop.sub.add(dyne_name="rpc")

We will also need to update the DYNE dictionary in our conf.py so that pop is aware of where the “rpc” code exists:

# poppy/conf.py
DYNE = {"poppy": ["poppy"], "rpc": ["rpc"]}

Now that we are able to load up a new subsystem we need to define it in our code! Start by making a new directory inside of poppy/ called rpc. When we added the new sub we specified the path to find the rpc sub to be in the rpc dyne.

Now create the poppy/rpc/init.py file and make an rpc server. This rpc server will expose all of the functions in the rpc plugin subsystem over a simple http server.

# poppy/rpc/init.py

import asyncio
import aiohttp.web


def __init__(hub):
    hub.rpc.APP = aiohttp.web.Application()
    hub.rpc.RUNNER = aiohttp.web.AppRunner(hub.rpc.APP)
    hub.rpc.ROUTES = [
        aiohttp.web.get("/", hub.rpc.init.router),
    ]
    hub.rpc.APP.add_routes(hub.rpc.ROUTES)


async def run(hub, addr: str = None, port: int = None, **kwargs):
    await hub.rpc.RUNNER.setup()
    site = aiohttp.web.TCPSite(hub.rpc.RUNNER, host=addr, port=port)
    await site.start()
    while True:
        await asyncio.sleep(0.1)


async def router(hub, request):
    try:
        data = await request.json()
    except:
        data = {}
    if "ref" in data:
        result = {}
        result["ref"] = await getattr(hub.rpc, data["ref"])(**data.get("kwargs"))
        return aiohttp.web.json_response(result)
    default_text = """example: curl -X GET http://{0}:{1} -d '{{"ref": "math.fib", "kwargs": {{"num": "11"}}}}'\n""".format(
        hub.OPT["poppy"]["addr"], hub.OPT["poppy"]["port"]
    )
    return aiohttp.web.Response(text=default_text)

As you can see, this uses the aiohttp library, and will need to be installed (and added to your requirements/base.txt):

$ pip3 install aiohttp

Now let’s have poppy.poppy.init.cli() call rpc.init’s run() function:

def __init__(hub):
    print("Hello World!")
    hub.pop.sub.add(dyne_name="rpc")


def cli(hub):
    hub.pop.config.load(["poppy"], cli="poppy")
    print(hub.OPT.poppy.addr)
    print(hub.OPT.poppy.port)

    kwargs = dict(hub.OPT.poppy)

    hub.pop.loop.create()

    coroutine = hub.poppy.init.run(**kwargs)
    hub.pop.Loop.run_until_complete(coroutine)


async def run(hub, **kwargs):
    await hub.rpc.init.run(**kwargs)

Congratulations! You now have a working rpc server that takes json requests and routes to plugins in the rpc sub. Now we just need to make a module in the rpc sub to route the requests to, lets call this file poppy/rpc/math.py:

async def fib(hub, num=10):
    num = int(num)
    if num < 2:
        return num
    prev = 0
    curr = 1
    i = 1
    while i < num:
        prev, curr = curr, prev + curr
        i += 1
    return curr


async def triple(hub, num=10):
    num = int(num)
    return num * 3

Now your rpc server can compute the Fibonacci sequence. So lets start up the server with the run.py script and then hit it with a curl command:

$ python3 ./run.py
======== Running on http://127.0.0.1:8888 ========
(Press CTRL+C to quit)

# Get a Fibonacci sequence using the generic router function

$ curl -X GET http://127.0.0.1:8888 -d '{"ref": "math.fib", "kwargs": {"num": "11"}}'
{"ref": 89}

# Call the Math Triple function using the generic router function

$ curl -X GET http://127.0.0.1:8888 -d '{"ref": "math.triple", "kwargs": {"num": "33"}}'
{"ref": 99}

# Request the root url. If you don't pass in any data it will respond with
# an example command you can run.

$ curl -X GET http://127.0.0.1:8888
example: curl -X GET http://127.0.0.1:8888 -d '{"ref": "math.fib", "kwargs": {"num": "11"}}'

Now that you have a project up and running you can play around with extending what pop can do and get familiar with it.

Docs Review¶

In this doc we introduced a lot of concepts, this is a whole new programming paradigm! To become more familiar with Plugin Oriented Programming and pop we already introduced these docs:

What is a hub and how to use it:: hub_overview
What a sub is and how to use it:: Plugin Subsystems Overview
What patters are and some examples of patterns that can help you start thinking in pop: Subsystem Patterns
How the built in configuration loading system conf works:: Configuration Reading and The Integrate System
How the concept of app merging works:: Application Merging Overview

Next Steps¶

Now that you have the tools you need to make pop work you will be able to start understanding how to think in and really use the power behind Plugin Oriented Programming! Take a look at these docs to get a better overview of Plugin Oriented programming:

Learning Plugin Oriented Programming¶

Learning and thinking in Plugin Oriented Programming starts here, it is a short doc trying to outline how to think about your applications so they can all be truly Plugin Oriented: Learning POP

The Story Behind Plugin Oriented Programming¶

Plugin Oriented Programming deviates from many of the norms in software development while working to evolve to the modern way of developing. Learn about Thomas Hatch and how he came up with the Plugin Oriented Programming paradigm: Origins of POP

Quickstart¶

Getting Started¶

Adding Configuration Data¶

Adding More Plugin Subsystems¶

Docs Review¶

Next Steps¶

Learning Plugin Oriented Programming¶

The Story Behind Plugin Oriented Programming¶

pop

Navigation

Related Topics