For a proof of concept i’ve had the opportunity to re-implement a Docker InfraKit plugin instance plugin from scratch (in another language). To help anyone else who decides to do something similar, I thought it best to capture some of the key points that will need implementing when writing your own plugin.
Starting from Scratch
If you’re going to write a plugin for InfraKit in another language then you’ll need to ensure that you implement all of the correct interfaces, APIs and methods needed to allow expected behaviour. Also if your plugin may need to implement the capability to maintain and store expected state somewhere (perhaps locally or in an external store).
- UNIX Sockets, that can be accessed by the InfraKit CLI tool
- HTTPD Server that is bound to the UNIX Socket
- JSON-RPC 2.0 that will be used as the transport mechanism for methods and method results
- Methods to:
- Validate the instance configuration
- Determine the state of an instance
- Provision new instances
- Destroy instances
- Describe instance parameters (configuration that can be made through the JSON methods)
- Ideally garbage collection of sockets upon exit, leaving abandoned sockets in the plugin directory can be somewhat confusing to the CLI tool.
The UNIX sockets should always reside in the same location so that the CLI can find them, which is an
.infrakit/plugins/ directory inside the home directory of the user that runs that plugins, so my plugins reside in
/home/dan/.infrakit/plugins/ It may be apparent that as we’re using UNIX sockets then we’re effectively binding Infrakit to the file system of a single system, however it’s not a technical challenge to bind your plugin socket to a TCP port through tools such as
Typically you would (under the covers) create a UNIX socket and bind it to a TCP port of either 80/443. However in this implementation we create a UNIX socket and bind it to a file on the local file system, so instead of connecting to an IP address/port we speak to a file on the file system in the same manner.
After some head scratching around some inconsistencies with the API it was decided by the InfraKit developers to move to JSON-RPC 2.0 as the method for calling procedures. This simply wraps methods and parameters into a standardised JSON format, and expects the method results to be reported in the same standardised way. It’s a slightly bizarre standard as only makes use of the
HTTPD POST method and expects a
HTTP 200 OK returned even in the event that functions fail. The reason for this is that the errors should be encapsulated in the returning JSON, essentially moving the error reporting down (or up, i’m not sure) the stack.
The typical workflow for provisioning through the Instance plugin is as follows:
- After a user commits some configuration the InfraKit CLI will parse it and pass the correct parameters to the instance plugin that is defined in the configuration.
- The Instance plugin will then read through the parameters and use them when interacting (API call, CLI scraping, vendor specific protocol API etc.) with the Infrastructure provider.
- The Provider will then provision a new instance.
- Once complete the provider will return the new instance details to the Instance Plugin.
- The Instance plugin will report back to InfraKit that the instance has been provisioned.
The workflow above is pretty simplistic, but I presume will be the common steps for interacting with most infrastructure providers however depending on your provider there can be some caveats. The main caveat relates to infrastructure that can’t be immediately provisioned and can result in InfraKit provisioning multiple instances as the instance won’t have been provisioned within the group plugin polling window. If your instance is going to take a number of minutes to provision and you poll for it every 30 seconds, it will result in InfraKit trying provision a new instance every timeout and once the instance count matches what is required it will have to start destroying excess instances as the come online.
The possible solutions are to either increase the poll timeout so that the instance will be provisioned within that window and be reported back as created, modify the group plugin, or develop some intelligence into your instance plugin. The plugin that I developed as part of a PoC had instances that would take ~4 minutes to be provisioned, which meant that the instance plugin needs a method for it to track what it had provisioned so that it could then check with the provider what state the instances were currently in.
There are a number of different ways in order for a plugin to handle state, it could well be that just querying the provider will result in the instance state. However some providers may make it difficult to identify instance created by InfraKit and instances created directly etc. So in my example I needed the plugin to keep track of instances and maintain state during plugin restarts.
- InfraKit requires the list of instances so that it can make sure that resource is allocated correctly, so it asks the instance plugin to describe all of its instances.
- The Instance plugin will iterate through all of the last known state information.
- Each of the instances in it’s last known state will be checked with the provider to determine if they’re created/in-progress/failed.
- State information is updated, with the latest information
- The instances that are created or in-progress are listed back to InfraKit, where it will either be satisfied with instance count or require some more instances provisioned.