AEP 1: Apiman policy plugins with full Angular UI/UX

[msavy]

Enhancement Proposal: Apiman policy plugins with full Angular UI/UX

Background

Apiman offers the ability to extend the functionality of Apiman via policy plugins. These plugins are a WAR file (essentially a ZIP with bundled dependencies and some metadata), comprising of two components:

Google doc version available here for those prefer that layout

• Policy implementation (Gateway): implements the functionality that does the actual work, loading the appropriate configuration and applying whatever arbitrary functionality is required to proxied traffic.
• User interface and metadata (Manager): provides some metadata describing the policy and a detailed JSON Schema describing and validating the allowed inputs for the policy.
  • These policies can be loaded into the manager at runtime via unique Maven coordinates (from Maven central, Sonatype repo, filesystem, etcetera).
  • The JSON Schema is transformed into an HTML form that allows users to input values and have those validated against the schema. For example, ranges, allowed values, types, and regexes.

Motivation

For simple policies JSON Schema is more than sufficient. It provides quite a rich range of sanity checking and even some simple conditional logic (e.g. oneOf, anyOf, or, etc), however for more sophisticated use-cases it simply can’t provide the level of customisation that is needed in several areas:

• Functionality: you are limited by what the JSON Schema specification and validations are able to describe. This means you can’t readily do dynamic lookups or domain-specific validation.
  • Broadly, you can only provide standardised form inputs and descriptions, and if your use case requires something more complex, it simply isn’t possible.
  • For example, you can’t invoke a remote service to pre-populate data (e.g. look up public key).
  • You can’t validate a format that isn’t describable by simple regex.
• User experience: to provide the best user experience it is often necessary to have a fully-customised UI.
  • It enables designers to provide an optimal workflow that provides users a smooth experience that matches the functionality of the plugin and does not require any awkward interactions (e.g. please paste something here, but not here).
  • Minimises the possibility of users making mistakes by providing fully-customised validation and clear error messages.
    • Not just generated error messages saying that you failed to match against a complex regex.
    • Can do complex validation such as parsing and validation of a certificate or key. This is impossible in a schema alone.
    • This means that we don’t discover certain configuration errors very late (e.g. pasted a key but the format is wrong or it’s the wrong thing).
  • Certain interactions and mechanisms that are impossible with JSON Schema. We can do anything we like with a customised UI, with commonly-wanted elements such as dials, auto-lookups, wizards, pop-up panes, drill-down, trees, (etc ad infinitum).

Existing solution

As some important background, I will outline the runtime program flow for the existing JSON Schema based Apiman policy plugins solution. As this proposal mainly focuses on the Apiman Manager, we will largely omit the gateway aspects.

• An administrator triggers the manager to load a new plugin via its unique Maven GAV; this is a type of URI commonly used for unique artifact identification in the Java/JVM ecosystem.
  • This load can be triggered via the Manager UI’s plugin section or API (either of which require administrator permissions).
• The Apiman Managerloads this from a repository (e.g. Maven central, Sonatype repo, local filesystem, etc).
• The fact the policy plugin is loaded is stored in the Apiman Manager database, including various metadata.
• When a user later attaches this policy plugin the UI requests the backend provide the corresponding JSON Schema, and uses a Javascript library and other ancillary code to render the schema into a form.

Proposal

For fully customisable Angular UI in policy plugins

Considered approaches

Previously, we had been primarily considering whether it was possible to bundle a full Angular module into Apiman policy plugins somehow and load this dynamically. This would essentially involve serving a static version of the Angular application to the client, and then dynamically enriching it when a plugin is loaded.

This proved to be challenging for several reasons. Foremostly, this does not seem to be well supported in the Angular community or codebase. Whilst Angular templates are capable of being compiled dynamically in the browser, the other aspects (e.g. components, pipes, directives, etc), generally expect to be compiled during a discrete compilation phase. Pursuing this approach would likely be complex and potentially fragile, breaking between Angular releases.

Goals

In brief summary, the functionality has the following requirements and characteristics:

• Extend the UI with a new Angular module(s) with arbitrary functionality.
• Allow this custom policy UI be displayed when the policy is added to an API (or plan, client, etc).
• Plugins are installed by an administrator.
• Plugins are installed and upgraded relatively infrequently.

Non-Goals

In reviewing these requirements, it became clear that the following are **not **requirements:

• That the code update has to be dynamically injected into the running Angular application.
  • Given the infrequency of the plugin updates, a reload is acceptable.
  • Mechanisms that prompt the user to update to the latest version are well-established.
• The injection/update has to be near-instantaneous.
  • As this is an infrequent administrator activity, waiting a modest amount of time to rebuild a new UI is acceptable.

Proposed approach

Given these requirements and non-requirements, I propose the use of code injection and recompilation. The flow would be approximately as follows:

• An administrator requests the Manager API to load a new policy plugin via its Maven GAV via the Angular app.
• The Manager API retrieves the plugin from an artifact repository.
  • The loaded artifact is inspected, and in this case the manager discovers that the plugin contains an Angular module (metadata specifics TBD).
• The Manager API responds to Angular app and indicates that recompilation is required for this plugin; the Angular app displays an appropriate wait indicator to the user.
• The Manager API invokes the UI service with the plugin’s GAV; the UI service recompiles the Angular app to include the new plugin’s UI module.
  • Specific mechanism TBD, but options include:
    • a system call (shell out) from the Java process to rebuild;
    • a special-purpose docker container invoked from Java with the results captured on-disk;
    • a standalone JS-based recompilation service that provides a programmatic interface for the UI Service to invoke via HTTP or similar.
• Upon completion of recompilation the UI service informs existing instances of the Angular app of availability of the new version (specific mechanism TBD, e.g. long-polling, WS, SSE, etc).
• User triggers a reload which loads the latest version including the plugin UI.

Benefits of this approach

• Achieves the core functionality requirements.
• Simple and robust.
  • Uses the standard tooling and approach to building and deploying an Angular application.
  • Implementation is not complicated.
  • Plenty of existing libraries are available to implement this approach.
• Code injection/enrichment/generation is a well-understood problem space. Whether injecting via an AST or a simple template.
• Refreshing the browser to deploy a new version of a SPA is a common and well-understood mechanism.
• Full recompilation ensures the application is well-optimised.
• Easily testable and does not require any browser-specific hacks.

Risks & mitigation

Browsers must be reloaded to see the new UI reflected.

We will display a notification to users of the application when a new version has been deployed, prompting them to reload when they are able to. This allows users to save any in-progress work before reloading.

As outlined in the following section, installation of new plugins is not frequent, so should a reload not prove overly burdensome to users.

In certain cases we can safely automate this process. For example, if the user navigates to a policy that is not built into the version of the application they are running, then reloading the application and navigating back to the same location would be acceptable and relatively transparent.

We may need to force a reload at a safe point if the user does not take any action after an extended period of time.

Recompilation is slower than dynamic/on-demand code extension.

This issue is not critical as loading of policy plugins is an administrator action and tends to be infrequent (occasional loading of plugins and version upgrades).

Full JS stack required to enable rebuilding process

This is a key area to manage.

Javascript, Node, NPM/Yarn, etc, will be required. There are issues around ensuring the appropriate tooling with the correct versions is available in the environment (Apiman runs on a wide variety of platforms, so the assumptions we can make are fairly minimal).

Furthermore, substantial memory resources can be consumed temporarily during compilation, which should be accounted for.

Potential approaches were outlined above, but could include use of a pre-packaged “known good” build setup of some sort (e.g. via a docker container).

JS dependency resolution will be required during build

The contributed UI module may have its own dependencies which will need to be resolved during the recompilation.

We will need to ensure:

• Dependencies can be successfully resolved, potentially including proxy settings, alternative repositories, etc.
• Assess vendoring for dependencies (bundling dependencies). This has some issues if there are clashes with the main project.
• Merge together the main project and the contributed module.

Build failures and dependency/interface issues

A variety of build issues are possible during the JS build process which could result in a recompilation failure, they include:

• Code incompatibilities between the plugin and the main Apiman UI.
• Code errors in the contributed code.
• Library incompatibilities.

To mitigate this we should:

• Avoid dependency changes that will break compatibility (both of libraries and the main UI).
• Demarcate and document public APIs from private to prevent plugin authors relying on code that could change between versions and cause breakages
• Regular automated testing that includes loading all community plugins into the Apiman UI to ensure they still function as intended.
  • Cypress is likely to be a good candidate for this, and perhaps we could allow authors to bundle tests into their plugin so that it can be tested cohesively.

[msavy]

/CC: @EricWittmann @volkflo @outofcoffee @kahboom @BastianGem (and anyone else interested :-)).

[volkflo]

I added comments in the linked doc because it is easier to follow the discussion.

[msavy]

Added responses to your points, but will repeat them here for others:

Goals & characteristics:

• The purpose of this is to bring parity (as far as we can!) between inbuilt policies and plugin policies. Once we get into the real nitty-gritty details, I'm sure there may be some differences - but I'd like to minimise that as much as possible.
• This means people downstream of Apiman can create 'first-class' policies with full UIs.
• Many of these will be very domain/platform-specific policies that don't make sense to contribute to the main Apiman project.

Reloading/restart:

• The restart/reload is only for the Angular in-browser application, not for the whole Apiman application. The Java bits have no problem with dynamic loading/extension as things stand.
• Plugin lifecycles (especially of contributed plugins) are separate to the main application. So people need to be able to update/upgrade potentially.

Libraries to achieve this:

• Libraries to achieve compilation just refers to things like: Gulp, Yarn, NPM, Node, Angular CLI, Typescript AST mutation (e.g. ts-morph), etc. I'm going to have some additional conversations with some friends who have more experience in this area.

Issues with having a Docker-based 'build agent':

• As per the doc, this is certainly something I'm also concerned about. I do not want to 'bake in' any requirement for Docker into Apiman, but this approach would provide a near-guaranteed reproducibility, whereas.
• Concerns around DIND performance I think are slightly less troublesome here given the infrequency of the need to recompile. But some places do block DIND - certainly a problem.
• DOOD is great, but quite a lot more complex to set up and isn't available everywhere.

Whether vendoring is the chosen approach for dep management (and offline):

I don't know yet. Vendoring is one approach (see second bullet point).

As outlined, there are multiple approaches with different pro/cons. It's an area that requires some though, definitely.

It's a tricky area because the plugin has its own dependencies, as well as the main project - so these need to remain compatible.

These will naturally change a bit over time as we fix issues, etc.

I also agree that it would be preferable to support fully-offline recompilation to avoid annoying issues with having to resolve repos.

I'm not sure if that would always be possible because the vendored versions might drift out of alignment, whereas on re-resolution mutually compatible versions might be found; this is one of the topics I'm interested in learning a bit more about (i.e. what's the best choice here). Hoping to speak to a few folk on this one and do some additional reading!

[EricWittmann]

This sounds like a really interesting concept/idea to me! I think a short POC is in order, to explore how feasible the approach might be. But it's a really nice alternative to attempting some sort of dynamic federated module approach. I can't see any reason why it wouldn't work. There is a lot of opportunity for caching as well, or even crowdsourcing the builds. Lots of options here!

[msavy]

@EricWittmann thanks for the mention of Dynamic Module Federation. Until you mentioned it when we chatted, I had not heard of it.

I will definitely consider it as one of the potential approaches, if it's viable for something like Angular.

Am I looking at the right thing here? https://webpack.js.org/concepts/module-federation/

[msavy]

BTW, if you have any good resources on dynamic federated modules I'd be interested to read it (bunch of google results, but not sure I'm looking at the right thing).

[agebert]

Marc thank you for the detailed proposal.

If you develop such a policy plugin you will certainly have a development environment and a production environment. For the development environment I see no problem for using some resources for compiling and Internet access for package resolution.
On the other hand if you want to deploy it to production it would be best to have as much as possible prepackaged and perhaps only bundling the Angular application. The same step you need if you want to upgrade Apiman itself.

[msavy]

Off this thread Eric provided me a couple of pointers for the new Webpack 5 federated modules stuff (mentioned in the other subthread briefly):

https://webpack.js.org/concepts/module-federation/
https://github.com/module-federation/module-federation-examples

I also found a couple of others that are relevant for Angular:

https://www.angulararchitects.io/aktuelles/the-microfrontend-revolution-part-2-module-federation-with-angular/
https://github.com/angular-architects/module-federation-plugin/blob/main/libs/mf/README.md

An approach that definitely deserves some attention. In some ways it's half way between recompilation and a fully dynamic approach -- but Web itself provides support for this.

When I have more time I'll do a small PoC for both approaches (perhaps weekend, we'll see).

[msavy]

Future Marc here 😂

I think as Module Federation has improved, it is obvious to me that this is the correct way to go about things.

It's still potentially non-trivial, as we still need some level of compatibility (and JS dependencies break compatibility way too frequently). But it looks easier to maintain than a custom recompilation approach.

When I have time I'll do a POC.