Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Image Serialization Plugin #906

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

mackoj
Copy link
Contributor

@mackoj mackoj commented Sep 15, 2024

This PR need to be merged after #905 that lays the groundwork for the usage of plugin inside swift-snapshot-testing.

Overview

This PR introduces a plugin for image serialization that all existing strategies will be using by default, enhancing the flexibility and extensibility of image handling within the framework. This plugin allows for the integration of various image formats through plugins, which can be dynamically discovered and registered at runtime. The core components of this implementation include the ImageSerializationPlugin protocol and the ImageSerializer class, designed to manage and utilize these plugins.

Key Features

  1. ImageSerializationPlugin Protocol: Defines methods for encoding and decoding images, allowing conforming classes to support various image formats. It combines the functionalities of ImageSerialization and SnapshotTestingPlugin for seamless integration with the existing plugin system.

    • imageFormat: A static property that specifies the format supported by the plugin.
    • encodeImage(_:): Encodes an image into the specified format.
    • decodeImage(_:): Decodes image data back into an image.
  2. ImageSerializer Class: Manages the encoding and decoding of images using available plugins. It:

    • Retrieves plugins from the PluginRegistry.
    • Encodes and decodes images using the appropriate plugin based on the requested format.
    • Falls back to PNG encoding/decoding if no suitable plugin is found.
  3. Dynamic Plugin Discovery: Plugins conforming to the ImageSerializationPlugin protocol are automatically discovered and registered using the Objective-C runtime, making it easy to extend supported formats without modifying core functionality.

Benefits

  • Extensibility: Easily add support for new image formats that works with all the already existing strategies through plugins without altering the core library.
  • Modularity: Keep the core library focused on its primary responsibilities while enabling third-party developers to extend functionality.
  • Dynamic Behavior: Plugins are discovered and registered automatically, reducing manual configuration and enhancing ease of use.

This PR lays the groundwork for a more modular and flexible approach to image serialization within SnapshotTesting, setting the stage for future enhancements and customizations through dedicated plugins.

Examples of plugins that use the ImageSerializationPlugin

Warning

These implementations are tests and prototypes, not production-ready. They are simply here to demonstrate how easy it is to use the ImageSerializationPlugin.

@mackoj mackoj marked this pull request as ready for review September 15, 2024 00:08
@mackoj mackoj mentioned this pull request Sep 15, 2024
@mackoj mackoj force-pushed the faet/ImageSerializationPlugin branch from 21e1a8d to 87ebba0 Compare September 16, 2024 09:40
@mackoj mackoj force-pushed the faet/ImageSerializationPlugin branch 4 times, most recently from 4e6843c to ddac472 Compare September 16, 2024 21:08
@mackoj
Copy link
Contributor Author

mackoj commented Sep 27, 2024

I hope this better explains what I’m trying to achieve with this PR. The FilePlugin doesn’t exist yet, but it’s a potential future plugin (prototype implementation not finished).

graph TD
    Dev --> A(Strategies)
    Dev -. "Third-Party" .-> B(Strategies)
    Dev --> C(Strategies)
    Dev -. "Third-Party" .-> D(Strategies)

    A --> swift-snapshot-testing
    B --> swift-snapshot-testing
    C --> swift-snapshot-testing
    D --> swift-snapshot-testing

    subgraph swift-snapshot-testing
        Plugin
        Plugin --> Image
        Plugin --> File
        Plugin --> Diff
    end

    Image -- "Default Value" --> PNG
    Image -. "Third-Party" .-> JXL
    Image -. "Third-Party" .-> HEIC
    File -- "Default Value" --> Local
    File -. "Third-Party" .-> SFTP
    File -. "Third-Party" .-> Database
    Diff -- "Default Value" --> Pixel
    Diff -. "Third-Party" .-> Hash+Pixel
    Diff -. "Third-Party" .-> MSE

    %% Style adjustments to highlight default values
    style PNG stroke:#FF0000,stroke-width:3px
    style Local stroke:#FF0000,stroke-width:3px
    style Pixel stroke:#FF0000,stroke-width:3px

    %% Style adjustments for optional nodes
    classDef optionalNode stroke-dasharray: 5,stroke:#333,fill:#E0E0E0,color:#000000

    linkStyle default stroke:#005f73
    classDef teal fill:#006d77,stroke:#333,color:#FFFFFF
    classDef lightTeal fill:#83c5be,stroke:#333,color:#000000
    classDef blue fill:#028090,stroke:#333,color:#FFFFFF
    classDef purple fill:#6a4c93,stroke:#333,color:#FFFFFF
    classDef lightBlue fill:#b2f7ef,stroke:#333,color:#000000
    classDef orange fill:#ff6700,stroke:#333,color:#000000

    Dev:::teal
    A:::lightTeal
    B:::optionalNode
    C:::lightTeal
    D:::optionalNode
    swift-snapshot-testing:::blue
    Plugin:::purple
    Image:::purple
    File:::purple
    Diff:::purple
    PNG:::purple
    JXL:::optionalNode
    HEIC:::optionalNode
    Local:::purple
    Database:::optionalNode
    SFTP:::optionalNode
    Hash+Pixel:::optionalNode
    MSE:::optionalNode
    Pixel:::purple
Loading

@mackoj
Copy link
Contributor Author

mackoj commented Oct 6, 2024

What has changed regarding the public API of swift-snapshot-testing.

Inside SnapshotTesting target:

  • PluginRegistry.registerPlugin has been added to manually register a plugin(on Apple platform it works automatically).
  • Inside AssertSnapshot.swift imageFormat has been added to allow the customization of imageFormat and all place where we can use .image stategies have now a new optional parameter imageFormat to override the imageFormat per test if needed.

The new SnapshotTestingPlugin target:

  • SnapshotTestingPlugin protocol that an all plugin should use to register this could change to package level.

The new ImageSerializationPlugin target:

  • ImageSerializationPlugin That represent what is a ImageSerializationPlugin plugin.
  • ImageSerialization all plugin that want to change the way image are serialized inside swift-snapshot-testing should conform to.
  • ImageSerializationFormat that represent the possible format for an ImageSerializationPlugin.

That's it.

@mackoj mackoj mentioned this pull request Oct 6, 2024
4 tasks
@mackoj
Copy link
Contributor Author

mackoj commented Oct 26, 2024

A plugin could also manage the diff algorithm. Instead of pixel comparison, we can use file hashing since generated images typically yield identical results. If the file hashes differ, we can then fall back to pixel-based diffing or even apply more advanced methods, like those below.

Algorithm Speed Complexity Best Use Case
Pixel-by-Pixel Very Fast Simple Exact match, quick comparisons
Mean Squared Error (MSE) Fast Low Rough comparisons
Structural Similarity (SSIM) Moderate Moderate Perceptual similarity
Histogram Comparison Fast-Moderate Moderate Similar content with transformations
Feature-Based (SIFT, SURF) Moderate-Slow High Similar objects with transformations
Perceptual Hashing (pHash) Fast Moderate Near-duplicate detection
Deep Learning Models Slow High Semantic similarity (same object, scene)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants