Skip to content

feat: Add support to warp (deform) SpriteComponent rendering via a grid - #4105

Open
adario wants to merge 22 commits into
flame-engine:mainfrom
adario:feat/sprite-warp-grid
Open

adario wants to merge 22 commits into
flame-engine:mainfrom
adario:feat/sprite-warp-grid

Conversation

@adario

@adario adario commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds the ability to warp (deform) the rendering of a SpriteComponent via a grid of normalised
source (UV) and destination positions, similarly to SpriteKit's SKWarpGeometryGrid.

  • New immutable WarpGrid (exported from components.dart): columns x rows cells, with
    normalised source and destination positions for each of the (columns + 1) * (rows + 1)
    vertices, stored in row-major order from the top-left corner (y pointing down, following
    Flame's conventions rather than SpriteKit's). Source positions are UVs relative to the
    sprite's source rectangle (not to the whole image), destination positions are fractions of the
    component's size (values outside 0..1 are valid). The grid never stores pixels: positions are
    converted to texture and component coordinates only when rendering, so the same grid keeps
    working when the component's size or sprite change.
  • New WarpInterpolation enum:
    • bilinear (default) interpolates each cell independently.
    • catmullRom interpolates across cells with Catmull-Rom splines, passing through every
      vertex, for smoother warps.
  • New HasWarpGrid mixin on SpriteComponent, adding the warpGrid and warpInterpolation
    properties It works with any SpriteComponent subclass, and with no grid the sprite is rendered
    as usual (drawImageRect).
  • SpriteComponent.render now delegates to a new protected renderSprite method,
    which the mixin overrides (render is @mustCallSuper). The default implementation renders
    the sprite exactly as before.
  • New WarpEffect (exported from effects.dart) to animate grids, similarly to SpriteKit's
    SKAction.warp(to:duration:): WarpEffect.by moves destination (and optionally source)
    positions by per-vertex offsets, WarpEffect.to moves both source and destination positions
    to those of another grid with the same dimensions (starting from an identity grid if the target
    has none). Like the other built-in effects it applies incremental changes, so several warp
    effects, and drags, can act on the same grid. It targets the new WarpGridProvider interface,
    which HasWarpGrid implements.

Each cell is subdivided into 4x4 sub-cells (a fixed subdivision level of 2, like SpriteKit's
default) and the sprite is drawn as a textured mesh with Canvas.drawVertices and an
ImageShader. The mesh, vertices and shader are cached per component and only rebuilt when the
grid, interpolation, sprite region, size or image change, so components sharing a Sprite can
each have their own warp. Warping only affects rendering: size, hit testing and collisions are
unchanged.

Workarounds for engine issues found while testing on macOS, iOS, Android (emulator) and web
(CanvasKit and skwasm), documented in the code:

  • Impeller ignores the paint's color filter in drawVertices: when a color filter is set (e.g.
    tint, hue, ColorEffect) it is applied through a saveLayer, on every backend for
    consistency.
  • Impeller crashes with some drawVertices blend modes (e.g. BlendMode.dst): the blend mode
    argument (which only combines vertex colors, not used here) is always srcOver, and the
    paint's blend mode is applied as usual.
  • On CanvasKit, ImageShader.dispose() also disposes its image: the shader is never disposed,
    only dereferenced (on removal, when the sprite changes, or when the grid is removed), so that
    the shared sprite image is left alone. Dropping it when the sprite changes also avoids keeping
    an image alive after it is released from the Images cache.
  • Texture coordinates are offset by -1/1024 texel, so that pixels sampling exactly between two
    texels (non-integer scales with nearest filtering) round the same way on every triangle, like
    drawImageRect, instead of showing 1-texel steps along the triangles' edges.

Tests cover the grid, the mesh builder (both interpolations), the component (identity grids
render like drawImageRect, including spritesheet regions, bleed, opacity, color filters and
non-integer scales; caching; sharing sprites; image eviction), the effect (relative, absolute,
reversed, combined, sequences, errors) and two golden files.

The new Sprite Warp example in the Sprites section shows two SpriteComponents with the
HasWarpGrid mixin that share the same sprite and the same WarpGrid: the left one uses
WarpInterpolation.bilinear, the right one WarpInterpolation.catmullRom. The left sprite has
a draggable handle on each vertex, connected by grid lines: dragging a handle moves the vertex's
destination position, and the right sprite mirrors the change.

Knobs:

  • Grid Size: the number of cells per side (3 to 8, 4 by default); changing it starts over with
    an undistorted grid.
  • Image: switches between a few sprites from the examples' assets, keeping the current warp.
  • Animate: waves the grid back and forth with an infinite WarpEffect.by; since the effect is
    incremental, the handles can still be dragged while it runs.
  • Reset: a button that resets the grid to the identity, like a double tap on the game; both are
    disabled while the animation runs. Since Widgetbook has no button knob, a small custom
    ButtonKnob is added to the examples' commons.

Note: the golden files were generated on macOS and may need to be regenerated on Linux.

Checklist

  • I have followed the Contributor Guide when preparing my PR.
  • I have updated/added tests for ALL new/updated/fixed functionality.
  • I have updated/added relevant documentation in docs and added dartdoc comments with ///.
  • I have updated/added relevant examples in examples or docs.

Breaking Change?

  • Yes, this PR is a breaking change.
  • No, this PR is not a breaking change.

Related Issues

Closes #4104

adario added 14 commits October 6, 2026 00:46
  Add an immutable WarpGrid (columns x rows cells, normalized source and
  destination positions, row-major from the top-left) and the
  WarpInterpolation enum, as the first building block for warping
  SpriteComponent rendering in the style of SpriteKit's SKWarpGeometryGrid.
  Build the triangle mesh used to render a warped sprite: each grid cell
  is subdivided into 4x4 sub-cells, with positions and texture coordinates
  interpolated either bilinearly (matching SpriteKit) or with Catmull-Rom
  splines, through a single tensor-product path over a linearly
  extrapolated grid border.
  Add warpGrid and warpInterpolation to SpriteComponent. When a grid is
  set, the sprite is drawn as a cached textured mesh through drawVertices,
  working around Impeller ignoring the paint's color filter (applied via a
  layer) and crashing on some vertices blend modes (always srcOver). The
  image shader is never disposed, since on CanvasKit that also disposes
  the sprite's shared image.
  Add golden tests for bilinear and Catmull-Rom warping and for paint
  effects on warped sprites, and tests for the interaction with image
  eviction. Drop the warp renderer when the sprite changes, so that its
  image shader does not keep the previous image alive after it is
  released.
  Offset the texture coordinates of warped sprites by -1/1024 texel, so
  that pixels sampling the image exactly between two texels (e.g. at
  non-integer scales with nearest filtering) round the same way on every
  triangle of the mesh, like drawImageRect does, instead of showing
  1-texel steps along the triangles' edges.
  Document WarpGrid and WarpInterpolation in the SpriteComponent docs, and
  add a "Sprite Warp" example comparing bilinear and Catmull-Rom warping,
  with draggable grid vertices.
… example

  Draw the grid vertices as stroked circles connected by line segments, and
  let the grid size be chosen between 3x3 and 8x8 cells with a knob.
  Move warpGrid and warpInterpolation from SpriteComponent into a new
  HasWarpGrid mixin on SpriteComponent, which overrides a new protected
  SpriteComponent.renderSprite method, so that warping composes with any
  SpriteComponent subclass. Update tests, example and docs accordingly.
  Let the Sprite Warp example warp one of several images (flame, zap,
  pizza, player, enemy), chosen with a dropdown knob. Changing the image
  keeps the current warp grid, while changing the grid size still resets
  it.
  Add a `Reset` button to the knobs panel of the Sprite Warp example, which
  resets the warp grid to the identity without recreating the game. Widgetbook
  has no button knob, so a small custom `ButtonKnob` (with a
  `context.knobs.button()` extension) is added to the examples' commons: its
  value is the number of presses, and the story resets the grid whenever the
  count changes. While the `Animate` knob is on, both the button and the
  double-tap reset are disabled.
Link the Sprite Warp example from the Warping section and the Warp Effects
page, describing its `Animate` knob and its `Reset` knob button, which resets
the grid to the identity and, like double-tap reset, is disabled while the
animation runs.
adario added 7 commits October 7, 2026 16:11
The Warping section of the sprite components docs only mentioned the
latest example changes. It now describes the whole example: two
HasWarpGrid sprites sharing a grid with bilinear and Catmull-Rom
interpolation, the draggable vertex handles, and the Grid Size, Image,
Animate and Reset knobs. The warp effects docs link to it.
When the target's grid already matched the destination grid, WarpEffect.to
still assigned a new grid every frame, rebuilding the warped sprite's mesh
for no visible change. The effect now leaves the grid untouched when it has
no offsets to apply.
WarpEffect assumed that the target's grid stayed set while the effect
ran, and failed with a null check error when it was set to null. It now
throws a StateError explaining that the grid was removed.
The limit on the number of cells of a WarpGrid, which keeps the rendered
mesh within 16-bit indices, was only checked by an assert, so release
builds silently rendered a garbled mesh. The default and identity
constructors are now factories that throw an ArgumentError before
allocating the grid, and WarpGrid.raw is the primary constructor.
The number of positions given to WarpGrid, or to its replacing methods,
was only checked by an assert, so release builds silently ignored extra
positions or failed with a RangeError when some were missing. It now
throws an ArgumentError.
WarpGrid only asserted that its columns and rows were positive, so
release builds created grids with NaN positions, and negative dimensions
could pass the mesh size check. It now throws an ArgumentError.
@adario
adario marked this pull request as ready for review October 7, 2026 15:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support to warp (deform) sprite rendering via an arbitrary grid

1 participant