feat: Add Sprite.contour and ImageExtension.contour to trace image outlines by adario · Pull Request #4097 · flame-engine/flame

adario · 2026-10-05T13:18:54Z

Description

This PR adds a few extensions to trace the non-transparent outline of an Image, in order to extract a Path contour to be used as e.g. a PolygonHitbox or a Polygon in flame_forge2d: its purpose is allowing the use of an image for collision detection and/or physics.

Support for this feature is organised thusly:

the contour extension on Image returns a Path with the non-transparent contours of the image;
the contours are traced by the new src/geometry/alpha_contours module, which uses the marching squares algorithm;
the contours are split in their convex pieces: the approach followed here is the Hertel/Mehlhorn algorithm;
the convenience contour extension on Sprite returns a Path with the sprite's image outline.

The SpriteBodyExample now uses the standard flame.png image to create its Sprite and BodyComponent.

The new CollidableSpritesExample (somewhat similar to CollidableAnimationExample) creates 4 sprites and related hitboxes from the flame, player, enemy and zap standard asset images: the sprites are kept rotating to show that collisions may occur on arbitrary edges.

Full disclosure: CollidableSpriteComponent.onCollisionStart calls a method thats checks whether to flip the velocity, unfortunately involving a dot product:

  @override
  void onCollisionStart(
    List<Vector2> intersectionPoints,
    PositionComponent other,
  ) {
    super.onCollisionStart(intersectionPoints, other);
    // Only a sprite that moves towards what it hit turns back, so that two
    // collisions that start at the same time do not cancel each other out,
    // and a sprite that is caught up by a faster one keeps going.
    if (velocity.dot(_towards(other, intersectionPoints)) > 0) {
      velocity.negate();
    }
  }

  /// The direction of the [other] component that this one hit at the
  /// [intersectionPoints]: that of the walls that the points are on, or of
  /// the center of the other sprite.
  Vector2 _towards(PositionComponent other, List<Vector2> intersectionPoints) {
    if (other is! ScreenHitbox) {
      return other.absoluteCenter - absoluteCenter;
    }
    final towards = Vector2.zero();
    for (final point in intersectionPoints) {
      if (point.x <= 1) {
        towards.x = -1;
      } else if (point.x >= other.size.x - 1) {
        towards.x = 1;
      }
      if (point.y <= 1) {
        towards.y = -1;
      } else if (point.y >= other.size.y - 1) {
        towards.y = 1;
      }
    }
    return towards;
  }

If the check is too "advanced" for such an example, we'll probably have to scrap the example, since without the check Bad Things Would Happen™.

The image_contour_test and convex_pieces_test contain tests for the new functionality; there's also a new image_contour benchmark.

Checklist

I have followed the Contributor Guide when preparing my PR.
I have updated/added tests for ALL new/updated/fixed functionality.
I have updated/added relevant documentation in docs and added dartdoc comments with ///.
I have updated/added relevant examples in examples or docs.

Breaking Change?

Yes, this PR is a breaking change.
No, this PR is not a breaking change.

Related Issues

Closes #4102

…tlines Trace the outlines of the non-transparent pixels of an image as the closed contours of a Path, to build hitboxes that follow the outline of a sprite, like a PathHitbox or a PolygonHitbox.fromPath. The outlines are found with marching squares on the alpha of the pixel centers, interpolated at the alphaThreshold, so that they follow the anti-aliased edges more closely than a pixel. Only the outer outlines are kept, and each separate part of the image gives a separate contour. - ImageExtension.contour traces a region of an image, scaled to a size. - ImageExtension.contourFromPixels does the same from raw RGBA pixels. - Sprite.contour traces the region of the sprite. The SpriteBodyExample of flame_forge2d now drops flames whose bodies collide as the convex pieces of the outline of the sprite, which the new Show pieces knob draws.

convexPieces splits a simple polygon, convex or concave, into convex pieces of at most maxVertices vertices, like the shapes that physics engines such as Box2D accept, by ear clipping it and merging the triangles back as long as they stay convex (the Hertel-Mehlhorn algorithm). The minDistance and minWidth parameters weld close vertices and leave out the pieces that are too narrow for the tolerances of an engine. It is now exported by package:flame/geometry.dart, and documented in the "Convex pieces" section of the shape components docs. The SpriteBodyExample of flame_forge2d uses it from there.

When the alpha of a pixel is exactly the alphaThreshold, the outline passes through the center of the pixel, where its edges are crossed at the same point. The repeated points looked collinear with their neighbors, so both were removed along with the corner, and a contour could be dropped altogether: an opaque square traced with an alphaThreshold of 1 gave no contour. The repeated points are now merged first. - contourFromPixels asserts that the region is within the image, instead of clamping it, which moved and stretched the contours. - ImageExtension.contour no longer imports the collisions library only for its documentation. - Document that the regions are rounded to whole pixels, that the pixels of the whole image are read back, and how to trace the sprites of a sprite sheet reading them once. - Document the cost of convexPieces, and complete the snippet of its docs.

Ear clipping can get stuck on a polygon that touches itself at a vertex, like the outline of two shapes that meet at a corner, and what was left of the polygon was then silently left out. Such outlines come, for example, from Sprite.contour when the alpha of some pixels is exactly the alphaThreshold. The polygon is now split where two of its vertices that are not consecutive are welded, and each of the resulting loops is split into pieces on its own. A loop that goes the other way is a hole that touches the outline, which is filled by the other loops. The docs now say so, and that the parts of a polygon that crosses itself may be left out.

The tracing of contourFromPixels scanned every cell of the image through the alpha of its four corners, which were copied into a list of doubles, and linked the outlines in a table with two entries for each pixel. Each pixel is now checked against the alphaThreshold once, into a mask of bytes, and the case of a cell is made of the bits of its corners, sharing two of them with the previous cell. The alpha is only read along the outlines, and the edges of the outlines are kept in a map. The contours are the same, while tracing is 2.5 to 3.5 times as fast, and takes about one byte for each pixel instead of sixteen. - Add image_contour_benchmark.dart, a standalone suite for tracing, convexPieces, and the whole way from the pixels to the convex pieces. - Document that the cost of convexPieces grows about with the square of the number of vertices, as measured, not with the cube.

Add a collision detection example, after CollidableAnimationExample, with four sprites of existing example images: flame.png, layers/player.png, zap.png and layers/enemy.png. They move along the same routes as the birds of the animation example while spinning around their centers, and turn back when they collide. The hitbox of each sprite is a PathHitbox made from Sprite.contour, so it follows the outline of the sprite rather than a hand-written polygon. A sprite only turns back when it moves towards what it hit, the walls that the intersection points are on or the center of the other sprite, so that two collisions that start at the same time do not cancel each other out, and a sprite that is caught up by a faster one keeps going instead of being pushed through it.

…hin the region Address the review of the image contours: - convexPieces returns copies of the vertices, since a PolygonHitbox changes its vertices in place, which corrupted the neighboring pieces and the source polygon. - The contours stay within the region at every alpha threshold, as the samples of the transparent border sit half a pixel outside of it. - The last vertices of a loop that are welded to its first one are left out explicitly in _splitAtTouches. - A single signedArea helper replaces the shoelace copies in convexPieces, PolygonComponent.isClockwise and the alpha contours, and the orientation is described as clockwise on the screen everywhere. - PathComponent.polygonsOf samples the polygons of a path without making a component, which the forge2d example, the docs and the benchmark use. - SpriteBodyWorld initializes its field in place of a constructor body.

spydon · 2026-10-05T19:58:34Z

+  /// three vertices, which go counterclockwise in the screen coordinate
+  /// system. See the constructor for the [sampling], the [tolerance] and the
+  /// [filter].
+  static List<List<Vector2>> polygonsOf(


Hmm... This is a bit weird, but I don't have time to come up with a better way now.
I'll add this comment as a note to look closer at this later.

Ok, but... I've not understood whether the method itself is weird, or the comment...

Copilot

Copilot review overview

🟡 Changes recommended

Scale-dependent contour simplification and potentially concave decomposition results violate the new APIs’ contracts.

Review effort: Balanced
Findings: 1 · 1

Open (2)

Prevent convex piece merging from creating concave polygons · New
Use scale-invariant corner comparison for small contours · New

What changed in this PR

Adds image-outline tracing and convex decomposition APIs for sprite-based collision detection and physics.

Changes:

Adds contour tracing for images and sprites.
Adds convex polygon decomposition.
Adds tests, benchmarks, documentation, and examples.

File	Description
`.github/.cspell/people_usernames.txt`	Adds algorithm author names.
`doc/flame/collision_detection.md`	Documents contour-based hitboxes.
`doc/flame/components/shape_components.md`	Documents convex decomposition.
`examples/lib/stories/bridge_libraries/flame_forge2d/flame_forge2d.dart`	Adds the piece-visibility control.
`examples/lib/stories/bridge_libraries/flame_forge2d/sprite_body_example.dart`	Demonstrates contour-based physics bodies.
`examples/lib/stories/collision_detection/collidable_sprites_example.dart`	Adds a contour-hitbox example.
`examples/lib/stories/collision_detection/collision_detection.dart`	Registers the new example.
`packages/flame/benchmark/README.md`	Documents the benchmark suite.
`packages/flame/benchmark/image_contour_benchmark.dart`	Benchmarks tracing and decomposition.
`packages/flame/lib/geometry.dart`	Exports convex decomposition.
`packages/flame/lib/src/extensions/image.dart`	Adds image contour APIs.
`packages/flame/lib/src/geometry/alpha_contours.dart`	Implements marching-squares tracing.
`packages/flame/lib/src/geometry/convex_pieces.dart`	Implements convex decomposition.
`packages/flame/lib/src/geometry/path_component.dart`	Exposes path polygon extraction.
`packages/flame/lib/src/geometry/polygon_component.dart`	Reuses signed-area calculation.
`packages/flame/lib/src/geometry/signed_area.dart`	Adds signed-area utility.
`packages/flame/lib/src/sprite.dart`	Adds `Sprite.contour`.
`packages/flame/test/extensions/image_contour_test.dart`	Tests contour APIs and hitboxes.
`packages/flame/test/geometry/convex_pieces_test.dart`	Tests polygon decomposition.
`packages/flame/test/geometry/path_component_test.dart`	Tests path-coordinate extraction.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

convexPieces accepted merged pieces with reflex corners shallower than half of minWidth, assuming the physics engine would straighten them, so it could return concave pieces whenever minWidth was positive. Merge the pieces only while they stay strictly convex, as documented, and test it with a minWidth.

traceAlphaContours dropped the points whose cross product was below an absolute threshold after scaling the contour to the requested size, so real corners vanished at small sizes, down to an empty path, while collinear points were kept at large ones. Compare the angle between the edges instead, which doesn't depend on the scale, and test the outline at tiny, large and non-uniform sizes.

The pieces were drawn by the body before its SpriteComponent child, which covered them entirely, and with an opaque white fill that hid where one piece ends and the next begins. Draw the sprite in the body's render before its shapes, and outline the pieces with the GlowingBody style.

Taps now add 8 m flames and 6 m pizzas in turn, both with bodies made of the convex pieces of their traced outlines, so the example shows that Sprite.contour works for different images. FlameBody becomes ContourBody, since it now holds any sprite.

…onent.

adario added 9 commits October 4, 2026 22:38

chore: Add Hertel/Mehlhorn in the cspell usernames.

62df8aa

Merge branch 'main' into feat/image-contours

7f70d9d

Merge branch 'main' into feat/image-contours

acff355

adario marked this pull request as ready for review October 5, 2026 16:51

spydon reviewed Oct 5, 2026

View reviewed changes

Merge branch 'main' into feat/image-contours

07d4065

spydon requested a balanced review from Copilot October 6, 2026 12:57

Copilot started reviewing on behalf of spydon October 6, 2026 12:58 View session

Copilot AI reviewed Oct 6, 2026

View reviewed changes

Comment thread packages/flame/lib/src/geometry/convex_pieces.dart Outdated

Comment thread packages/flame/lib/src/geometry/alpha_contours.dart

adario added 8 commits October 6, 2026 15:49

Merge branch 'main' into feat/image-contours

a2e81e4

feat: Add FPS display in SpriteBodyExample and CollidableSpritesExample.

dcebd80

Merge branch 'main' into feat/image-contours

b6c9744

Merge branch 'main' into feat/image-contours

74d938f

adario mentioned this pull request Oct 6, 2026

Add support to trace image outlines as contours for collisions/physics #4102

Open

1 task done

adario added 3 commits October 6, 2026 22:30

refactor: Use proper named position parameter in CollidableSpriteComp…

14cc338

…onent.

feat: Add initial spin and zap sprite in SpriteBodyExample.

82d3895

Merge branch 'main' into feat/image-contours

dfca0eb

Navigation Menu

Repository navigation

feat: Add Sprite.contour and ImageExtension.contour to trace image outlines - #4097

feat: Add Sprite.contour and ImageExtension.contour to trace image outlines#4097
adario wants to merge 22 commits into
flame-engine:mainfrom
adario:feat/image-contours

adario commented Oct 5, 2026 •

edited

Loading

Uh oh!

spydon Oct 5, 2026

Uh oh!

adario Oct 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

adario commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Breaking Change?

Related Issues

Uh oh!

spydon Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

adario Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

adario commented Oct 5, 2026 •

edited

Loading