Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,23 @@
character, and decomposes characters the fonts lack where one has their parts, like HarfBuzz.
- Fallback fonts are only loaded when the text needs them, and glyphs (also decoded and scaled emoji bitmaps) are
kept per font and size across calls.
- `DecodeOptions.maxPixels` and `AnimationDecodeOptions.maxPixels` (default 268,402,689, that's 16383², like sharp):
larger images, and animations whose frames take more pixels together, throw a RangeError before anything that size
is allocated. Tiny files declaring huge images (decompression bombs) could take gigabytes or kill the process: a
250-byte JPEG of 65500x65500, a 115 kB GIF with 5000 frames on a 4000x4000 screen. `maxPixels: Infinity` lifts the
limit. TIFFs also can't take more than 8 bytes of samples per pixel of the limit, or tiles much larger than the
image.

### Performance

- Lossless WebP encoding is about 40% faster and decoding about 15% faster (1920×1080 photo: 735 instead of 1220 ms,
82 instead of 95 ms), with identical output: the predictors run as one loop per mode instead of a switch per pixel,
residual costs come from a table, and the LZ77 search hashes every pixel pair once.

- Decoding PNGs takes less memory (an 8000x8000 RGBA image: 785 instead of 1272 MB at most): the rows are unfiltered in
place, and `Image.decode` keeps the decoded pixels instead of copying them. `Animation.decode` decodes still PNGs like
images (10 times as fast for large ones), and composites APNG frames that replace pixels row by row.

### Fixes

- The outline of `Image.renderText` (`strokeWidth`) showed through translucent text, it now only shows around the
Expand All @@ -56,6 +66,11 @@
of a RangeError, and text larger than 2^28 pixels throws.
- Glyph outlines reaching far outside their bitmap rasterize correctly (the parts outside used to change the slope of
the edges inside).
- Damaged images throw "Image: damaged or truncated image data" (with the original RangeError as its cause) instead
of RangeErrors from reading past their end. JPEGs with more than 4 components are rejected (each took memory the size
of the image).
- `blur` and `sharpen` hung with an infinite radius, they throw a RangeError now; huge finite radii take as long as
small ones (`blur(1e9)` took minutes).

## 3.2.1

Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ deno add jsr:@matmen/imagescript # Deno (npm:imagescript works too)
- WebPs (lossy, lossless and with alpha; bit-exact with libwebp)
- TGAs (color mapped, truecolor and greyscale; uncompressed and run length encoded)
- JPEGs, TIFFs, PNGs and WebPs are rotated as their Exif (or, for PNGs, XMP) orientation says
- images larger than 268 megapixels (`maxPixels`) are rejected before decoding, so small files declaring huge images
can't exhaust the memory
- [Decoding animations](https://imagescript.matmen.dev/classes/Animation.html#decode) (GIF, APNG and WebP) with all
frames, durations and disposal modes
- [Rendering SVGs](https://imagescript.matmen.dev/classes/Image.html#rendersvg) (shapes, paths, gradients, strokes, clip paths,
Expand Down
38 changes: 30 additions & 8 deletions src/animation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@ import { ImageType } from './image-type.ts';
import { optimize, OptimizeOptions, OptimizedFrame } from './util/optimize.ts';
import { Metadata, resolveMetadata, pngText, exifData, gifComments } from './metadata.ts';
import { isOptions } from './util/options.ts';
import { MAX_PIXELS, decoding } from './util/limits.ts';

// loop counts: ImageScript (like GIF) counts repetitions (-1 forever, 0 play once), APNG and WebP count plays (0 forever)
const toPlays = (loopCount: number): number => loopCount < 0 ? 0 : loopCount + 1;
const fromPlays = (plays: number): number => plays === 0 ? -1 : plays - 1;

// frames decoded fully composited, as full canvas frames
function canvasFrames(width: number, height: number, frames: Array<{ pixels: Uint8Array, duration: number }>): Frame[] {
function canvasFrames(width: number, height: number, frames: Array<{ pixels: Uint8Array | Uint8ClampedArray, duration: number }>): Frame[] {
return frames.map(({ pixels, duration }) => {
const frame = new Frame(width, height, duration, 0, 0, Frame.DISPOSAL_BACKGROUND);
// the composited frames are copies already, the frame can keep them
Expand All @@ -33,6 +34,12 @@ export interface AnimationOptions {
export interface AnimationDecodeOptions {
/** Whether to stop decoding after the first frame. Default: `false` */
onlyFirstFrame?: boolean;
/**
* The most pixels to decode, of all frames together (every frame is a full canvas): larger animations throw a
* RangeError before anything that size is allocated, so small files declaring huge animations (decompression
* bombs) can't exhaust the memory. Default: `268402689` (16383², like sharp), `Infinity` for no limit
*/
maxPixels?: number;
}

/** Options of {@link Animation.encodeWebP}, and those all animation encoders have */
Expand Down Expand Up @@ -330,31 +337,46 @@ export class Animation extends Array<Frame> {
*/
static decode(input: BinaryData, onlyExtractFirstFrame?: boolean): Promise<Animation>;
static async decode(input: BinaryData, onlyFirstFrameOrOptions: boolean | AnimationDecodeOptions = false): Promise<Animation> {
const onlyExtractFirstFrame = isOptions(onlyFirstFrameOrOptions) ? onlyFirstFrameOrOptions.onlyFirstFrame ?? false : onlyFirstFrameOrOptions;
return decoding(() => Animation.__decode__(input, onlyFirstFrameOrOptions));
}

/** @internal */
private static async __decode__(input: BinaryData, onlyFirstFrameOrOptions: boolean | AnimationDecodeOptions): Promise<Animation> {
const { onlyFirstFrame: onlyExtractFirstFrame = false, maxPixels = MAX_PIXELS } = isOptions(onlyFirstFrameOrOptions) ? onlyFirstFrameOrOptions : { onlyFirstFrame: onlyFirstFrameOrOptions };
const data = view(input);
const type = ImageType.getType(data);

if (type === 'gif') return Animation.__fromGIF__(data, onlyExtractFirstFrame, Animation);
if (type === 'gif') return Animation.__fromGIF__(data, onlyExtractFirstFrame, Animation, maxPixels);

if (type === 'webp') {
const decoded = webp.decodeAnimation(data, onlyExtractFirstFrame);
const decoded = webp.decodeAnimation(data, onlyExtractFirstFrame, maxPixels);
const frames = canvasFrames(decoded.width, decoded.height, webp.composite(decoded));
return new Animation(frames, fromPlays(decoded.loopCount));
}

// (still PNGs decode like images, keeping their orientation as it is like before)
if (type === 'png' && !png.isAnimated(data)) {
const image = await Image.decode(data, { maxPixels, applyOrientation: false });
return new Animation(canvasFrames(image.width, image.height, [{ pixels: image.bitmap, duration: 0 }]), -1);
}
if (type === 'png') {
const decoded = await png.decodeAnimation(data, onlyExtractFirstFrame);
const decoded = await png.decodeAnimation(data, onlyExtractFirstFrame, maxPixels);
const frames = canvasFrames(decoded.width, decoded.height, png.compositeAnimation(decoded));
return new Animation(frames, decoded.frames.length > 1 ? fromPlays(decoded.plays) : -1);
}

if (type) return new Animation([Frame.from(await Image.decode(data), 0)]);
if (type) return new Animation([Frame.from(await Image.decode(data, { maxPixels }), 0)]);
throw new Error('Unsupported image type');
}

/** @internal */
static __fromGIF__<T extends Animation>(data: Uint8Array, onlyExtractFirstFrame: boolean, Class: new (frames: Frame[], loopCount: number) => T): T {
const decoded = gifCodec.decode(data, onlyExtractFirstFrame);
static __fromGIF__<T extends Animation>(data: Uint8Array, onlyExtractFirstFrame: boolean, Class: new (frames: Frame[], loopCount: number) => T, maxPixels = MAX_PIXELS): Promise<T> {
return decoding(() => Animation.__gifFrames__(data, onlyExtractFirstFrame, Class, maxPixels));
}

/** @internal */
private static __gifFrames__<T extends Animation>(data: Uint8Array, onlyExtractFirstFrame: boolean, Class: new (frames: Frame[], loopCount: number) => T, maxPixels: number): T {
const decoded = gifCodec.decode(data, onlyExtractFirstFrame, maxPixels);
const frames = gifCodec.composite(decoded).map(({ pixels, delay }) => ({ pixels, duration: 10 * delay }));
return new Class(canvasFrames(decoded.width, decoded.height, frames), decoded.loopCount);
}
Expand Down
7 changes: 6 additions & 1 deletion src/codecs/gif.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { quantize } from './quantize.ts';
import { MAX_PIXELS, PixelBudget, checkPixels } from '../util/limits.ts';

// ImageScript disposal modes (any, keep, previous, background) <-> GIF disposal methods (unspecified, none, background, previous)
const DISPOSAL_TO_GIF = [0, 1, 3, 2];
Expand Down Expand Up @@ -206,7 +207,7 @@ function writeSubBlocks(out: ByteWriter, data: Uint8Array): void {
* @param {boolean} [onlyFirstFrame=false]
* @return {{width: number, height: number, loopCount: number, frames: GIFFrame[]}}
*/
export function decode(data: Uint8Array, onlyFirstFrame = false): DecodedGIF {
export function decode(data: Uint8Array, onlyFirstFrame = false, maxPixels = MAX_PIXELS): DecodedGIF {
const reader = new ByteReader(data);

const signature = reader.string(6);
Expand All @@ -222,6 +223,8 @@ export function decode(data: Uint8Array, onlyFirstFrame = false): DecodedGIF {
const frames: DecodedGIFFrame[] = [];
let loopCount = 0;
let control: GraphicControl | null = null;
// the frames' color indices, and later a canvas per frame
const budget = new PixelBudget(maxPixels);

try {
parse();
Expand All @@ -230,6 +233,7 @@ export function decode(data: Uint8Array, onlyFirstFrame = false): DecodedGIF {
if (err !== UNEXPECTED_END) throw err;
}

checkPixels(width, height, maxPixels, Math.max(1, frames.length));
return { width, height, loopCount, frames };

function parse(): void {
Expand Down Expand Up @@ -287,6 +291,7 @@ export function decode(data: Uint8Array, onlyFirstFrame = false): DecodedGIF {

const minCodeSize = reader.u8();
const compressed = readSubBlocks(reader);
budget.take(fw, fh);
const indices = lzwDecode(compressed, minCodeSize, fw * fh);

const { dispose = 0, delay = 0, transparentIndex = -1 } = control || {};
Expand Down
8 changes: 7 additions & 1 deletion src/codecs/jpeg.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
* (Copyright 2014 Mozilla Foundation, Apache License 2.0). See THIRD_PARTY_NOTICES.md
*/

import { MAX_PIXELS, checkPixels } from '../util/limits.ts';
import { orientation as exifOrientation } from './exif.ts';
import { concat } from '../util/mem.ts';

Expand Down Expand Up @@ -60,6 +61,8 @@ interface Frame {
export interface DecodeOptions {
/** overrides the color transform detected from JFIF/Adobe markers (used by TIFF) */
transform?: 'ycc' | 'rgb' | 'cmyk' | null;
/** the most pixels to decode */
maxPixels?: number;
}

function clamp8(v: number): number {
Expand All @@ -79,7 +82,7 @@ function clamp8(v: number): number {
* @param {{transform?: 'ycc'|'rgb'|'cmyk'}} [options] overrides the color transform detected from JFIF/Adobe markers (used by TIFF)
* @return {{width: number, height: number, pixels: Uint8Array}}
*/
export function decode(data: Uint8Array, { transform }: DecodeOptions = {}): { width: number, height: number, pixels: Uint8Array, orientation: number } {
export function decode(data: Uint8Array, { transform, maxPixels = MAX_PIXELS }: DecodeOptions = {}): { width: number, height: number, pixels: Uint8Array, orientation: number } {
if (data[0] !== 0xff || data[1] !== 0xd8) throw new Error('JPEG: invalid signature');

const quantizationTables: Int32Array[] = [];
Expand Down Expand Up @@ -160,6 +163,7 @@ export function decode(data: Uint8Array, { transform }: DecodeOptions = {}): { w
case 0xc0: case 0xc1: case 0xc2: { // SOF baseline, extended, progressive
if (frame) throw new Error('JPEG: multiple frames are not supported');
if (data[start] !== 8) throw new Error(`JPEG: ${data[start]}-bit precision is not supported`);
checkPixels(u16(start + 3), u16(start + 1), maxPixels);
frame = createFrame(marker === 0xc2, u16(start + 1), u16(start + 3), data.subarray(start + 5, end));
break;
}
Expand Down Expand Up @@ -235,6 +239,8 @@ function createFrame(progressive: boolean, height: number, width: number, compon
}

if (!width || !sampling.length) throw new Error('JPEG: invalid dimensions');
// each component takes coefficients for the whole image (grey, YCbCr/RGB and CMYK have up to 4)
if (sampling.length > 4) throw new Error(`JPEG: ${sampling.length} components are not supported`);

const hmax = Math.max(...sampling.map(c => c.h));
const vmax = Math.max(...sampling.map(c => c.v));
Expand Down
33 changes: 25 additions & 8 deletions src/codecs/png.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { crc32 } from './crc.ts';
import { MAX_PIXELS, checkPixels } from '../util/limits.ts';
import { concat } from '../util/mem.ts';
import { encode as utf8, decode as utf8Decode } from '../util/utf8.ts';
import { orientation as exifOrientation, xmpOrientation } from './exif.ts';
Expand Down Expand Up @@ -559,7 +560,7 @@ async function decodeImageData(p: Parsed, chunks: Uint8Array[], width: number, h

const rowBytes = Math.ceil(pw * bitsPerPixel / 8);
const size = ph * rowBytes;
const pixels = unfilter(data, dataOffset, ph, rowBytes, bpp);
const pixels = unfilter(data, dataOffset, ph, rowBytes, bpp, !interlace);
dataOffset += size + ph;

toRGBA(pixels, rowBytes, pw, ph, rgba, { row: y => ((ys + y * dy) * width + xs) * 4, step: dx * 4 });
Expand All @@ -574,9 +575,10 @@ async function decodeImageData(p: Parsed, chunks: Uint8Array[], width: number, h
* The orientation comes from the eXIf chunk, or, if there is none, from the Exif text ImageMagick wrote before eXIf
* existed ("Raw profile type exif") or from the XMP packet
*/
export async function decode(array: Uint8Array): Promise<Decoded> {
export async function decode(array: Uint8Array, maxPixels = MAX_PIXELS): Promise<Decoded> {
const p = parse(array);
const { width, height } = p.header;
checkPixels(width, height, maxPixels);
const pixels = await decodeImageData(p, p.idat, width, height);
return { width, height, pixels, orientation: p.exif ? exifOrientation(p.exif) : await textOrientation(p.text) };
}
Expand Down Expand Up @@ -610,9 +612,11 @@ export function isAnimated(array: Uint8Array): boolean {
* a single frame
* @return plays: 0 = forever
*/
export async function decodeAnimation(array: Uint8Array, onlyFirstFrame = false): Promise<{ width: number, height: number, plays: number, frames: APNGFrame[] }> {
export async function decodeAnimation(array: Uint8Array, onlyFirstFrame = false, maxPixels = MAX_PIXELS): Promise<{ width: number, height: number, plays: number, frames: APNGFrame[] }> {
const p = parse(array);
const { width, height } = p.header;
// every frame is composited onto a canvas of its own
checkPixels(width, height, maxPixels, !p.animation || p.animationError || onlyFirstFrame ? 1 : Math.max(1, p.frames.length));

// broken animations show the default image, like browsers do
if (!p.animation || p.animationError) {
Expand Down Expand Up @@ -647,18 +651,29 @@ export function compositeAnimation({ width, height, frames }: { width: number, h
for (const f of frames) {
const previous = f.dispose === 2 ? canvas.slice() : null;

for (let y = 0; y < f.height; y++)
for (let y = 0; y < f.height; y++) {
// replacing: whole rows
if (f.blend === 0) {
canvas.set(f.pixels.subarray(y * f.width * 4, (y + 1) * f.width * 4), ((f.y + y) * width + f.x) * 4);
continue;
}
for (let x = 0; x < f.width; x++) {
const s = (y * f.width + x) * 4, d = ((f.y + y) * width + f.x + x) * 4;
const sa = f.pixels[s + 3], da = canvas[d + 3];
if (f.blend === 0 || sa === 255 || da === 0) {
if (f.blend === 0 || sa !== 0) canvas.set(f.pixels.subarray(s, s + 4), d);
if (sa === 255 || da === 0) {
if (sa !== 0) {
canvas[d] = f.pixels[s];
canvas[d + 1] = f.pixels[s + 1];
canvas[d + 2] = f.pixels[s + 2];
canvas[d + 3] = sa;
}
} else if (sa !== 0) {
const u = sa * 255, v = (255 - sa) * da, al = u + v;
for (let c = 0; c < 3; c++) canvas[d + c] = (f.pixels[s + c] * u + canvas[d + c] * v) / al | 0;
canvas[d + 3] = al / 255 | 0;
}
}
}

output.push({ pixels: canvas.slice(), duration: f.duration });

Expand Down Expand Up @@ -718,8 +733,10 @@ function hexProfile(text: string): Uint8Array {
return out;
}

function unfilter(data: Uint8Array, offset: number, height: number, rowBytes: number, bpp: number): Uint8Array {
const out = new Uint8Array(height * rowBytes);
// undoes the row filters; in place writes the rows over the data itself (every row ends up before where its data was,
// and the data is read ahead of what's written)
function unfilter(data: Uint8Array, offset: number, height: number, rowBytes: number, bpp: number, inPlace = false): Uint8Array {
const out = inPlace ? data.subarray(offset, offset + height * rowBytes) : new Uint8Array(height * rowBytes);
const first = Math.min(bpp, rowBytes);

// the first row and the first pixel of every row have zero neighbours, those are handled by separate loops
Expand Down
4 changes: 3 additions & 1 deletion src/codecs/tga.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { encode as utf8 } from '../util/utf8.ts';
import { MAX_PIXELS, checkPixels } from '../util/limits.ts';

// Truevision TGA: color mapped, truecolor and greyscale images, uncompressed or run length encoded

Expand Down Expand Up @@ -91,10 +92,11 @@ function readColor(d: Uint8Array, o: number, bits: number, alphaBits: number, ou
/**
* Decodes a TGA into RGBA
*/
export function decode(d: Uint8Array): { width: number, height: number, pixels: Uint8Array } {
export function decode(d: Uint8Array, maxPixels = MAX_PIXELS): { width: number, height: number, pixels: Uint8Array } {
const h = readHeader(d);
if (!h || !isTGA(d)) throw new Error('TGA: invalid header');
const { width, height, depth } = h;
checkPixels(width, height, maxPixels);
const base = h.imageType & ~8, rle = (h.imageType & 8) !== 0;
const bytesPerPixel = Math.ceil(depth / 8);

Expand Down
8 changes: 7 additions & 1 deletion src/codecs/tiff.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { decompressAsync } from './zlib.ts';
import * as jpeg from './jpeg.ts';
import { MAX_PIXELS, checkPixels, checkBytes } from '../util/limits.ts';

const TYPE_SIZES: Record<number, number> = { 1: 1, 2: 1, 3: 2, 4: 4, 5: 8, 6: 1, 7: 1, 8: 2, 9: 4, 10: 8, 11: 4, 12: 8, 13: 4, 16: 8, 17: 8, 18: 8 };

Expand Down Expand Up @@ -46,7 +47,7 @@ interface SampleReader {
(rowOffset: number, n: number): { raw: number, value: number };
}

export async function decode(data: Uint8Array): Promise<{ width: number, height: number, pixels: Uint8Array, orientation: number }> {
export async function decode(data: Uint8Array, maxPixels = MAX_PIXELS): Promise<{ width: number, height: number, pixels: Uint8Array, orientation: number }> {
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);

let little: boolean;
Expand Down Expand Up @@ -178,6 +179,11 @@ export async function decode(data: Uint8Array): Promise<{ width: number, height:
return data.subarray(offset, Math.min(data.length, offset + count));
};

checkPixels(width, height, maxPixels);
// tiles may reach past the image, but not by much (a segment's buffer is that large), and samples can take more
// memory than the pixels
checkPixels(segmentWidth, segmentHeight, Math.max(4 * width * height, 1 << 20));
checkBytes(Math.ceil(width * samplesPerPixel * bps / 8) * height, maxPixels);
const pixels = new Uint8Array(width * height * 4);

// JPEG compressed segments decode straight to RGBA
Expand Down
Loading
Loading