LangArena: Programming Language Evaluation Suite

Results

What is it?

My experiment. It evaluates 27 programming languages across 50 real-world-inspired tasks using 5 metrics: runtime, runtime score, memory usage, compile time, and expressiveness. The results are aggregated to provide a rough picture of each language's performance. This is a comparative evaluation, not a race to crown the fastest language. 8/50 tests cover ecosystem capabilities Base64, JSON, CSV, Regexp - while the rest are pure algorithms.

The implementations follow a simple, Rosetta Code-like approach: comparable implementations of the same algorithms, with exceptions where necessary. The aim is not to squeeze the maximum performance out of each language, but to write roughly equivalent code across languages. Idiomatic constructs are used where they don't compromise comparability. This reflects typical production code written by teams: readable, maintainable solutions using standard containers, libraries, idioms, and compiler flags.

The suite began with my original implementation in Crystal. The other languages were ported from it with LLM assistance. Each task has a checksum to validate implementation correctness and prevent DCE.

The main table shows results for standard safe release flags. A separate "Hacking" section explores unsafe flags - disabling bounds checks, runtime checks, and other trade-offs that prioritize speed over safety.

Since this is a language-level evaluation, cross-language comparisons are rough and approximate. Comparing different runtimes or compilers for the same language, however, gives a fairer test - OpenJDK/GraalVM, Node/Bun/Deno, Clang/GCC, CRuby/Spinel, and others.

Languages: C, C++, Crystal, Rust, Go, Swift, C#, Java, Kotlin, TypeScript, Zig, D, V, Julia, Nim, F#, Dart, Python, Odin, Scala, C3, Ruby, PHP, Mojo, JavaScript, Gossamer, Goose.

Note on Mojo: this is a young language, and many tests use Python interop instead of native implementations, simply because I can't compile libraries for JSON, Regex. Performance is still very raw.

Participating Compilers/Runtimes:

• C: Clang, GCC, mycc, cproc
• C++: Clang++, G++
• Rust: rustc, rustc/WASM (Node, Wasmtime, Wasmer, WasmEdge)
• Zig: Zig
• Crystal: Crystal
• Mojo: Mojo
• D: DMD, LDC
• V: V/GCC, V/Clang
• Go: Go
• C#: .NET/JIT, .NET/AOT
• F#: .NET/JIT
• Nim: Nim/GCC, Nim/Clang, Nim++/GCC, Nim++/Clang++, Nim/NLVM
• Julia: Julia
• Swift: Swift
• Odin: Odin
• C3: C3
• Java: OpenJDK, OpenJDK/EA, GraalVM
• Kotlin: Kotlin/JVM, Kotlin/GraalVM
• Scala: Scala/JVM, Scala/GraalVM
• Dart: Dart/AOT, Dart/JIT
• TypeScript: Node, Bun, Deno
• Python: PyPy
• Ruby: CRuby/YJIT, TruffleRuby, Spinel
• PHP: PHP/JIT
• JavaScript: Node, Bun, Deno
• Gossamer: Gossamer
• Goose: Goose

Benchmarking Methodology

• Each test runs in a Docker container.
• Each test runs in isolation, with data preparation excluded from the measured runtime.
• All implementations get a separate warmup phase before measurement, which matters most for JIT-based languages.
• Checksums validate results across implementations.
• Dependency fetching is excluded from the cold compile time measurement.
• Incremental compile time is measured after a single file is changed and recompiled.

Sources

Benchmark ideas were taken from:

• The Computer Language Benchmarks Game
• My own collections: benchmarks, jit-benchmarks, crystal-benchmarks-game, crystal-metric
• Crystal code samples

Related

• index-c - Experiment: GCC and Clang performance over 12 years.
• index-cpp - Experiment: G++ and Clang++ performance over 10 years.

Hardware

AMD Ryzen 7 3800X 8-Core, 78GB RAM (x86_64-linux-gnu)

Running

Local run without docker:

cd Lang and run scripts ./test (for fast testing) and ./run (for local measure):

cd rust
./test [BenchName]
./run [BenchName]

Run all

Requires only: docker, docker compose, and ruby. Warning: Docker images take up more than 35 GB of disk space.

sh build-docker.sh
ruby benchmarks.rb

Generate Website

cd docs
ruby gen.rb ../results/2026-02-02-x86_64-linux-gnu.js
open index.html