Skip to content

feat(bundle): build an agent's image from its folder's Dockerfile - #10

Merged
earakely-scale merged 14 commits into
mainfrom
edgararakelyan/bundle-built-images
Oct 6, 2026
Merged

earakely-scale merged 14 commits into
mainfrom
edgararakelyan/bundle-built-images

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

An agent folder with a Dockerfile and no image was already parsed and planned as a built image, but materialize refused to write it. A bundle run now builds it, so a bundle can carry an agent's source instead of naming an image someone registered first:

agents/solver/
  Dockerfile        # built from this folder
  agent.toml        # optional: default_env_vars, [metadata]
  ...

What it does

  • Build and write. materialize copies the folder's files into a temporary build context, runs build_image() over it on this machine for its own platform, and writes the result as the @local docker_image <agent id>__agent_image. The copy is what the ledger hashes (build_context_files): every file of the folder, agent.toml included, less what the OS or Python leaves behind (__pycache__, .DS_Store, ...). Links inside the bundle are copied as their targets. The folder's .dockerignore still applies to the build.
  • Written like any image. The image is pushed to the local registry and saved as a tarball in the local object store, and the same copy is kept as its build context for install/v1. The agent is written over it right after, as for a store image.
  • Ledger tracking.
    • An unchanged folder reuses the image.
    • A changed file rebuilds it and rewrites the agent over it.
    • What the build fetches, such as the base image a FROM tag names, isn't an input.
  • Dry run. agent-env run --dry-run lists the image apart from its agent, as agents/solver (Dockerfile image): v1 (new), and predicts a rebuild and the agent rewritten over it. It builds nothing.
  • Docker. Needed only for an image that will be built: one the ledger reuses needs no docker, in the run or the dry run. Without docker, an image that needs building is refused before anything is written.
  • Errors. A failed build is a bundle problem naming the agent's folder, with the first line and the last 40 lines of docker's output. A failed push, save or local-registry start is a one-line problem too. Neither is a traceback.
  • Run output. materialize(on_build=...) is called before each build. agent-env run prints agents/solver (Dockerfile image): building with docker, which can take minutes and labels the image's write apart from its agent's.
  • Interrupted puts. DockerImageArtifact.put now writes its tarball and build context under an attempt prefix, as file artifacts already do. A put that stopped before writing its document, say on Ctrl-C during a long upload, no longer blocks every later put of that id. Only new objects' keys change; readers follow the URLs in the document.

Limits

  • The image is built for this machine's platform and stays on this machine, so only the local sandbox runs it. Remote providers need it in a store they can reach. Refusing a built image on a remote provider before the run is the planned minimal provider preflight's job.
  • The first push starts the local registry, which pulls registry:2 from Docker Hub.
  • Env folders with a Dockerfile are still refused, since envs can't be written from a bundle yet. Their image inputs will be the same build_context_files.
  • The bundle docs moved off the README to the docs site, so the authoring text for this lands there separately.

Tests

  • materialize:

    • a folder is built and the agent written over its image;
    • a rerun builds nothing and reuses both;
    • a changed file rebuilds and rewrites the agent;
    • an agent.toml edit rebuilds the image and rewrites the agent;
    • the context is the hashed files: OS and Python leftovers stay out (and editing one reuses the image), and a link is copied as its target;
    • a dry run predicts a rebuild and the rewrite of its agent, and builds nothing;
    • a reused image needs no docker, in the run and the dry run;
    • an image to build without docker is refused before any write, in the run and the dry run;
    • a failed build and a failed push are bundle problems naming the agent.
  • run: the progress lines, and the CLI dry run's line for a built image.

  • DockerImageArtifact.put: a put that stops before its document doesn't block the next.

  • ledger: what a built image is made from.

  • Unit tier: 5,750 passed.

  • Integration (slow tier, real docker builds, agents deployed on the local sandbox): tst/integration/cli/run_bundle_built_agents_test.py. One bundle holds four agents, each built from a different shape of Dockerfile, and each replies with what its build put in it:

    agent Dockerfile its reply shows
    plain at the folder's root, no agent.toml the greeting file it copied
    whole COPY ., with a .dockerignore, a link, __pycache__ and .DS_Store only notes.md: the ignored file and the leftovers stayed out, and the link was copied as its target
    subdir docker/Dockerfile, named by agent.toml the greeting agent.toml's env vars set
    staged multi-stage Dockerfile.agent the greeting an earlier stage wrote
    • Each agent's task deploys it, prompts it and checks the reply.
    • A rerun builds nothing.
    • Editing one agent's file rebuilds only that agent; changing its __pycache__ or .DS_Store doesn't.
    • A second test checks that a Dockerfile that fails to build gives one problem with docker's output and writes no agent.
    • Both pass on CI's Linux runner, and locally on macOS.
    • The test closes the process's grant server when it ends. Its agents moved their trajectories through it, with a certificate from the test's state root, which a later test's agents wouldn't trust.

End to end (local sandbox, default local stores)

I ran a bundle whose agents/echo/ folder holds the integration suites' echo A2A agent and its Dockerfile. Its task deploys the agent, prompts it once and checks the reply with response_contains. Command: agent-env run <bundle> --sandbox local.

run image agent smoke task
dry run v1 (new), nothing built or written v1 (new) would run
first built, v1 v1 passed
after editing agent.py and adding __pycache__/ and .DS_Store rebuilt, v2 (files changed: agent.py) v2 (its image v1 → v2) passed, and the reply came from v2
rerun, nothing changed v2, unchanged v2, unchanged passed
dry run after editing only __pycache__/ v2, unchanged v2, unchanged would run

The saved build context of v2 holds Dockerfile, agent.py and agent.toml, and its tarball sits under its attempt prefix (.../2-<id>/...).

🤖 Generated with Claude Code

RetriggerConfidence Score: 5/5

The changes since the previous review appear safe to merge; no new issue was found.

Summary

Bundle runs can now build an agent’s Docker image from the files in its folder and write the agent to use it. The image is tracked for reuse, and interrupted image uploads get their own object keys.

  • Bundle runs build an agent’s image from its folder before writing the agent.
  • Interrupted image puts no longer block a later write.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Agent folder] --> B[Stage build files]
    B --> C[Build image]
    C --> D[Push and save image]
    D --> E[Write image artifact]
    E --> F[Write agent]
Loading

Reviews (6) · Last reviewed commit: "refactor(artifact): name a docker image ..."

earakely-scale and others added 4 commits September 30, 2026 06:42
The six put commands ran nine copies of the same `docker build` (a2a-agent, gateway, mcp-server,
service-db x3, website x2, website-browser). They now call agent_env.utils.docker_build.build_image(),
which library code can import too: stdlib only, no click, no providers.

- build_image(dockerfile, context, tag, *, platform, build_args=None) builds only; each caller still
  puts the image. platform has no default, so a caller chooses; None or "" builds host-native.
- A failed build raises DockerBuildError with docker's whole output (stderr merged into stdout, bytes
  that don't decode replaced). The CLI renders it as one user-facing error and exits 1, replacing the
  five per-site prefixes and service-db's "Aborted!". A missing docker binary is the same one line
  instead of a traceback.
- DEFAULT_BUILD_PLATFORM moves to the new module; docker_build_platform_args goes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…age captures output

- The put test stopped each command at its first build, so service-db's db-web and db-mcp builds and
  website's frontend build were never checked. A test with the builds succeeding now asserts all of them;
  website's backend and frontend get separate folders so a swapped context shows.
- A fake docker on PATH checks that both streams, and a byte that isn't UTF-8, reach DockerBuildError.
- Drop two mock names the lift left unused.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An agent folder with a Dockerfile and no `image` was parsed and planned as a built image, then refused
by materialize. It is now built with `build_image()` on this machine, for its own platform, and written
as the @Local docker_image `<agent id>__agent_image` (pushed to the local registry, saved as a tarball in
the local object store, its build context kept for install/v1), just before the agent written over it.

- The ledger tracks built images: an image is made from every file of its folder, the entry's toml too,
  since a Dockerfile can copy it. An unchanged folder reuses the image; a changed file rebuilds it and
  rewrites the agent. What the build fetches (base image, packages) isn't an input.
- Without docker on PATH, a built image is refused before any write. A failed build is a BundleError
  naming the agent's folder, with the end of docker's output.
- `materialize(on_build=...)` is called before each build; `agent-env run` prints it and labels image
  writes apart from their agent.
- `COPY .` (or `ADD .`) keeps the whole build context in an image's saved context; the COPY-source
  parser dropped `.`, which left install/v1 with only the Dockerfile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested review from a team and polakamtejas as code owners September 30, 2026 17:13
Comment thread src/agent_env/bundle/ledger.py Outdated
Comment thread src/agent_env/bundle/materialize.py Outdated
Comment thread src/agent_env/artifact/artifacts/docker_image.py Outdated
Comment thread src/agent_env/bundle/materialize.py Outdated
Base automatically changed from edgararakelyan/build-image-lift to main September 30, 2026 17:26
earakely-scale and others added 6 commits October 5, 2026 17:59
Re-applies only this branch's own change on top of main; the build_image()
lift it was stacked on landed separately. The README hunks are dropped: the
bundle docs moved to the docs site.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… hashes

The build ran over the agent's folder while the ledger hashed the folder less
what the OS or Python leaves behind (`__pycache__`, `.DS_Store`, ...), and docker
copied links as links. A change there could reuse an image built from other
files. The image is now built from a temporary copy of `build_context_files`
(the folder's files and its toml) and that copy is saved as its build context,
so the hash, the build and the saved context hold the same files. The
folder's `.dockerignore` still applies to the build.

This also drops the change to the shared COPY-source parser, so `a2a-agent put`,
`env mcp-server put` and GitHub builds behave as on main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The docker check refused every built image when docker wasn't on PATH,
including one the ledger would reuse, and it ran in the dry run too. It now
runs after the ledger is opened and refuses only the images it would build,
still before anything is written, in the run and the dry run alike.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A RuntimeError from the image store (the local registry failing to start),
docker push or docker save reached the user as a traceback. It is now a
problem naming the agent's folder, like a failed build.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DockerImageArtifact.put uploaded its tarball and build context to write-once
keys named by version, then wrote the document. A put that stopped in between,
say on Ctrl-C during a long upload, left those keys behind, and every later
put of the id picked the same version and failed on them. The objects now go
under `ArtifactStore.attempt_prefix`, as file artifacts' already do. Only new
objects' keys change; readers follow the URLs in the document.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…it with derive_id

The image id a bundle builds is now spelled by `derive_id`, as every other
derived id is (the same bytes as before). New tests: the dry run lists an image
it would build, predicts a rebuild and the agent written over it, and builds
nothing; a run says before it builds and names the image apart from its agent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner October 6, 2026 02:06
Comment thread src/agent_env/bundle/materialize.py
Comment thread src/agent_env/bundle/materialize.py
earakely-scale and others added 4 commits October 5, 2026 20:32
…ock is held

Another run writing the same ids may be building the image; once it releases
the lock, the ledger can reuse what it built, so a run without docker waits
for it instead of refusing. The check still comes before any write.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e, for real

A slow-tier test runs a bundle of four agents built by docker and deployed on
the local sandbox, each with a Dockerfile of another shape:
- at the folder's root, with no agent.toml;
- `COPY .`, with a .dockerignore, a link and what the OS or Python leaves
  behind;
- in a subfolder named by agent.toml, whose env vars reach the container;
- multi-stage under another name.
Each agent replies with what its build put in it, and its task checks the
reply. A rerun builds nothing, an edited file rebuilds only its own agent, and
a Dockerfile that fails is one problem with docker's output. HOME stays put:
docker's credential helper can hang a build when it moves.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The test's agents move their trajectories through this process's local grant
server, which serves a certificate from the test's state root. A later test
in the same process gave its agent another root's CA, so its upload failed
with transfer_unavailable. The test now closes the server, and the next one
to issue a grant starts it afresh.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The locals in DockerImageArtifact.put and put_from_github held object URLs
from whichever object store is configured (file://, s3://, gs://), but were
still named tar_gz_s3_url and build_context_s3_url. They now match the fields
they fill, tar_gz_object_url and build_context_object_url. put_tar's keyword
arguments and the stored documents' keys keep their names.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 26128d4 into main Oct 6, 2026
13 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/bundle-built-images branch October 6, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant