llcppg Design

1. Overview

llcppg converts C/C++ headers into Go bindings that can be compiled by LLGo (and used from XGo). It takes an AST of the C/C++ declarations and produces:

Go source code for a binding package.
C/C++ wrapper code for symbols that have no linkable entry point (inline functions, template instantiations).
A public-symbol table (llcppg.pub) that lets other llcppg runs refer to this package's types by their C/C++ names.

The heart of the tool is the cl package. Everything upstream of it (parsing, configuration) produces its input; everything downstream (formatting, writing files, generating wrappers) consumes its output. This document describes the contract of cl, the three-phase pipeline it uses internally, and the reasoning behind that pipeline.

Goals

Convert C and C++ declarations (types, functions, variables, macros, enums, templates) into idiomatic Go declarations.
Handle mutual and cyclic references between types without depending on declaration order in the headers.
Handle C++ overloading (functions, template functions, template classes) deterministically.
Make cross-library references possible: a library that depends on another converted library must be able to name that library's Go types.

Non-goals

Parsing C/C++ source. cl consumes an already-built AST.
Emitting source text directly. cl builds a typed Go package through gogen, which is responsible for producing source.

2. Public API

The package exports one entry point:

func NewPackage(pkgPath, pkgName string, files []Source, conf *Config) (ret Package, err error)

Parameter	Meaning
`pkgPath`	Import path of the generated Go package.
`pkgName`	Name of the generated Go package.
`files`	The input: one `Source` per header, each carrying its AST.
`conf`	Conversion configuration (name transformation rules, dependency information, and so on).

The result is:

type Package struct {
	*gogen.Package
	Wrap   *WrapFile
	Public []Entry // public entries
}

2.1 `*gogen.Package`

The embedded gogen.Package is the Go-side model of the output. It holds type declarations, functions, variables and constants as a typed program rather than as text. Callers use it to run the final code generation step and write .go files. Because it is typed, cl can build a type once and refer to it from many places, and gogen can verify that those references are consistent.

2.2 `Wrap`

Some C/C++ entities cannot be called through a plain symbol lookup:

inline functions that never get an out-of-line definition,
member functions and free functions of template instantiations.

For these, Wrap collects the information needed to generate a small C/C++ translation unit that defines exported wrapper functions with stable symbol names. The generated Go code calls the wrappers instead of the original entities. Wrap is therefore the bridge between "the C++ source level" and "something LLGo can link against".

2.3 `Public`

Public is a list of entries describing the C/C++ symbols that this package exposes. Each entry records the mapping between the C/C++ name (cName) and the Go symbol name chosen for it in the generated package. This list is serialized as llcppg.pub.

When another project depends on this library and is converted with llcppg, it reads this file so that a reference to, say, std::string or sqlite3_stmt in its own headers resolves to the correct Go type in the dependency's package, rather than being regenerated or guessed.

3. The three-phase pipeline

C and C++ headers are not ordered in a way that suits Go code generation. Types refer to each other, sometimes cyclically; templates are declared before they are fully defined; the same name can denote many overloaded entities. Processing declarations in a single pass in source order does not work.

NewPackage therefore splits the work into three phases:

AST decls ──► [1. preload] ──► [2. load] ──► [3. compile] ──► Package

  preload    -- addLoadUnit -------> load
  macro/enum ----------------------> load      (processed directly)
  load       -- addLoadUnit -------> load      (new units; always run before any compile unit)
  load       -- addCompileUnit ----> compile

3.1 Phase one: preload

Preload walks the declaration part of the AST once. For each declaration (type, variable, function, and so on) it records the declaration's prototype information, without building any Go types yet.

Its most important job is to learn, for every cName (the full name of a C/C++ symbol), how many overloaded instances exist. Only three kinds of symbols can legitimately share a name:

functions,
template functions,
template classes.

All other symbols cannot be redeclared under the same name, so they need no overload bookkeeping. Knowing the instance count up front matters because the Go name of each instance depends on whether it is alone or one of several; that decision must be made before any code refers to it.

Declarations that need deeper analysis are not processed immediately. Preload registers them for later phases:

type compileFunc = func(ctx *pkgCtx)

func (p *pkgCtx) addLoadUnit(f compileFunc)

Each registered compileFunc is a closure that captures one declaration and, when run with the package context, performs that declaration's work. This keeps preload cheap and order-independent, and it keeps the knowledge of "what to do with this declaration" next to the declaration itself.

Special case: macros and enums. These do not go through the deferred registration. They are processed directly in the load stage, because load-time type construction may depend on their results (for example, an array length written as a macro, or an enum used as a field type).

3.2 Phase two: load

Load constructs the Go types that correspond to C/C++ types. This is where cyclic references show up. Consider:

typedef unsigned long size_t;

template <class _Tp>
class weak_ptr;

template <class _Tp>
class shared_ptr {
public:
  typedef weak_ptr<_Tp> weak_type;
  typedef _Tp element_type;

private:
  element_type* __ptr_;
  size_t* __cntrl_;
};

template <class _Tp>
class weak_ptr {
public:
  typedef _Tp element_type;

private:
  element_type* __ptr_;
  size_t* __cntrl_;
};

shared_ptr mentions weak_ptr (through the weak_type typedef) before weak_ptr is defined, and in general two such classes can reference each other. If building a type required its dependencies to be fully built first, this input would deadlock.

Two steps per type

The solution is to separate creating a type name from giving it content. A class or typedef is converted in two steps, matching the gogen API:

func (p *Package) NewTypeDefs() *TypeDefs
func (p *TypeDefs) NewType(name string, tparams []*TypeParam, src ...ast.Node) *TypeDecl
func (p *TypeDecl) InitType(pkg *Package, typ types.Type) *types.Named
func (p *TypeDecl) AliasType(pkg *Package, typ types.Type) *types.Alias

Declare. NewType registers the name (with its type parameters, for templates) and returns a TypeDecl. After this call the type can be referenced by anyone, even though it has no definition yet.
Define. Once every relevant name has been declared, InitType supplies the underlying type for a class or struct (producing a *types.Named), or AliasType supplies the target of a typedef (producing a *types.Alias).

Load units can register more load units

Declaring a type is always possible, but defining it is not: InitType and AliasType need the thing they point to to exist. A load unit that discovers it cannot finish yet therefore does what it can (declare the name with NewType) and registers a new load unit with addLoadUnit for the remainder:

func (p *pkgCtx) addLoadUnit(f compileFunc)

So addLoadUnit is not only called by preload; it is re-entrant, and the load phase keeps running until its queue, including units added while it runs, is empty.

Two ordering rules follow:

A unit registered during load is appended to the load queue behind the units already waiting, so it naturally runs after the definitions it was waiting for.
Every load unit runs before any compile unit, no matter which was registered first. A compile unit registered early with addCompileUnit still waits until the load queue is completely drained, including load units created later. This guarantees that when compile starts, every type is both declared and defined.

Applied to the example

size_t is declared with NewType and defined right away with AliasType; its target is a builtin type and needs nothing else.
weak_ptr first appears only as a forward declaration, so at this point there is no implementation to convert.
The shared_ptr unit runs next. It declares SharedPtr[_Tp] and starts converting its members. At typedef weak_ptr<_Tp> weak_type; it can only call NewType to declare the new type SharedPtrWeakType[_Tp]. It cannot call AliasType to define it, because the implementation of weak_ptr has not appeared yet. So this unit registers a new load unit with addLoadUnit whose job is to finish the definition later.
The unit for the real weak_ptr definition runs and defines it with InitType.
The re-registered unit now runs, weak_ptr is available, and weak_type is defined.

The rule of thumb: declaring a name can happen at any time; defining a type waits until what it refers to is defined, and a unit that has to wait re-queues itself with addLoadUnit instead of blocking.

Nested classes become top-level types

Go has no nested types. Whenever C++ nests a class (or a typedef that needs its own Go type) inside another class, the converter hoists it to a package-level type whose name is the outer name followed by the inner name. Type parameters of the enclosing template are carried over.

For the example above:

shared_ptr becomes a generic top-level type:

type SharedPtr[_Tp any] struct {
	// ...
}

weak_type, nested inside shared_ptr, becomes its own top-level generic type alias:
```
type SharedPtrWeakType[_Tp any] = WeakPtr[_Tp]
```

This is also why the deferred definition above matters: SharedPtrWeakType is a separate Go type that has to be declared with NewType and then defined on its own with AliasType, and its target, WeakPtr[_Tp], depends on weak_ptr being defined.

Why this works

The cycle is broken at the level of names rather than at the level of definitions. Go itself allows recursive and mutually recursive named types, and gogen's declare-then-define API exposes exactly that capability, so the converter does not have to invent a workaround or reorder the C++ source.

Handing work over to compile

Load does not only build types. Functions and variables are not processed in load at all; when load reaches them, they are deferred to the next phase with a second registration call that mirrors addLoadUnit:

func (p *pkgCtx) addCompileUnit(f compileFunc)

addCompileUnit takes the same compileFunc closure shape as addLoadUnit, but the closure it receives is run in the compile phase rather than the load phase. This is the mechanism by which load hands off to compile: the phase that owns type construction queues up every function and variable closure, and none of them runs until all load units have finished and every type is declared and defined.

Using a registration queue for both phases keeps the structure symmetric. Each phase is simply "run every unit queued for me", and a phase can enqueue work for itself (load re-registering load units) as well as for the one that follows it. The compile phase starts only when the load queue is empty.

3.3 Phase three: compile

Compile runs the units registered through addCompileUnit during the earlier phases. These are the functions and variables. They share one useful property: no other declaration can refer to them in a way that affects type construction. A function's signature depends on types, but no type depends on a function or a global variable. This means they can be processed last, after every type is complete, and no cyclic problems arise.

In this phase cl:

builds Go function declarations from C/C++ prototypes, using the overload counts from preload to choose names,
records inline and template functions in Wrap so wrapper code can be generated for them,
emits variable declarations.

4. Overloading

Because preload knows the number of instances for each cName, later phases can treat overloaded entities uniformly: a symbol with a single instance keeps its plain name, while a symbol with several instances is given distinct Go names so that they do not collide. The same facility applies to template functions and template classes. Since only these three kinds of symbols can be overloaded, no other declaration pays the cost.

5. Cross-package references and `llcppg.pub`

Public is populated as declarations are completed: each exported C/C++ symbol contributes an entry that maps its cName to the Go symbol name in the generated package. Serialized as llcppg.pub, this becomes part of the library's binding.

When a downstream library is converted:

The configuration lists its dependencies.
For each dependency, its llcppg.pub is loaded.
When load meets a type whose cName is not defined in the current headers, it is looked up in the dependency tables, and the reference is emitted against the dependency's Go package instead of creating a duplicate.

This keeps bindings composable: a library built on top of another one shares the same Go types rather than shadowing them with incompatible copies.

6. Data flow summary

Stage	Input	Output
preload	Decl AST	Prototype info, overload counts, load units queued via `addLoadUnit`; macros and enums sent straight to load
load	Queued load units, macros, enums	Declared and defined Go types (`NewType`, then `InitType` / `AliasType`), with nested classes hoisted to top-level types; units that must wait re-register via `addLoadUnit`; function and variable work queued via `addCompileUnit`
compile	Compile units queued via `addCompileUnit` (function and variable decls)	Go functions and variables in `gogen.Package`, entries in `Wrap`
finish	Completed context	`Package{ Package, Wrap, Public }`

7. Design rationale

Phases instead of source order. C/C++ headers are authored for a compiler that tolerates forward declarations; Go generation needs a stricter dependency picture. Separate phases make that picture explicit.
Deferred closures (addLoadUnit). Registering work instead of doing it immediately keeps preload simple and lets load and compile control ordering.
Declare-then-define types. This mirrors how both C++ and Go treat named types, and it makes cyclic references a non-issue.
Re-entrant load units. A unit that cannot finish a definition yet declares the name and re-queues the rest instead of blocking, so no ordering of declarations in the header can deadlock the converter.
Load strictly before compile. Because all load units, including ones created during load, finish before any compile unit runs, compile never observes a half-defined type.
Flattened nesting. Go has no nested types, so nested C++ classes become top-level generic types named by concatenating the outer and inner names.
Functions and variables last. Nothing depends on them, so they cannot participate in cycles and can be processed with all type information available.
Public table as a first-class output. Treating symbol mapping as part of the result, not a side effect, is what makes multi-library conversion reliable.

Navigation Menu

Repository navigation

llcppg Design

1. Overview

Goals

Non-goals

2. Public API

2.1 `*gogen.Package`

2.2 `Wrap`

2.3 `Public`

3. The three-phase pipeline

3.1 Phase one: preload

3.2 Phase two: load

Two steps per type

Load units can register more load units

Applied to the example

Nested classes become top-level types

Why this works

Handing work over to compile

3.3 Phase three: compile

4. Overloading

5. Cross-package references and `llcppg.pub`

6. Data flow summary

7. Design rationale

Uh oh!

Clone this wiki locally

llcppg Design

1. Overview

Goals

Non-goals

2. Public API

2.1 *gogen.Package

2.2 Wrap

2.3 Public

3. The three-phase pipeline

3.1 Phase one: preload

3.2 Phase two: load

Two steps per type

Load units can register more load units

Applied to the example

Nested classes become top-level types

Why this works

Handing work over to compile

3.3 Phase three: compile

4. Overloading

5. Cross-package references and llcppg.pub

6. Data flow summary

7. Design rationale

Uh oh!

Clone this wiki locally

2.1 `*gogen.Package`

2.2 `Wrap`

2.3 `Public`

5. Cross-package references and `llcppg.pub`