Compiler internals

The TinyFX compiler is responsible for language semantics once. Its output is a checked Frontend, which all execution targets consume. Backends may choose their own representation and calling convention, but they must not repeat semantic resolution.

The important implementation entry points are:

• crates/tinyfx-compiler/src/frontend.rs — source loading, parsing, checking, and stage analysis;
• crates/tinyfx-compiler/src/check — declarations, expression checking, overload resolution, defaults, captures, and resource identity;
• crates/tinyfx-compiler/src/stage.rs — GPU compatibility and shader/compute validation;
• crates/tinyfx-compiler/src/wgsl — shader and compute emission;
• crates/tinyfx-compiler/src/cpu_ir.rs and typescript.rs — generated-program lowering and readable output.

Frontend compilation

compile_with_std is the simple entry point for a single source file. compile_with_imports additionally accepts an ImportResolver supplied by the CLI, WASM binding, or language server.

The compiler:

1. lexes source text;
2. parses a spanned, error-recovering AST;
3. resolves user imports transitively;
4. appends the embedded standard-library modules;
5. builds declarations and checks bodies;
6. analyzes GPU legality and shader/compute sites; and
7. returns a Frontend plus every diagnostic found along the way.

The entry module is module zero. Imports are organizational: all compiled modules currently share one program namespace. Imports under /_std/ are accepted as standard-library organization and do not call the host resolver.

The standard library is ordinary TinyFX source embedded at compile time from std/*.tfx. It includes mathematical, geometry, color, image, mesh, model, profiling, option, and environment modules. This makes the standard library part of every front-end compilation rather than a runtime package lookup.

Parser recovery is intentional. A syntax error produces placeholder nodes and the checker is skipped when errors would cause cascades. Editor integrations can still fall back to a cached standard-library front end for useful completion while a document is incomplete.

The checked Frontend

The AST is treated as immutable after parsing. Semantic facts live in SideTables, keyed by a module/node identity. This is the central rule for compiler work: add a semantic decision to a side table, then have every affected backend consume that fact.

The main categories are:

Advanced-compute sites additionally record explicit resource access roles, the unordered determinism boundary, slot target identity and consumption, logical operation plans, and opaque GPU-count origins. WGSL, VM bytecode, generated TypeScript, pass explanation, and .tfxb consume these facts; they do not re-resolve resource aliases or operation effects.

This avoids backend-specific semantic drift. For example, a named argument still evaluates in source order, but the side table tells each backend which positional parameter receives it. A default argument is selected by the checker; the VM and generated code preserve its callee-owned semantics without re-deciding whether it applies.

Type checking and stage analysis

The type table interns types and stores nominal identity for structs, traits, and enums. Type inference is bidirectional: expected types flow into literals, while initializers and return statements provide inferred types where an annotation is absent. Overload selection happens at the call site.

Stage analysis then determines whether each function is GPU-compatible:

• gpu functions must validate as GPU-compatible;
• cpu functions may not be reached from a GPU context;
• ordinary functions are analyzed from their bodies and call graph;
• recursive GPU call paths are rejected because WGSL cannot express them;
• shader vertex/fragment bodies and parallel-for kernels are checked against their applicable restrictions.

The analysis records enough provenance to produce call-chain diagnostics, instead of merely saying that a call is illegal. Fragment derivatives and explicit-gradient texture sampling are tracked separately from ordinary GPU compatibility because they are legal only in fragment code and must remain in uniform control flow.

Enums are nominal in the checked program but lower to integer representation in GPU code. Named enum members can be GPU values; dynamic enum construction remains a CPU operation.

Captures and shader interfaces

Shader and closure captures are explicit semantic values, not an unstructured environment blob. A capture is a root plus an optional struct-field path. The checker narrows broad reads such as env to the field path actually used, so a shader can upload env.camera without uploading the whole environment.

Capture order matters because it participates in uniform packing, VM instructions, and resource binding. It is first-use order after narrowing, not declaration order. Local-rooted captures are snapshotted when a shader value is created; global-rooted captures are read at draw time.

The WGSL emitter turns the checked shader interface into:

• explicit uniforms, varyings, and attributes;
• packed uniform fields for numeric captures and promoted globals;
• group-one resource bindings for textures, samplers, and storage buffers;
• default vertex behavior for fragment-only shaders when appropriate;
• fragment output and depth metadata; and
• lifted/specialized arrow functions for GPU higher-order calls.

parallel for follows the same model. It becomes a compute blueprint with a domain, captures, resource roles, written-resource order, uniform layout, and bounded workgroup candidates. The compiler emits all supported entry points for those candidates; the runtime chooses only among safe candidates for its device and workload.

Sibling lowerers

VM lowering

tinyfx_vm::compile_program lowers the Frontend into a bytecode Program. While doing so it requests WGSL blueprints for shader and compute sites and builds exact packing plans for runtime use.

Generated-program lowering

cpu_ir::lower produces a compiler-owned structured CpuProgram, then typescript::emit writes readable TypeScript/ESM. The IR retains authored and emitted-safe names, logical source locations, reachability, resource identities, CPU lifecycle structure, and physical WebGPU descriptions without leaking compiler-local type IDs.

It is not VM bytecode expressed in TypeScript. The generated CPU program uses JavaScript helpers for TinyFX semantics and yields host/GPU work at explicit boundaries. The generated WebGPU path consumes strict physical blueprints and the compiler’s WGSL.

When a feature is unsupported for the generated target, lowering reports a source diagnostic instead of emitting a plausibly typed but semantically incorrect fallback.

Operation catalog and declarations

The operation catalog in operation_catalog.rs gives logical API operations stable identities and records their stage, target availability, owner, invocation style, backend routes, and conformance evidence. The Builtins registry is the checker’s materialized lookup structure. Regular function, method, and property signatures are expanded from catalog definitions; a small amount of checker-specific glue remains where the language has special syntax or lookup behavior.

The public operation reference is generated from this catalog by:

tfx reference --format markdown

That generated reference is the right place for exhaustive operation listings. Narrative compiler documentation should explain ownership and lowering rules, not manually duplicate hundreds of signatures.

The serialized BKey and MKey tables remain append-only. Their position is part of the bytecode wire contract even when the operation catalog provides a more useful logical identity.

Formatting and IDE support

The formatter is a token-stream reprinter, not an AST pretty-printer, because the AST intentionally drops comments. It preserves comments and authored line breaks while normalizing whitespace and indentation, and canonicalizes implicit line-end statement terminators by inserting ;. Its core invariants are:

• explicitly terminated input re-lexes to the same token stream; and
• formatted output still parses.

The second invariant matters because adjacent greater-than tokens can represent a shift operator. Inserting whitespace between them can change parsing.

Shared editor semantics live in ide.rs: completions, expected-type enum completion, signature help, hover, and definitions all operate over a checked Frontend. The LSP and compiler WASM bindings are adapters over that same analysis rather than separate language implementations.