Handwired: Programs as Neural Networks
How Handwired replaces control flow with numeric data flow, and what that does—and does not—make possible.
1. Summary
Handwired, by Cheng Lou, explores an unusual interpretation of a neural network: a program whose behavior is constructed explicitly rather than learned from examples. Its numeric operations encode the intended logic, so the initial program needs no training.
The central technique is to replace control flow with data flow. Instead of deciding which instruction to execute next, the programmer builds connected arithmetic operations that produce the same result under declared input and numerical assumptions. This extends beyond thresholds to bounded loops, selection, state machines, and structured parsing.
The qualification is important: Handwired is not an automatic compiler for arbitrary JavaScript. JavaScript builder code constructs a numeric circuit; the surrounding host and execution backend still perform essential work. The project is best understood as an exploration of explicit programs in a neural-network-compatible representation, not a drop-in JavaScript accelerator.
2. What is actually built?
The technical documentation describes a program as a fixed list of operations over 32-bit floating-point numbers (FP32). Its four instruction kinds are:
- Weighted sums: affine combinations of inputs and intermediate values.
- ReLU-gated products: products involving a rectified linear unit,
relu(x) = Math.max(0, x). - Boolean rounding: operations that turn scores into exact zero-or-one values.
- Table reads: operations that retrieve a value using a numeric index.
A JavaScript builder runs once and appends operations to a Circuit. Constants, inputs, event bits, and persistent state occupy cells in a Float32Array; wires refer to those cells or to operation results. The runtime evaluates the resulting program.
The host writes inputs, invokes evaluation, copies next-state outputs back into persistent cells, and performs effects such as DOM updates. A circuit therefore need not contain the entire application or its event loop. The main runtime evaluates on the CPU; the JSON search demo has its own WebGPU backend.
3. A concrete conditional
Consider this JavaScript expression:
const result = a > 4 ? 10 : 12;For small integer inputs, the same result can be expressed arithmetically:
const relu = x => Math.max(0, x);
const above4 = relu(a - 4) - relu(a - 5);
const result = above4 * 10 + (1 - above4) * 12;For integers at or below 4, both ReLU terms are zero. At 5 and above, their difference is one, provided the arithmetic stays within an exact representable range. The resulting bit selects an answer without a branch instruction.
The domain restriction is part of the construction. At a = 4.5, the selector is 0.5 and the answer is 11, unlike the original conditional. The simple formula is not an exact comparison over arbitrary real-valued inputs. Handwired supplies more elaborate comparison builders with explicit numerical contracts; the example illustrates the mechanism, not a universal translation.
4. How other control flow is represented
The examples below explain the constructions in ordinary JavaScript or pseudocode, not the literal Circuit builder API. The project’s patterns guide describes the implementations and their costs.
4.1. Fixed loops: expand the computation
A loop with a known number of iterations can become a fixed chain:
let total = 0;
for (let i = 0; i < 4; i++) total += values[i];The builder can emit an addition for each iteration, connecting each partial sum to the next. A loop in the builder creates circuit structure; it is not necessarily a loop in the finished circuit.
When regrouping is valid, a balanced tree can shorten the dependency chain:
The first two additions are independent. For larger inputs, the longest chain can fall from linear to logarithmic depth. That is not automatically faster: tree constructions can add work, and floating-point addition is not associative. Matching a sequential FP32 loop bit for bit may require preserving its order.
4.2. Conditional loops and early exits: bound and mask
A conditional loop can be represented by a maximum number of steps, with a bit that determines whether each step updates the state. For integer x between 0 and 9:
while (x > 1) x -= 2;At most four iterations are needed. Construct four copies of this conceptual step, feeding each output into the next:
const active = bit(x > 1); // Conceptual exact comparison: 0 or 1.
const nextX = x - 2 * active;Once the condition becomes false, subsequent steps subtract zero and preserve the state:
A break can similarly become a carried “finished” bit that disables later updates. A continue can mask the remainder of one iteration while permitting the next iteration to proceed.
This preserves results, not early-exit savings. The fixed circuit still runs its operations. Masked candidate computations must also remain numerically valid: multiplying an unwanted infinity by zero produces NaN, not a safely ignored value. The maximum step count must cover all admitted inputs; an arbitrarily chosen limit is not equivalent to an unbounded loop.
4.3. State machines: make transitions into data
Nested conditionals or a switch over a finite state can become a transition table:
const nextState = transitions[eventType][currentState];A button might have states for idle, pressed inside, and pressed outside. Its state and event type are numeric indices; a table read returns the next state. For an event sequence, each result becomes the next transition’s input.
There is also a parallelization opportunity: two events together define a new transition table, “apply the earlier event, then the later one.” Composing these tables is associative, so a tree or prefix scan can combine them while retaining event order. This trades additional table work for a shorter dependency chain. It becomes expensive when the state space is large.
State machines are used concretely in the JSON demo’s tokenizer, grammar checker, and search automaton. These are substantially richer computations than a single threshold test.
4.4. Persistent state: feed outputs back between evaluations
A fixed circuit need not be stateless across application frames. For example, a toggle’s next state can be described as:
const nextOn = clicked * (1 - on) + (1 - clicked) * on;For exact bits, a click flips the state and no click preserves it. The host stores nextOn back into the on cell before the next evaluation. This is a recurrent state update implemented around a fixed computation, not an infinite sequence of layers embedded in one evaluation.
The repository’s toggle template demonstrates the larger pattern, including animation and DOM writes.
4.5. Repeated steps: sometimes the backend keeps a real loop
Handwired does not always expand repeated work into one enormous circuit. The JSON search demo constructs the numeric computation for one character’s work. Its generated GPU shader contains an ordinary for loop that runs the step over document positions, carrying registers between iterations.
This separates the size of the step program from the number of times it runs. It also makes the boundary explicit: the numeric step is a circuit, while the runner supplies iteration. This loop runner belongs to the JSON demo rather than a universal looping facility in the core runtime.
4.6. Algorithmic restructuring: avoid the original loop
Some control flow disappears by choosing a different formulation:
- Search: compute each candidate’s hit bit, then select the first hit with a tree. When the predicate changes only once along an ordered list, counting the passing items can directly give the boundary index.
- Sorting: compute an item’s rank by counting how many items precede it, with an explicit tie-breaking rule. This exposes parallel comparisons but can require quadratic work.
- Linear dynamics: replace repeated applications of a fixed linear update with a matrix power. In one documented spring implementation, the host computes the power and the circuit applies the resulting matrix.
- Nested parsing: encode a bounded stack numerically. The JSON highlighter stores object-versus-array information as one bit per nesting level and reports an error when its supported depth is exceeded.
These are deliberate algorithm and representation choices, not automatic consequences of replacing if with ReLU.
5. A real application: JSON search on the GPU
The JSON search demo searches up to 100,000 generated documents at each keystroke. It uses two numeric programs:
- A JSON highlighter and validator identifies character kinds and errors.
- A regex-like search program finds matches, optionally restricted to keys or string values.
Documents are uploaded once and remain on the GPU. Each document runs the same operations independently, which makes parallel execution useful. Search patterns are encoded in numeric tables read by already-compiled programs; every keystroke does not require compiling a new shader.
The search performs a reverse pass and a forward pass to identify leftmost, longest, non-overlapping, nonempty matches. That differs from JavaScript RegExp’s first-alternative semantics for some patterns. The accepted pattern syntax is also a subset of JavaScript’s, not a drop-in replacement.
The implementation has concrete bounds: documents have a capacity of 512 code points, the JSON parser supports nesting through depth 16, and the default search programs hold up to 8, 16, or 32 automaton states. Excess length, depth, or pattern complexity is reported rather than silently treated as supported.
The author reports roughly 11–15 ms per search at keystroke-like intervals over 100,000 documents on an Apple M5 Max in Chrome 155. Native RegExp is sometimes faster for ordinary patterns. These are upstream measurements, not benchmarks reproduced for this report. The relevant advantage is batching many inputs with resident GPU data, not a guarantee that expressing logic as a network is faster.
The other main demos are HandwiredOS, a small desktop with windows and a dock, and a learned photo gallery whose layout coefficients are trained using the program’s gradients.
6. Exactness and differentiability are separate questions
“No training required,” “exact,” and “differentiable” describe different properties and should not be conflated.
No training required means the programmer supplies the initial logic and coefficients. It does not prohibit subsequently training selected coefficients.
Exactness depends on the representation and its contract: integer ranges, input grids, table bounds, FP32 rounding, and operation order. An exact integer state machine is different from an animation whose numeric trajectory is close to, but not bit-identical with, a JavaScript reference. JavaScript’s ordinary numbers use double precision, so FP32 execution alone does not promise identical results for arbitrary source programs.
Differentiability is useful through arithmetic paths, but hard decisions do not automatically provide useful gradients. ReLU has a kink at zero; a Boolean rounding operation is discontinuous; and a discrete table index does not have an ordinary informative derivative with respect to the index. The documented backward evaluator rejects Boolean atoms by default, and index gradients require an explicit zero-gradient policy when differentiation reaches an index. Table values can still receive gradients.
Consequently, the claim that the whole program is “fully differentiable” needs qualification. Training through a hard choice may require a different, soft formulation. Representing program behavior as arithmetic does not make every branch decision smoothly learnable.
7. Assessment
Handwired is not limited to if statements or thresholds. Its simple operations can compose into bounded iteration, finite-state transitions, persistent state, selection, and parsing. Some tasks become trees or closed-form computations; others remain sequential steps executed by a surrounding runner.
The strongest mental model is designing a numeric circuit for an algorithm. Like a digital circuit, it can implement complex behavior from simple parts, but the representation, resource bounds, and execution arrangement matter. Arbitrary recursion, unbounded iteration, dynamic allocation, and external side effects do not automatically fit inside one fixed graph.
Its research appeal is the bridge between explicitly programmed logic and trainable computation. Its practical GPU appeal is more conditional: repeated work across many independent inputs can benefit, while small, irregular, or strongly sequential tasks may be better served by ordinary code.
8. Sources
This report synthesizes the discussion and the following primary project documentation. Links point to the evolving main branch, not a pinned release.