Systems Migration:
What Breaks When Models Port C to Rust

Abstract

Automated C-to-Rust migration is marketed as a solved problem: run a transpiler or prompt a large model, take the output, fix the build errors. The produced code compiles. It runs. It looks right. The overwhelming majority of bugs appear only on edge cases the smoke test did not hit: aliasing introduced during translation, silent i32/usize sign conversions, off-by-one errors on CSC column-pointer arrays, and floating-point drift after a reduction was reordered.

The CSparse suite (Tim Davis, "Direct Methods for Sparse Linear Systems"^[1]) is an unusually good test bed. The C is dense numerical code with no undefined-behavior ambiguity; every task has a deterministic reference output that the Rust port can be compared against bit-exactly; and the port is line-by-line translatable at the arithmetic level while memory management (malloc/free) requires the translator to rethink ownership rather than transliterate. The environment covers 28 tasks spanning column-norm kernels, triangular solves, transposition, graph reachability, and a full LU factorization.

This report focuses on three of those tasks: one where every model failed (csparse_lu), one where four of five produced correct output and then silently missed the performance gate (csparse_transpose), and one where almost every model succeeds (vec_add), establishing the contrast. The same five frontier models were run once each, 140 episodes total.

Summary Statistics

Overall Score by Model

Featured Task Scores by Model

The featured-task slice understates the middle of the distribution. Across all 28 tasks, Mistral Large 3 and Kimi K2.5 each cleared a ≥ 0.5 bar on 12 tasks; Flash 3 Preview on 11; Sonnet 4.6 on 10; Flash 2.5 on 5. Easy kernels (vec_add, vec_scale, csparse_lsolve) are near-saturated. The 7 all-zero tasks cluster at the top of the difficulty ranking: csparse_lu, csparse_symperm, csparse_dfs, csparse_transpose, csparse_cumsum, vec_dot, and all three Verus formal-proof tasks.

Scoring

Each task runs deterministically inside a Docker container with no network and no GPU. Scores are continuous in [0, 1]. A port must clear three composed layers before it earns credit:

• Compilation gate. cargo build --release against the Rust crate must succeed, with the scored function exported as #[no_mangle] pub unsafe extern "C" fn matching the C signature. A Rust port that does not expose the exact FFI shape cannot be linked against the reference driver and scores zero.
• Bit-exact correctness. The Rust output is compared against the C reference on seeded randomized sparse matrices. Any numerical mismatch, including floating-point drift that exceeds tolerance, scores zero. The comparison uses element-wise equality with tight tolerances; a reordered reduction typically fails.
• Performance sigmoid. Geometric mean of per-config speedup (Rust wall-time vs. C wall-time) is fed into a sigmoid whose inflection sits above 1×. Exact coefficients are not disclosed. A port that matches the C reference at equal speed scores near zero; a port that achieves roughly 2× scores near 0.5; a port around 4× approaches 1.0. The measurement uses 50 warmup plus 200 timed iterations with the median reported. Expected run-to-run noise is ±2–5% in raw speedup, which maps to ±0.01–0.03 in final score.

On top of those three gates, idiomatic-Rust multipliers penalize raw-pointer operations relative to std::slice::from_raw_parts bridges, and penalize cargo clippy warnings. Both multipliers bottom out at 0.40–0.50 floors and top out at 1.00. A port that wraps everything in unsafe to silence the borrow checker is still correct, but the safety multiplier discounts it materially. Exact weights are not disclosed in this public report to keep the environment useful as a benchmark.

Task 1: csparse_lu

What the agent must solve

Port cs_lu, the CSparse sparse LU factorization with partial pivoting. Given a square matrix A in CSC format and a symbolic analysis S, produce factors L, U, and a row permutation pinv such that L·U = A(pinv, q). The Rust port links against pre-compiled C helpers (cs_spsolve, cs_spalloc, cs_sprealloc, cs_spfree) via FFI; the scored function itself must re-implement the outer loop, pivot selection, and L/U assembly in Rust.

The C reference is 95 lines. The difficulty is not the line count. It is that every single step of the loop has a data-dependence edge across the raw-pointer graph: L->p, L->i, L->x, U->p, U->i, U->x, pinv, x, xi, and two possible reallocations via cs_sprealloc that can invalidate the locally-cached slice references the Rust port wants to hold. Translating this as "wrap everything once in from_raw_parts_mut" does not work, the reallocation moves the backing storage, and the slice is dangling.

The LU factorization algorithm is textbook (see Davis §6.5^[1]); the challenge is the memory model.

C reference: the outer loop

/* Sparse LU factorization of A(q,:), with partial pivoting.
   Returns L, U, pinv such that L*U = A(pinv, q). */
csn *cs_lu(const cs *A, const css *S, double tol)
{
    /* ... allocations, init ... */
    for (k = 0; k < n; k++) {
        Lp[k] = lnz;
        Up[k] = unz;
        /* Grow L, U if needed, INVALIDATES Li,Lx,Ui,Ux */
        if (lnz + n > L->nzmax && !cs_sprealloc(L, 2 * L->nzmax + n)) { ... }
        if (unz + n > U->nzmax && !cs_sprealloc(U, 2 * U->nzmax + n)) { ... }
        Li = L->i; Lx = L->x; Ui = U->i; Ux = U->x;  /* re-read
                                                   after realloc */
        col = q ? q[k] : k;
        top = cs_spsolve(L, A, col, xi, x, pinv, 1);
        /* find pivot, divide by pivot, append row to L and U */
    }
    ...
}

Rust scaffold (shipped to the agent)

#[no_mangle]
pub unsafe extern "C" fn cs_lu(
    a: *const Cs,
    s: *const Css,
    tol: c_double,
) -> *mut Csn {
    todo!("implement cs_lu")
}

Observed failure mode: L mismatch on test 0 (n=10)

Four of five models produced code that compiled. Three of those four hit the same correctness failure: the scoring harness reports "L mismatch on test 0 (n=10)". That is, the very first, smallest benchmark instance, a 10×10 matrix, already diverges. The Rust port is returning an L factor whose column pointers or row indices do not match the C reference for an input that has only a handful of nonzeros.

Gemini 2.5 Flash failed during cargo build with Could not resolve host: index.crates.io, an environment-level DNS transient on that episode, unrelated to the model's code. The score of 0.000 is counted for consistency, but it is not a reasoning failure on this task.

Why n=10 fails

The dominant class of errors in the traces is not "we wrote the wrong algorithm." It is "we wrote the right algorithm against a view of the matrix that went stale." The CSparse pattern is: grab Li, Lx, Ui, Ux as pointer-typed locals at the top of the loop body; call cs_sprealloc only if the current capacity is about to be exceeded; re-acquire the locals after each potential reallocation. Translating this directly into Rust with slice::from_raw_parts_mut once at the top of the loop produces a slice whose backing storage has already moved by the time the write happens.

The n=10 case is small enough that some configurations happen to not trigger a realloc on that iteration, which is why the bug is not "always". It is "whenever reallocation happens within the loop body the port has been holding on to a dead slice." Two mistakes hide inside this pattern:

• A raw *mut c_int re-read from (*L).i after cs_sprealloc is safe in C and safe in Rust. A &mut [c_int] constructed before the cs_sprealloc call has &mut provenance tied to the pre-realloc allocation and is undefined behavior to use afterwards.
• A sign conversion from c_int column pointers to usize indices without handling the empty-column sentinel (Lp[k] == Lp[k+1]) can turn a zero-length run into a negative step that wraps to a very large usize. On an n=10 matrix with a sparse column, this is the first value the harness sees.

Agent trace: Kimi K2.5 (score 0.000, 43 tool calls)

The episode budget ran out before the root cause was isolated. Across all three compiling models (Flash 3 Preview, Kimi, Mistral), the trajectory is similar: 30+ tool calls, more than half a million tokens, a final submission that compiles and almost matches the reference. "Almost" scores zero.

Task 2: csparse_transpose

What the agent must solve

Port cs_transpose, sparse matrix transposition in CSC. Given a matrix A with m rows, n columns, column pointers Ap, row indices Ai, and optional values Ax, produce C = A', an n×m CSC matrix. The standard two-pass algorithm: first pass counts the number of entries per row of A (which become columns of C), then writes a cumulative-sum as Cp; second pass scatters Ai/Ax into Ci/Cx using a working copy of Cp as the per-column write cursor.

The algorithm is simple enough that a reasonable translation into Rust writes itself. Four of the five models produced exactly that.

Observed result: correct output, below-parity speed

Every port that built and ran produced output that was bit-exactly identical to the C reference across all five benchmark configurations (500×500 through 8000×8000). Every port also ran at effectively the same speed as the C reference, within measurement noise of 1.00×. The speedup gate rewards the port only when it actually beats the C reference; parity is explicitly not enough. All four correct ports land on the sigmoid's flat lower region and score zero.

Why parity is not enough

For transposition, the runtime is memory-bandwidth-bound. The working vector is sized to the larger of m and n; both passes over the nonzeros are linear. A straightforward Rust port compiles to roughly the same machine code as the C reference; LLVM has no autovectorization opportunity because the writes are scattered, not strided.

A port that would clear the gate on csparse_transpose needs one of:

• Two passes merged into one with a pre-computed per-row cursor array that shifts the scatter pattern to something the prefetcher handles better.
• SIMD-assisted radix-like binning of row indices (useful only at high density; all five benchmark configurations here are sparse).
• A rewrite that swaps the scatter direction: iterate destination-first so writes are sequential even though reads become indirect.

None of the five models attempted any of these. Each produced a straightforward two-pass transliteration. The resulting port is production-reasonable code that a human reviewer would approve. It is also exactly the kind of port that looks like a pass and is not.

Agent trace: Mistral Large 3 (score 0.000, correct, 0.99×)

Mistral is not wrong about its code. It is producing a reasonable port. The report here is not "Mistral failed to write good Rust." It is "a good Rust port of cs_transpose is not what the migration actually needed to deliver." The customer who ran this port against the rest of their CSparse-based pipeline would notice the performance regression only in aggregate, weeks later, probably in a flame-graph review.

Task 3: vec_add

What the agent must solve

The control case. Given x, y, n, write z[i] = x[i] + y[i] for i in 0..n. Three benchmark configurations: n = 50K, 200K, 500K. The C reference is three lines.

Observed result

Every model compiles. Every model is correct. All five models beat the C reference, the Rust port is meaningfully faster than the literal C in vec_add.c. The spread (1.19× to 1.60×) reflects how aggressive each model was about idiomatic slice iteration: a zip(&x).zip(&y).for_each pattern autovectorizes cleanly under -O2, whereas a manual indexed loop with bounds-check does not.

Side-by-side: C reference and a passing Rust port

/* Elementwise addition of two arrays:
   z[i] = x[i] + y[i] for i in 0..n */
void vec_add(
    const double *x,
    const double *y,
    double *z,
    int n)
{
    int i;
    for (i = 0; i < n; i++) {
        z[i] = x[i] + y[i];
    }
}

#[no_mangle]
pub unsafe extern "C" fn vec_add(
    x: *const c_double,
    y: *const c_double,
    z: *mut c_double,
    n: c_int,
) {
    let n = n as usize;
    let xs = std::slice::from_raw_parts(x, n);
    let ys = std::slice::from_raw_parts(y, n);
    let zs = std::slice::from_raw_parts_mut(z, n);
    for ((zi, xi), yi) in
        zs.iter_mut().zip(xs).zip(ys)
    {
        *zi = *xi + *yi;
    }
}

This is what the easy case looks like. The Rust port is actually faster than the C. The reason: in the C, the compiler does not know whether x, y, z alias, so it cannot trivially autovectorize without restrict. In the Rust port, the bridge to &[f64] and &mut [f64] carries non-aliasing provenance, and LLVM produces an auto-vectorized loop for free. The same performance advantage is the cause of the most common subtle failure mode in the harder tasks: a model that invents aliasing between two &mut slices during translation writes Rust code that is unsound, and the resulting port either miscompiles under aggressive optimization or fails bit-exactness because two writes to the same address now reorder.

Why this task is the control

vec_add has no shared state, no sparse structure, no index gymnastics, no FFI object model beyond raw pointers. Any model that cannot clear this task has a problem with the basic mechanics, reading the scaffold, writing the signature, returning. All five models cleared it. Which means the failures on harder tasks are not about "can it write Rust at all." They are specifically about what changes when the port has to manage ownership across reallocation, match a reference byte-for-byte, or beat C on memory-bandwidth-bound code.

Significance

• Compilation is not correctness, and correctness is not performance. On csparse_transpose four of five models produced bit-exactly correct output. All four scored zero because they matched, rather than beat, the C reference. A single-gate "does it work" benchmark would have called this task solved. The composed three-layer gate is what catches the difference between a port and a migration.
• The hardest task class is not the longest one. cs_lu is 95 lines of C. cs_transpose is 30. Every model failed cs_lu on an n=10 input, not because the algorithm is long, but because the interior of the loop mutates a data structure whose backing storage may move. That is a pattern Rust's type system refuses to express the same way C does, and no model in this evaluation found the translation that survives reallocation.
• "Wrap it in unsafe to make the borrow checker stop complaining" is a distinct and common failure mode. Several of the failing CSparse ports in the run files show the agent using raw-pointer .add() indexing for the entire loop body, bypassing the slice bridge, in order to suppress borrow-checker errors. These ports compile and are often correct on the happy path, but the safety multiplier explicitly discounts them. More importantly, the pattern signals that the agent has given up on carrying non-aliasing information through the translation, which is exactly the information a migration was supposed to add.
• Seven of twenty-eight tasks score 0.000 across all five models. Those tasks cluster at the difficulty boundary: cs_lu, cs_symperm, cs_dfs, cs_transpose, cs_cumsum, vec_dot, and the three Verus formal-proof tasks. The Verus tasks require a formal layer Rust's types cannot express alone, aliasing properties, array-length invariants, and functional-correctness specifications the type system has no vocabulary for. Nobody cleared them.

For customers running an actual C-to-Rust migration, the practical takeaway is not "models cannot do this." On easy kernels they clearly can, and the resulting Rust outruns the C. The takeaway is that the ports which need human review most urgently are not the ones that fail the smoke test. They are the ones that pass the smoke test, look correct, and miss the bit-exactness or performance gate that the CI pipeline does not check. The three-layer structure is the cheapest way to find those ports before a customer does.

For RL training, the partial-credit structure gives a signal at every skill level, vec_add provides early reward, csc_matvec and vec_scale provide mid-level reward, and the all-zero cluster provides a frontier that no amount of prompt engineering reaches. Scoring runs entirely inside a sandboxed Docker container with no network and no GPU; results are deterministic down to the seeded benchmark matrices.