Valen Creator Pioneers Cross-Language Generics with Rust
Evan Ovadia's 'Golden Spike' connects Rust libraries to new languages, bypassing C ABI limitations.
1Breaking the C ABI bottleneck
Evan Ovadia's work enables direct generic type sharing between Rust and other languages, circumventing C ABI's lack of generics support. The C ABI forces data into primitive types, losing rich type information. Ovadia's solution preserves generics across language boundaries, allowing complex data structures to traverse ecosystems intact. This addresses a fundamental limitation in current interop approaches, where libraries must sacrifice functionality to be usable across languages.
2Rust compiler as embeddable library
The solution leverages Rust's existing but rarely used capability to run its compiler (rustc) as a library (rustc_driver). This allows Valen to embed Rust's compiler directly in its memory space. By treating rustc as a library rather than an external process, Valen gains fine-grained control over compilation. This architectural choice enables the tight integration needed for cross-language generic resolution while maintaining Rust's compilation guarantees.
3Bidirectional compiler collaboration
Ovadia implemented a two-way communication protocol between compilers. When Rust encounters Valen types, it delegates compilation to Valen's compiler and awaits results. This reciprocal arrangement required patching Rust's compiler to recognize foreign types. The 100-line patch assumes LLVM backend usage, making it non-portable but functional. This creates a symbiotic relationship where both compilers contribute to final code generation.
4Practical implications for new languages
Valen demonstrates how new languages can bootstrap from Rust's ecosystem without reinventing foundational libraries. Developers can focus on language design rather than rebuilding mature libraries like wgpu. This approach could accelerate development of domain-specific languages that need Rust's performance or safety features. It shifts the interoperability paradigm from lowest-common-denominator (C ABI) to shared capability.
5Technical debt and limitations
The solution carries significant technical debt - the Rust patch is implementation-specific and non-upstreamable. It depends on LLVM and makes assumptions about Rust's internals. Community feedback highlights concerns about Rust's type system constraints for universal interop. Some suggest Ada might offer better foundations. The approach works today but may require maintenance as Rust evolves.
6Industry reception and skepticism
Programming communities recognize the achievement while noting practical hurdles. Reddit and Lobste.rs users praise the ambition but question Rust's suitability as universal FFI replacement. Comments highlight the tension between Rust's safety guarantees and flexibility needs for cross-language work. The discussion reveals broader industry frustration with C's dominance in interop scenarios.
7Historical parallel to transcontinental rail
Ovadia metaphorically calls this the 'Golden Spike' moment, referencing the US transcontinental railroad completion. Like connecting disparate rail systems, his work links compiler ecosystems. The metaphor emphasizes how interoperability unlocks new possibilities - just as the railroad enabled cross-country commerce, cross-language generics could enable new programming paradigms.
8Future of language interoperability
This experiment suggests a path beyond C-based FFI, potentially influencing future language design. While not production-ready, it demonstrates what's possible when compilers cooperate rather than compete. The approach could inspire similar efforts for other language pairs, gradually reducing C ABI dependence. It represents a conceptual shift in how we think about multi-language systems.