Assassyn: A Unified Abstraction for Architectural Simulation and Implementation

Here is a peer review of the paper from the perspective of 'The Guardian'.

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present Assassyn, a high-level programming framework intended to unify architectural simulation and RTL implementation from a single source description. The core abstraction models pipeline stages as functions that are invoked asynchronously, akin to asynchronous event handling in software. The system compiles this high-level description into two targets: a cycle-accurate simulator in Rust and a synthesizable RTL implementation in SystemVerilog. The authors evaluate Assassyn by implementing several hardware designs, from simple components to an out-of-order CPU, and compare them against handcrafted RTL and HLS-generated designs on metrics of productivity (LoC), performance, and area. The central claim is that this unified approach ensures perfect alignment between simulation and implementation while offering high productivity and generating quality hardware.

Strengths

1. Guaranteed Simulator-RTL Alignment: The paper's strongest contribution is the co-generation of a simulator and RTL from a single source. The claim that "all the cycles counts from Verilator simulated RTL exactly match our Assassyn-generated Rust simulator" (Section 7, Q5, page 11) is a significant and valuable property, directly addressing a well-known pain point in hardware development.

2. Coherent Abstraction Model: The core concept of modeling pipelined hardware as a system of asynchronously communicating functions is internally consistent and provides a structured way to think about dataflow and control flow between stages.

3. Demonstrated Breadth of Application: The authors have applied the framework to a non-trivial range of designs, including a systolic array, an in-order CPU, and a rudimentary out-of-order CPU. This demonstrates that the abstraction is not limited to a single architectural pattern.

Weaknesses

My primary concerns with this submission relate to the rigor of the evaluation and the strength of the claims, which appear to be overstated based on the evidence provided.

1. Unsupported Claims of "Comparable" Quality to Handcrafted RTL: The abstract and Section 7 (Q3) claim the generated RTL achieves "comparable perf/area compared to handcrafted RTL." The paper's own data contradicts this. In Figure 14 (page 10), the Assassyn-generated 5-stage CPU has an area of 365µm², a 42% overhead compared to the 257µm² Sodor reference. A 42% area increase is by no means "comparable" in a hardware context. The normalized perf/area metric in Figure 12 obscures this critical detail. The performance claim is also weak; a 2.6% IPC improvement (Figure 15a) is marginal and comes at a significant area cost. The claims must be revised to accurately reflect the data.

2. Flawed and Misleading Productivity Metrics: The Lines of Code (LoC) comparison in Section 7 (Q2, page 10) is methodologically unsound. The authors explicitly state, "We excluded all the highly overengineered common modules in Chipyard-related reference designs for Sodor CPU, and Gemmini systolic array, or the LoC comparison will be badly skewed." This is a textbook case of cherry-picking the comparison baseline to support a claim. A rigorous comparison requires including all necessary code or providing a principled justification for exclusions. As it stands, the 70% LoC claim is not credible. Furthermore, LoC is a poor proxy for design effort or complexity.

3. Absence of Stated Limitations and Overstated Generality: The paper presents Assassyn as a "general-purpose" framework but fails to discuss its limitations. What architectural features can it not express? The paper is silent on critical topics such as multi-clock domains, asynchronous resets, complex memory interfaces (e.g., AXI), or arbitrary finite state machines that do not fit the simple pipeline-stage model. The admission that radix_sort and merge_sort were "particularly challenging to manually manage" (Section 8.2, page 13) suggests the abstraction has weaknesses for control-intensive algorithms, but this is not explored as a formal limitation. Without a "Limitations" section, the "general-purpose" claim is unsubstantiated.

4. Superficial Simulator Performance Comparison: The comparison against gem5 (Section 7, Q5, page 11) is an apples-to-oranges comparison. Gem5 is a detailed, full-system simulation framework capable of booting operating systems. Assassyn appears to be a module-level or core-level simulator. The authors' own admission that "for longer-running workloads like qsort and rsort, gem5 achieves an order of magnitude speedup" invalidates the headline claim of achieving "2.2-8.1x simulation speedup" as a general finding. The speedup is only valid relative to Verilator and for short-running workloads.

5. Insufficient Detail on Advanced Implementations: The out-of-order (OoO) CPU case study (Section 7, Q6, page 12) is used to demonstrate scalability to complex designs. However, the implementation is described at a very high level. Critical microarchitectural details are missing: What are the sizes of the reservation station and reorder buffer? What is the issue width? What specific OoO features (e.g., register renaming scheme, memory disambiguation) were implemented? Without these details, it is impossible to assess whether Assassyn truly scales to a realistic OoO core or merely a toy example. The reported 1.26x speedup for a 1.43x area overhead is also a questionable design trade-off.

Questions to Address In Rebuttal

1. Please justify the claim that a 42% area overhead for the in-order CPU (Figure 14) is "comparable" to a handcrafted design. Please provide a quantitative definition of "comparable" that you will adhere to throughout the paper.

2. Please either (a) re-run the LoC comparison against the Sodor and Gemmini baselines without excluding the Chipyard modules you deemed "overengineered," or (b) provide a much more rigorous, file-by-file justification for why specific modules in the reference are not part of the core design and should be excluded.

3. Please enumerate the architectural features and hardware design patterns that Assassyn cannot currently support. Specifically address multi-clock designs, standard bus protocols like AXI, and arbitrary FSMs that do not map cleanly to pipeline stages.

4. To substantiate the claim of scalability to complex designs, please provide the key microarchitectural parameters for your OoO CPU implementation (e.g., issue/retire width, ROB size, reservation station size, load/store queue size).

5. The arbiter generation (Section 4.2) appears to rely on user-provided tags like #priority_arbiter. How does the framework handle more complex arbitration schemes (e.g., weighted round-robin, matrix arbiters)? Is the generation fully automatic for any topology, or does it require manual intervention for non-trivial cases?

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents Assassyn, a novel programming framework designed to unify the processes of architectural simulation and hardware implementation. The authors identify a fundamental and persistent challenge in computer architecture: the divergence between high-level performance models (often written in C++/Python) and low-level hardware implementations (written in RTL). This divergence creates significant overhead in maintaining alignment and verifying correctness across separate codebases.

The core contribution of Assassyn is a unified, high-level programming abstraction based on the concept of asynchronous event handling. The key insight is that the behavior of a pipelined hardware architecture can be elegantly modeled as a set of stages that asynchronously invoke one another, passing data forward. This single abstraction is expressive enough to describe complex hardware behavior and, crucially, can be compiled to generate both a cycle-accurate simulator (in Rust) and a synthesizable RTL implementation (in SystemVerilog). The evaluation demonstrates that this approach not only achieves its unification goal but also results in high productivity, high-quality hardware, and a simulation performance that is both fast and perfectly aligned with the generated RTL.

Strengths

1. Elegant and Powerful Core Abstraction: The central idea of using asynchronous event handling (or more simply, asynchronous function calls) as a unifying paradigm for both simulation and synthesis is exceptionally strong. The paper does an excellent job in Section 2 (Page 3) of motivating this by identifying the "push/pull mismatch" between simulation and RTL and highlighting the "transposed" correspondence between an event trace and an RTL waveform (Figure 2, Page 3). This is a deep and insightful observation that forms a solid foundation for the entire framework. It reframes hardware design from a structural netlist-centric view to a behavioral, dataflow-centric one.

2. Tackles a Foundational Problem: The work addresses a genuine and costly problem in the field. The effort spent bridging the gap between architectural models and RTL is a major bottleneck in both academic research and industrial practice. By proposing a fundamental solution rather than an incremental fix, this work has the potential for significant impact on how hardware is designed and verified.

3. Demonstrated Generality: A major risk for any new hardware description paradigm is that it only works for a narrow class of problems. The authors effectively mitigate this concern through a well-chosen set of evaluation targets. By successfully implementing not only a simple linear pipeline (in-order CPU) but also a complex 2D dataflow architecture (systolic array) and a design with significant control-flow complexity (an out-of-order CPU), they make a convincing case for the framework's generality. The bind abstraction for handling multi-source dataflows (Section 3.7, Page 5) is a particularly clever feature that enables this expressiveness.

4. Strong and Well-Rounded Evaluation: The paper is supported by a comprehensive evaluation that validates its primary claims. The results are compelling across multiple axes:

   • Productivity: A ~30% reduction in lines of code compared to handcrafted RTL is a significant improvement (Figure 11, Page 9).
   • Quality of Results (QoR): The generated RTL is shown to be comparable in performance and area to manual designs and vastly superior to a traditional HLS flow (Figure 12, Page 10). This is a critical result that proves the abstraction does not come at an unacceptable hardware cost.
   • Simulator Fidelity and Performance: The fact that the generated simulator is perfectly cycle-aligned with the RTL is a massive advantage over conventional simulators like gem5. Achieving this alignment while also being 2.2-8.1x faster than RTL simulation (Verilator) is the "killer app" of this unified approach (Figure 16, Page 11).
   • Agility: The progressive CPU design case study (Section Q6, Page 12) is a masterful demonstration of the framework's power, showing how architects can rapidly iterate on design choices and immediately understand the performance and area trade-offs.

Weaknesses

1. Scalability to Full System-on-Chip (SoC) Complexity: The presented examples, while impressive, are largely self-contained compute kernels or processors. It is less clear how the Assassyn paradigm scales to the integration challenges of a full SoC, which involves heterogeneous IP, complex memory subsystems, and standardized bus protocols (e.g., AXI, CHI). The simple async_call model may need significant extension to handle the transactional, multi-master nature of modern interconnects.

2. Positioning Relative to Modern Hardware Construction Languages: The paper positions itself well against traditional RTL and HLS. However, it could be more deeply contextualized against the landscape of modern hardware generator frameworks like Chisel/FIRRTL and SpinalHDL. While the authors correctly characterize Chisel as a more powerful "wrapper" for RTL (Section 8, Page 12), these frameworks are also evolving to incorporate higher-level abstractions. The paper would be strengthened by a more direct comparison of Assassyn's asynchronous event paradigm against the generator-based and metaprogramming paradigms used in these more established high-level tools.

3. Abstraction of Physical Realities: While the abstraction is a key strength, it may also hide important low-level details from the designer too effectively. The framework automates the generation of FIFOs and state machines, but provides little insight into how a designer would manage physical design challenges like timing closure, clock domain crossing, or layout-dependent wire delays. For a design to be practical, there must be a path for physical implementation feedback to influence the high-level source code. This feedback loop is not discussed.

Questions to Address In Rebuttal

1. Debugging and Verification: The paper claims that the serialized nature of the event-driven simulator simplifies debugging. While this is likely true for functional bugs, how does a designer debug a performance bug or a hardware-specific issue that only manifests in the generated RTL? For instance, if an unexpected stall occurs due to a full FIFO that was automatically generated, what tools or methodologies does Assassyn provide to trace this back to the high-level source and understand the cause?

2. Interoperability with Legacy IP: A key requirement for any practical hardware design flow is the ability to integrate existing, third-party IP, which is almost always delivered as standard Verilog/VHDL. How does an Assassyn-generated module communicate with a standard RTL module? Is there a defined methodology for wrapping legacy IP or for generating standard interface wrappers (e.g., AXI-Stream) on Assassyn modules?

3. The "C-like" Analogy: In the discussion (Section 8, Page 13), the authors make a powerful analogy, framing Assassyn as a "C-like" abstraction for hardware compared to the "assembly" of SystemVerilog. Could the authors elaborate on this? Specifically, what are the hardware design equivalents of software's unstructured goto that their paradigm deliberately constrains or eliminates, and what structured programming principles does it enforce in its place? Sharpening this analogy could further clarify the work's conceptual contribution.

Excellent. I will now embody "The Innovator" and provide a peer review focused exclusively on the novelty of the work presented in "Assassyn: A Unified Abstraction for Architectural Simulation and Implementation."

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present Assassyn, a programming framework intended to unify architectural simulation and RTL implementation from a single, high-level source code. The core abstraction proposed is based on asynchronous event handling, where pipeline stages are modeled as functions that are invoked asynchronously (async_call), and dataflows between stages are managed through function arguments and a bind primitive for multi-source inputs. The framework includes a compiler that transforms this high-level description into two backends: a cycle-accurate simulator in Rust and a synthesizable RTL implementation in SystemVerilog. The authors claim this is the "first RTL generator that is both high-level, and general-purpose" (Page 2, Section 1).

My assessment is that while the specific implementation and the synthesis of ideas from software engineering is elegant, the fundamental claim of novelty is overstated. The concept of a unified, high-level abstraction for generating both simulation models and RTL is not new. The novelty of Assassyn is therefore limited to its specific choice of abstraction—asynchronous function calls—which itself has strong conceptual parallels to existing paradigms in prior art that are not adequately addressed in this paper.

Strengths

1. Elegant Abstraction: The application of software engineering concepts like async function calls and bind (function partials) to hardware description is a clean and intuitive mapping for pipelined and dataflow architectures. The systolic array example (Page 6, Figure 5) is a particularly strong demonstration of the bind primitive's expressiveness for regular structures.
2. Demonstrated Generality: The authors successfully demonstrate their framework on a diverse set of designs, from simple components (priority queue), to dataflow accelerators (systolic array), to complex control-flow architectures (in-order and out-of-order CPUs). This lends credibility to the "general-purpose" aspect of their claim, differentiating it from more domain-specific languages for hardware generation.
3. Unified Backend Generation: The direct generation of a perfectly aligned simulator and RTL from a single source is a significant engineering achievement. The authors' empirical demonstration of subtle behavioral mismatches in established tools like gem5 (Page 11, Section Q5) provides a compelling motivation for this unified approach.

Weaknesses

My critique is centered on the core novelty claim.

1. The Concept of a Unified High-Level Abstraction is Not Novel: The paper frames the separation of simulation and RTL codebases as the primary problem it solves with a novel unified approach. However, this unification has been a central goal and achievement of high-level hardware construction languages for over two decades.

   • Bluespec SystemVerilog (BSV) [1]: BSV provides a high-level abstraction based on Guarded Atomic Actions (Rules). A design is expressed as a set of rules, and the BSV compiler generates both cycle-accurate, executable C++ models and synthesizable Verilog RTL from this single source. The core premise of unification is identical.
   • Chisel [6]: Chisel is a Scala-embedded DSL that generates Verilog. The same Chisel code is used to generate the RTL and can be executed to produce simulation behavior (via tools like Treadle or Verilator-backed simulation). While the authors argue Chisel adheres "closely to the RTL's programming and execution model" (Page 4, Section 2.1), it is nonetheless a unified, high-level, and general-purpose framework.

2. Conceptual Overlap with Prior Abstractions is Not Addressed: The paper's primary contribution must lie in its specific abstraction (async_call + wait_until). However, this model is functionally and conceptually very similar to the Rule-based model in BSV, a comparison the paper entirely omits.

   • An Assassyn stage defined with a wait_until condition is analogous to a BSV Rule with an implicit guard condition. The Assassyn stage "executes" when its condition is met; the BSV Rule "fires" when its guard condition is met.
   • Both abstractions aim to raise the level of thinking away from manual clock-edge management and state machine encoding. The "delta" between Assassyn's model and BSV's seems to be primarily syntactic and stylistic (imperative/functional async calls vs. declarative Rules) rather than a fundamental leap in expressive power or a new hardware paradigm. The lack of a direct comparison to BSV is a critical flaw in the paper's argument for novelty.

3. The Novelty Claim is Imprecise: The statement that Assassyn is the "first RTL generator that is both high-level, and general-purpose" (Page 2, Section 1) is demonstrably false, as BSV and Chisel both fit this description. The authors must narrow their claim to the specifics of their programming model and provide a compelling argument for why it represents a significant advancement over these established systems.

Questions to Address In Rebuttal

1. Please provide a detailed technical comparison between Assassyn's asynchronous event-handling model (async_call, wait_until) and the Guarded Atomic Actions (Rules) paradigm in Bluespec SystemVerilog. What specific hardware designs or behaviors can be expressed more easily or efficiently in Assassyn that are difficult or impossible in BSV, and vice-versa? Without this comparison, the novelty of the core abstraction is unclear.
2. Given the existence of Chisel and Bluespec SystemVerilog as prior art for unified, high-level, general-purpose RTL generation, please refine the central novelty claim of this paper. What precise, novel contribution does Assassyn make that these frameworks do not?
3. The proposed abstraction appears well-suited for architectures that can be modeled as a static graph of components pushing data to one another. How does the abstraction handle more dynamic or irregular control and data flows, such as a complex on-chip network router or a shared bus arbiter where components must pull data based on state, rather than having it pushed via a pre-bound connection?