Neoscope: How Resilient Is My SoC to Workload Churn?

Review Form:

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present Neoscope, an ILP-based design space exploration tool intended to find System-on-Chip (SoC) configurations that are resilient to "workload churn"—the evolution of software over a device's lifetime. They introduce a quantitative framework to describe churn using two metrics, Magnitude (ε) and Disruption (Δ), and categorize churn into four types. The primary contribution is a method that, unlike prior work (HILP), claims to find a near-optimal SoC for a given workload in a single solver invocation by inverting the JSSP formulation. The paper then uses this tool to explore the design space under various churn scenarios and for different optimization objectives (performance, energy, cost, etc.).

However, the practical utility of the tool and the validity of its conclusions are severely undermined by a series of significant, and in some cases unjustified, abstractions and simplifications in both the hardware and workload modeling. The "optimality" claimed is only with respect to a model that deviates substantially from real-world systems, casting doubt on the real-world applicability of the derived insights.

Strengths

1. Problem Formulation: The paper correctly identifies a critical and timely problem. As hardware lifetimes extend, designing for future, unknown workloads is an increasingly important challenge for computer architects. The attempt to formalize this problem is commendable.
2. Churn Metrics: The introduction of Magnitude (ε) and Disruption (Δ) (Section 3.2, page 4) provides a simple and intuitive, if basic, vocabulary for quantifying and discussing workload churn. This framework offers a useful conceptual lens.
3. ILP Formulation Extension: The technical extension to the HILP model using "exclusive machine ranges" (Section 3.4, page 5) to allow the solver to select hardware configurations is a clever methodological contribution. This inversion of the problem is the core technical novelty of the tool itself.

Weaknesses

1. Critically Oversimplified Hardware Models: The credibility of the entire study hinges on the fidelity of its models, which are demonstrably insufficient.
   • The model for Dedicated Special Accelerators (DSAs) as having a fixed "4x efficiency advantage over a GPU with the same area" (Section 4, page 7) is arbitrary and lacks empirical grounding. The paper's central trade-off between general-purpose GPUs and specialized DSAs is entirely dependent on this single, hardcoded number. The conclusions are therefore not generalizable and may be artifacts of this specific assumption.
   • The decision to omit Dynamic Voltage and Frequency Scaling (DVFS) from the model (Section 3, Implementation, page 6) is a fatal flaw for any analysis claiming to optimize for energy, power, or cost efficiency. Modern SoCs rely fundamentally on DVFS to manage these resources. Conclusions about the energy efficiency of "medium-size SoCs" (Abstract) or the layout of "carbon-optimal SoCs" (Section 5.3) are rendered speculative at best, as they ignore the primary mechanism for energy management in real hardware.
2. Artificial Workload Manipulation: The experimental setup relies on unrealistic modifications to the workload benchmarks.
   • The authors state they "artificially squash all setup and teardown phase times by 20x" (Section 4, page 7) to prevent CPU bottlenecks. This is a significant intervention that fundamentally alters the nature of the applications, biasing the problem to be almost entirely about accelerator performance. This calls into question any conclusions about the required number of CPU cores or the overall SoC composition, as a primary source of system-level contention has been artificially removed.
   • The churn models themselves are synthetic. While based on real-world growth rates, the mechanisms—such as uniformly multiplying AIO counts or swapping the two most expensive AIOs (Section 4, page 8)—do not capture the true nature of software evolution. Real churn often involves the introduction of entirely new algorithms and computational patterns (i.e., new AIO types), which this framework is incapable of modeling. The study is therefore limited to resilience against re-balancing of existing work, not the emergence of new work.
3. Unjustified Claims of Optimality: The paper repeatedly uses terms like "globally optimal" and "near-optimal" (Abstract, Section 3.3). This is misleading. The tool finds an optimal solution for an abstracted, simplified model. Given the points above, there is a significant risk that a configuration deemed optimal by Neoscope would be far from optimal when implemented in a real system with DVFS, realistic CPU overheads, and more complex DSA/GPU trade-offs. The conclusions are presented with a degree of certainty that the underlying model does not support.

Questions to Address In Rebuttal

1. Please provide a rigorous justification for the "4x efficiency advantage" of a DSA over a GPU. Crucially, provide a sensitivity analysis showing how the key findings (e.g., the SoC compositions in Figure 8) change if this factor is varied (e.g., to 2x or 8x).
2. How can the paper's conclusions regarding energy, cost, and carbon efficiency (Sections 5.3, 5.4, 6) be considered valid in the complete absence of DVFS modeling? Please defend this methodological choice and explain why it does not invalidate your findings.
3. Justify the decision to reduce CPU-bound phase times by a factor of 20x. How would the optimal SoC configurations change if this artificial reduction was removed or lessened (e.g., to 2x)?
4. The study explores churn by re-weighting existing AIOs. How would your framework and conclusions be affected by a more realistic churn scenario where entirely new AIOs (incompatible with existing DSAs) are introduced over the SoC's lifetime? Please address this fundamental limitation of your churn model.
5. Given the significant abstractions in the model, do you agree that all claims of "optimality" should be strictly qualified as "optimal with respect to the proposed abstract model" rather than implying real-world optimality?

Of course. Here is a peer review of the paper from the perspective of "The Synthesizer."

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces the concept of "workload churn" to characterize the evolution of software over the increasing lifespan of hardware. It makes a significant conceptual contribution by formalizing churn along two axes: Magnitude (change in total work) and Disruption (change in the mix of operations). Based on this framework, the authors present Neoscope, a multi-objective pre-silicon design space exploration (DSE) tool. Neoscope's core technical innovation is its use of Integer Linear Programming (ILP) and job-shop scheduling concepts, not just to find an optimal schedule for a given System-on-Chip (SoC), but to co-optimize the schedule and the SoC configuration itself within a single invocation. Building on prior work like HILP [32], it "inverts" the problem to find the ideal hardware for a given workload trajectory. The authors use Neoscope to explore the resilience of SoCs to different churn profiles (Minimal, Perturbing, Escalating, Volatile) and derive several key architectural insights regarding specialization, over-provisioning, and the impact of optimizing for metrics like energy, cost, and carbon footprint.

Strengths

1. Excellent Problem Formulation and Timeliness: The single most important contribution of this paper is the clear and quantitative framing of the "workload churn" problem. The concepts of Magnitude and Disruption provide a much-needed vocabulary and analytical framework for a challenge that is central to modern computer architecture. As hardware replacement cycles lengthen and software evolves at an accelerating pace, designing for lifetime performance consistency rather than just launch-day peak performance becomes paramount. This work is perfectly positioned at the intersection of these trends.

2. Novel and Powerful Methodology: The technical approach of Neoscope is both clever and impactful. By extending the ILP model from HILP [32] to include hardware selection via "exclusive machine ranges" (Section 3.4, page 5), the authors elegantly sidestep the need for massive, brute-force parameter sweeps that plague traditional DSE. This makes a comprehensive exploration of resilience not just theoretically possible, but computationally tractable. This is a significant step forward for early-stage architectural modeling.

3. Rich and Insightful Architectural Analysis: The paper does not merely present a tool; it effectively uses the tool to uncover non-obvious design principles. The findings—that small SoCs are particularly vulnerable to high-Disruption churn, that over-provisioning is beneficial for high-Magnitude churn (Section 5.1, page 9), and that medium-sized SoCs strike the best balance for energy efficiency (Section 5.3, page 10)—are valuable heuristics for architects. The analysis of carbon optimality (Section 6, page 11), which suggests that large amounts of "dark silicon" in the form of specialized DSAs can be optimal, is particularly counter-intuitive and thought-provoking. This demonstrates that the tool can genuinely guide architects toward better, more sustainable designs.

4. Excellent Contextualization: The work is well-situated within the existing literature. It clearly identifies its intellectual lineage from models like MultiAmdahl [48] and HILP [32] and articulates its value proposition against more detailed simulators like Aladdin [35]. The integration of carbon modeling concepts from FOCAL [10] shows a commendable awareness of adjacent, important research areas.

Weaknesses

While the core ideas are strong, the work rests on several abstractions whose implications could be explored more deeply.

1. Fidelity of the Churn Model: The modeling of churn by adjusting AIO [33] counts is a necessary and effective abstraction for the ILP framework. However, true workload evolution can be more complex than simply changing the frequency of existing operations. A fundamental algorithmic shift (e.g., the move from CNNs to Transformers in AI) might introduce entirely new types of operations that are not in the initial AIO set. The current model captures this as a high-Disruption event but doesn't fully account for the introduction of fundamentally new work primitives.

2. Simplicity of the Hardware Model: The paper models DSAs as having a fixed "4x efficiency advantage over a GPU with the same area" (Section 4, page 7). This is a reasonable simplification for an early-stage tool, but the true benefits of specialization are highly dependent on the specific task. The strength of the paper's conclusions, particularly those regarding the dominance of DSAs in carbon-optimal designs, may be sensitive to this assumption.

3. ILP Scalability Concerns: The authors are transparent about the significant computational cost (up to 10,000 CPU hours, Section 4, page 7), but a more formal discussion on the scalability limits of the ILP formulation would be beneficial. How does the solver complexity scale with the number of IP block options, AIO types, or churn time-steps? Understanding these boundaries is crucial for assessing the tool's applicability to even more complex, future SoCs.

Questions to Address In Rebuttal

1. Could the authors elaborate on how the Neoscope framework might handle a "Volatile" churn scenario where entirely new AIOs (not present at year 0) are introduced at year 5? Does the model assume a static, known universe of all possible AIOs for the device's lifetime, or could it be adapted for such unforeseen changes?

2. The conclusion that dark silicon (in the form of DSAs) is carbon-optimal is fascinating. How sensitive is this finding to the 4x DSA efficiency assumption? For example, if the advantage were only 2x, at what point on the operational-vs-embodied carbon spectrum (the α_E2O sweep in Figure 10) does the more general-purpose GPU become the more carbon-optimal choice?

3. The paper focuses on pre-silicon DSE. Given its powerful modeling capabilities, could the authors speculate on how the Neoscope methodology could be applied in a post-silicon context? For instance, could it be used to guide strategic software updates for an existing device fleet to maximize performance on aging hardware, or to inform resource management on FPGAs in the field?

Of course. Here is a peer review of the paper from the perspective of "The Innovator."

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents Neoscope, a design space exploration (DSE) framework for System-on-Chips (SoCs) intended to be resilient to "workload churn." The authors' claimed contributions are twofold. First, they propose a conceptual framework to quantify workload churn using two metrics: Magnitude (ε), representing the change in total work, and Disruption (Δ), representing the change in the operational mix. Second, they present a multi-objective optimization tool, based on Integer Linear Programming (ILP), that extends a prior framework (HILP [32]) to synthesize a near-optimal SoC configuration in a single invocation, rather than requiring an iterative parameter sweep over fixed SoC designs. The core technical mechanism for this is an ILP formulation that allows the solver to select hardware components from a library. The paper then uses this tool to explore the design space under various churn scenarios and for different optimization objectives (performance, cost, carbon, etc.).

Strengths

The primary novel contribution of this work is methodological, residing in the formulation of the SoC design problem. The authors correctly identify a key limitation in their direct predecessor, HILP [32]: HILP can find an optimal schedule for a workload on a fixed SoC, but finding the optimal SoC itself requires an external, brute-force search loop.

The core innovation in Neoscope is the extension of the HILP model to absorb this outer search loop into the ILP formulation itself. This is achieved by introducing what the authors call "exclusive machine ranges" (Section 3.4, page 5). By modeling a library of IP blocks (e.g., a GPU with 1, 2, 4, or 8 SMs) as a set of mutually exclusive "machines" in the Job-Shop Scheduling Problem (JSSP) formulation, the solver is forced to choose exactly one configuration for each component type as part of the optimization. This transforms the problem from "optimal scheduling on a fixed design" to "simultaneous optimal design selection and scheduling." This is a significant and elegant step forward from the prior art they cite. While ILP-based synthesis is not new in itself, its application to invert the DSE paradigm at this level of architectural abstraction is a genuinely novel approach within this specific context.

The second, softer contribution is the formalization of "churn" via the ε and Δ metrics (Section 3.2, page 4). While the concept of workload evolution is not new, providing a simple, quantitative framework to describe it is valuable. It provides a clear and useful vocabulary for the problem domain.

Weaknesses

My main critique concerns the framing of the novelty. While the application of the ILP technique is novel, the underlying technique itself is not.

1. The Core ILP Mechanism is Standard: The concept of "exclusive machine ranges" is a standard modeling technique in mathematical programming, often implemented using Special Ordered Sets (SOS1) or simple binary variable constraints to enforce a "pick one from N" choice. The authors should explicitly acknowledge that the novelty is not in the invention of this ILP constraint style, but in its specific application to augment the JSSP model for SoC synthesis. The current framing could be misinterpreted as the invention of a new class of constraint.

2. Overlaps with High-Level Synthesis (HLS): The problem of simultaneously performing resource allocation and scheduling is the central challenge of High-Level Synthesis. For decades, HLS research has employed ILP-based methods to solve this exact problem, albeit at a lower level of abstraction (i.e., synthesizing RTL from C/C++). For instance, ILP formulations have been used to determine the optimal number of functional units (adders, multipliers) and to schedule operations onto them to meet latency or area constraints. While Neoscope operates at a much coarser, architectural level (CPUs, GPUs, DSAs vs. ALUs), the fundamental problem structure is conceptually identical. The paper is missing a discussion of this vast body of prior art and fails to clearly differentiate its contribution from it. The novelty is in the level of abstraction, not in the fundamental approach of using ILP for co-optimization of resources and schedule.

3. Breadth of the "First Tool" Claim: The abstract claims Neoscope is the "first multi-objective pre-silicon design space exploration tool for investigating... architectures that are resilient to workload churn." This claim is too strong. DSE tools are numerous, and many are multi-objective. The unique, novel element is the single-invocation synthesis method based on the HILP extension. The novelty claim should be sharpened to focus on this specific mechanism, as it is the paper's most defensible and significant contribution.

Questions to Address In Rebuttal

The authors should use the rebuttal to clarify the precise boundaries of their novel contributions.

1. Can the authors please confirm that the core "exclusive machine ranges" technique is a standard ILP modeling pattern and re-state their contribution as the novel application of this pattern to the architectural DSE problem defined in HILP [32]?

2. Please elaborate on the relationship between Neoscope and the body of work on ILP-based High-Level Synthesis. How is your problem formulation fundamentally different from prior HLS work that also co-optimizes resource allocation (i.e., component selection) and scheduling, beyond the obvious difference in architectural abstraction level (SoC blocks vs. functional units)?

3. The proposed churn framework (ε and Δ) is based entirely on aggregate execution times per AIO type. Does this abstraction hide other critical aspects of churn, such as changes in memory access patterns, communication-to-computation ratios, or control flow divergence, which might render a synthesized SoC suboptimal despite ε and Δ remaining stable? Why is this execution-time-centric view sufficient?