REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processing

Of course. Here is a peer review of the paper from the perspective of "The Guardian."

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The paper proposes REIS, an In-Storage Processing (ISP) system designed to accelerate the retrieval stage of Retrieval-Augmented Generation (RAG) pipelines. The authors identify the I/O transfer of the vector database from storage to host memory as the primary performance bottleneck. To address this, REIS implements three key mechanisms: (1) a database layout that separates embeddings from document chunks and links them via the NAND Out-Of-Band (OOB) area; (2) an ISP-friendly implementation of the Inverted File (IVF) ANNS algorithm; and (3) an ANNS engine that repurposes existing NAND peripheral logic (e.g., page buffers, XOR logic, fail-bit counters) to perform Hamming distance calculations directly within the flash dies. The authors claim that this approach significantly improves performance and energy efficiency over conventional CPU-based systems and prior ISP accelerators, crucially, without requiring any hardware modifications.

Strengths

1. Problem Motivation: The paper correctly identifies and quantifies a critical performance bottleneck in large-scale RAG systems: the I/O cost of loading the vector database from storage (Section 3.1, Figure 2). The motivation is clear and empirically grounded.
2. Resourceful Mechanism Design: The core idea of leveraging existing, but typically inaccessible, peripheral logic within NAND flash dies for computation (Section 4.3, Figure 6) is a creative and efficient use of resources. Using in-plane XOR and bit-counters for Hamming distance is a clever way to avoid adding dedicated MAC units.
3. Comprehensive Baselines: The evaluation compares REIS against a high-end CPU system, an idealized "No-I/O" case, and two state-of-the-art ISP-based accelerators (NDSearch and ICE). This provides a robust context for assessing the claimed performance improvements.

Weaknesses

The paper’s central claims rest on a series of optimistic assumptions and questionable design choices that undermine its practical viability and rigor.

1. The "No Hardware Modification" Claim is Disingenuous: The authors repeatedly emphasize that REIS requires no hardware modifications, but this claim is misleading.

   • Hybrid SLC/TLC Partitioning: The system relies on programming a portion of the TLC SSD as SLC to achieve the reliability needed for ECC-less operation (Section 4.1.2). This is a major architectural decision with a direct cost: it sacrifices two-thirds of the capacity in that partition. This massive reduction in storage density is a practical modification with significant cost implications ($/GB) that are not acknowledged.
   • Firmware and Logic Overhaul: The proposal requires new, custom NAND flash commands (Table 2) and a significantly modified SSD controller to manage the complex, multi-step IVF search process (Section 4.4.2). This includes orchestrating data movement between latches, triggering in-plane computations, and managing custom data structures like the TTL in DRAM. This is a substantial firmware and controller logic modification, not a simple software overlay.

2. Unrealistic System-Level Requirements: The proposed database layout imposes impractical constraints on the storage system.

   • Physical Contiguity: The coarse-grained access scheme requires that the database regions for embeddings and documents be physically contiguous (Section 4.1.4 and Section 8). On a multi-terabyte SSD that undergoes wear-leveling, garbage collection, and bad block management, ensuring and maintaining large contiguous physical blocks is operationally infeasible. The paper dismisses this as a one-time "upfront overhead," but this fundamentally conflicts with how modern FTLs manage flash media over their lifetime.
   • FTL Simplification: The paper proposes a lightweight "R-DB" mapping structure to reduce DRAM footprint, effectively bypassing a conventional page-level FTL for the RAG database. However, it fails to adequately address how critical flash management tasks (e.g., bad block retirement, read disturb mitigation, wear-leveling) would be handled within these large, statically mapped regions. Stating that maintenance operations are "rare" (Section 4.1.4) is insufficient for a system intended for reliable deployment.

3. Insufficient Algorithmic Justification and Analysis:

   • Dismissal of HNSW: The authors discard graph-based algorithms like HNSW based on the high-level argument of "irregular data access patterns" (Section 4.2). This justification is superficial. They provide no simulation or detailed analysis of how HNSW would actually perform on their ISP architecture. The CPU-based comparison in Figure 5 is irrelevant for proving its unsuitability for an ISP design. It is plausible that caching the upper layers of an HNSW graph in the SSD's DRAM could yield competitive performance.
   • Distance Filtering Sensitivity: The performance of REIS leans heavily on the efficacy of Distance Filtering (DF), which is shown to provide the largest single optimization boost (Section 6.3, Figure 9). However, the selection of the filtering threshold appears heuristic. The claim that it "only weakly depends on the dataset size" (Section 4.3.3) is a strong one, yet it is supported by tests on only four datasets. This critical parameter’s robustness across different data modalities, embedding models, and query distributions is not proven.

4. Reliability Concerns Are Understated: The decision to disable ECC for the binary embeddings is a critical point of failure. The paper’s entire justification rests on Enhanced SLC Programming (ESP) achieving zero Bit Error Rate (BER). While ESP improves margins, claiming 0 BER over a drive's full lifetime, across variations in temperature and P/E cycles, is highly optimistic and lacks sufficient supporting data for the specific context of performing in-die computations. A single uncorrected bit-flip in an embedding could silently corrupt a search result.

Questions to Address In Rebuttal

1. Please provide a detailed cost-benefit analysis to justify the "no hardware modification" claim. Specifically, quantify the effective cost increase ($/GB) from sacrificing TLC capacity for the SLC partition. Furthermore, characterize the engineering complexity of implementing the new NAND commands and the bespoke IVF control logic in the SSD controller firmware.
2. Address the physical contiguity requirement in a real-world scenario. How would REIS handle database deployment on a partially filled, fragmented drive? What is the performance overhead of the defragmentation process required to create these contiguous regions, and how does this affect the total time-to-solution?
3. Provide a more rigorous, quantitative argument for choosing IVF over HNSW specifically for an ISP architecture. A simple reference to access patterns is insufficient. This requires at least a detailed simulation of HNSW’s data access patterns at the NAND channel/die/plane level to demonstrate that it would indeed lead to under-utilization.
4. Substantiate the claim that ESP can guarantee 0 BER for the SLC partition over the entire operational lifetime (e.g., 5 years, specified DWPD) of an enterprise SSD. Please provide either experimental data under accelerated aging or citations to literature that validates this specific claim in the context of forgoing ECC.
5. How is the Distance Filtering threshold determined for a new, arbitrary dataset? Please provide evidence of its robustness and performance impact on datasets with fundamentally different distributions from the BEIR benchmarks used in the paper (e.g., non-textual embeddings).

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents REIS, a complete, in-storage processing (ISP) system designed to accelerate the retrieval stage of Retrieval-Augmented Generation (RAG) pipelines. The authors correctly identify that for large knowledge bases, the I/O transfer of embeddings and documents from storage to host memory constitutes a primary performance and energy bottleneck.

The core contribution is a holistic, hardware/software co-designed system that pairs a storage-friendly Approximate Nearest Neighbor Search (ANNS) algorithm (Inverted File, or IVF) with a clever method of execution that leverages existing, unmodified computational logic within the NAND flash dies of a modern SSD. This is complemented by a tailored database layout that efficiently links embeddings to their corresponding documents and manages data placement to maximize internal parallelism. By moving the entire retrieval process (both search and document fetching) into the storage device, REIS demonstrates an order-of-magnitude improvement in performance (avg. 13x) and energy efficiency (avg. 55x) for the retrieval stage over a high-end server baseline.

Strengths

The true strength of this paper lies in its synthesis of ideas from disparate domains—information retrieval, computer architecture, and storage systems—into a single, cohesive, and remarkably practical solution.

1. Problem Significance and Framing: The paper tackles an exceptionally timely and important problem. As LLMs become ubiquitous, RAG is emerging as the dominant paradigm for grounding them with factual, up-to-date information. The authors provide clear empirical evidence in Section 3.1 (page 5, Figure 2) that I/O, not computation, is the scaling limiter for RAG, effectively motivating the need for a systems-level solution.

2. Holistic, End-to-End System Design: This is not merely a paper about accelerating ANNS. It addresses the entire retrieval problem. The authors have thought through the full data path: from the choice of an ISP-friendly algorithm (IVF over HNSW, justified in Section 4.2), to the physical data layout (Section 4.1), the low-overhead linkage of embeddings to documents using the NAND OOB area (Section 4.1.3), and the final return of only the relevant document chunks to the host. This complete vision is rare and highly valuable.

3. Pragmatism and High Potential for Adoption: The most significant contribution, from a practical standpoint, is the commitment to using existing hardware. The core computational kernel of the REIS ANNS engine (Section 4.3) is built upon repurposing standard peripheral logic in NAND flash dies—using XOR gates for distance calculation (for binary quantized vectors) and fail-bit counters for population counting. This "zero-cost" hardware approach dramatically lowers the barrier to adoption compared to prior academic proposals that require bespoke accelerators or significant modifications to the SSD controller. It transforms the problem from one of hardware design to one of firmware and system software, which is a much more tractable path to real-world impact.

4. Connecting Architectural Principles to Application Needs: The work serves as a powerful case study in the value of near-data processing. It demonstrates a deep understanding of the internal architecture of modern SSDs—leveraging channel, die, and plane-level parallelism—and directly maps these architectural features to the needs of a cutting-edge AI workload.

Weaknesses

The weaknesses of the paper are less about flaws in the core idea and more about the boundaries and future implications of the proposed system, which could be explored more deeply.

1. Implicit Assumption of a Static Knowledge Base: The proposed database layout, particularly the coarse-grained access scheme (Section 4.1.4) and the reliance on physical data contiguity, is highly optimized for a read-only or infrequently updated dataset. The paper would be strengthened by a more thorough discussion of the challenges of handling dynamic RAG databases where new documents are frequently added, updated, or deleted. The proposed defragmentation would be a significant overhead in such scenarios.

2. Specialization vs. Generality: The REIS engine is exquisitely tuned for the IVF algorithm on binary quantized embeddings. This tight co-design is its strength, but also a potential weakness. What is the path forward if retrieval techniques evolve? For instance, if future research demonstrates a clear superiority of graph-based methods even in an ISP context, or if higher-precision vectors are required, it's unclear how the REIS framework would adapt without the very hardware modifications it so successfully avoids.

3. Interaction with Standard SSD Management: The paper mentions that REIS operates in an exclusive "RAG-mode" to simplify the design and avoid interference with normal FTL operations like garbage collection (Section 7.2). While a pragmatic choice, this raises questions about the cost of context switching between modes and the performance implications for mixed-workload environments where the SSD must serve both RAG queries and traditional I/O requests.

Questions to Address In Rebuttal

1. Could the authors elaborate on the cost and complexity of updating a REIS-managed database? What would be the performance impact of frequent appends or updates, and can the reliance on physical contiguity be relaxed without sacrificing too much performance? An alternative linkage mechanism is briefly mentioned in Section 8, but its trade-offs are not fully explored.

2. The "exploit latent computation" approach is the paper's most brilliant aspect. Beyond the specific mapping of Hamming distance to XOR and popcount logic, have the authors considered what other computational primitives might be hiding in plain sight within SSDs? Does this work suggest a new direction for SSD design, where manufacturers might formally expose a small set of simple, parallel primitives (e.g., bitwise AND/OR, simple comparisons) for general-purpose, in-situ computation?

3. Regarding the "RAG-mode" vs. "normal-mode" operation, what is the anticipated latency for loading the respective FTL metadata and switching between these modes? In a multi-tenant cloud environment, how would the system arbitrate between a high-priority RAG query and an incoming write operation that might trigger garbage collection?

Of course. Here is a peer review of the paper from the perspective of "The Innovator."

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper introduces REIS, a retrieval system designed to accelerate the retrieval stage of Retrieval-Augmented Generation (RAG) pipelines via In-Storage Processing (ISP). The authors' central claim to novelty rests on a cohesive framework of three core mechanisms: (1) an ISP-tailored implementation of the cluster-based Inverted File (IVF) ANNS algorithm, chosen for its regular access patterns which are amenable to NAND flash architecture; (2) a hardware-less ANNS computation engine that repurposes existing SSD peripheral logic (latches, fail-bit counters) to perform binary distance calculations without adding new hardware; and (3) a novel database layout that links embeddings to documents using the NAND flash Out-Of-Band (OOB) area and employs a lightweight FTL to reduce host-side overhead. The work positions itself as the first complete ISP-based RAG retrieval system that avoids the pitfalls of prior ISP-ANNS accelerators, namely the use of ISP-unfriendly algorithms and the introduction of significant hardware modifications.

Strengths

The primary strength of this work lies in its specific, technically deep novel contributions that are elegantly tailored to the constraints of existing hardware.

1. Novel Repurposing of Existing Hardware for ANNS: The most significant novel idea is the in-storage ANNS engine detailed in Section 4.3 (Page 9). While prior work has proposed ISP for ANNS, those works typically involve adding dedicated accelerators (e.g., DeepStore [192]) or have different computational models. REIS’s proposal to perform XOR and population count (for Hamming distance on binary embeddings) by repurposing the existing Sensing Latch (SL), Cache Latch (CL), Data Latch (DL), and the fail-bit counter is a genuinely clever and non-intrusive approach. This "zero-hardware-modification" principle is a powerful and novel contribution in the domain of computational storage.

2. Novel Algorithm-Hardware Co-Design: The explicit choice of the IVF algorithm over the more commonly accelerated graph-based algorithms like HNSW (used in NDSearch [299]) is a key element of novelty. The authors correctly identify that HNSW’s irregular, pointer-chasing access patterns are a poor fit for the highly parallel but block-oriented access of a modern SSD. Their justification in Section 4.2 (Page 8) for selecting IVF due to its contiguous, streaming-friendly access patterns represents a novel insight in the co-design of ANNS algorithms and storage-level processing.

3. Novel System-Level Integration for RAG: The paper proposes a complete system, not just an ANNS accelerator. The mechanism for linking embeddings to their corresponding documents directly within the storage device using the OOB area (Section 4.1.3, Page 7) is a novel solution to the often-overlooked document retrieval part of the RAG pipeline. This integration ensures that the benefits of in-storage search are not lost to a subsequent, slow document fetch operation.

Weaknesses

My critique is focused on contextualizing the novelty and questioning the practical generality of some of the proposed mechanisms. The core ideas are strong, but their foundations rest on enabling technologies that are not themselves novel.

1. Foundational Mechanisms Are Not Novel: The core mechanism of performing bulk bitwise operations inside a NAND flash die is not new. The authors' own citation, Flash-Cosmos [224], is the foundational work that demonstrated this capability. REIS appears to be a novel and compelling application of this pre-existing technique to the ANNS problem (specifically, using XOR for Hamming distance). The authors should be more precise in claiming their contribution is the specific application and system integration, not the invention of in-flash bitwise computation.

2. Component Ideas Lack Originality: Similarly, the concepts of Hybrid SSDs (using SLC for performance-critical data and TLC for capacity) and leveraging the OOB area for metadata are established concepts in SSD design. While their application here—SLC for embeddings, TLC for documents, and OOB for document pointers—is novel in the context of RAG, the underlying architectural ideas are prior art.

3. Contiguity Requirement is a Significant Caveat: The proposed coarse-grained access scheme (Section 4.1.4, Page 7) relies on storing database regions in physically contiguous blocks to enable a lightweight FTL. This is a very strong assumption that is difficult to guarantee in a real-world, dynamic storage system that suffers from fragmentation. This practical limitation may reduce the novelty of the FTL optimization, as it is only applicable under idealized conditions that are not typical of general-purpose storage.

Questions to Address In Rebuttal

1. Please clarify the precise delta between the in-plane computation proposed in REIS and the foundational work in Flash-Cosmos [224]. Is the contribution the specific sequence of operations (Input Broadcasting, XOR, popcount) to calculate Hamming distance for ANNS, rather than the underlying mechanism of in-flash computation itself?

2. The selection of IVF is justified by its streaming-friendly access pattern. Is this contribution the specific choice of IVF, or the more general principle of selecting ANNS algorithms with high data locality for ISP? Have other streaming-friendly algorithms (e.g., certain forms of LSH) been considered, and if so, why was IVF superior?

3. The reliance on physical data contiguity (Section 4.1.4, Page 7 and Section 8, Page 14) is a major practical concern. How does the system handle a database update that leads to fragmentation? Does the need to perform potentially costly defragmentation operations negate the performance benefits of the lightweight FTL, thereby limiting the novelty of this specific optimization to write-once, read-many scenarios?