ComputeDRAM: Hardware Memory Controller Technology for Enabling Massively Parallel In-Memory Computation on Standard Memory Chips

Background

Modern computing systems rely heavily on memory technologies such as DRAM, PCM, flash, and RRAM to store and retrieve vast amounts of data for processing by CPUs, GPUs, and other accelerators. As applications in artificial intelligence, big data analytics, and high-performance computing continue to grow, the demand for faster and more energy-efficient data processing has become critical. Traditional architectures require data to be moved from memory to processing units for computation, resulting in significant energy consumption and latency due to the so-called “memory wall.” This bottleneck limits the overall system performance and efficiency, especially as data sizes increase and the need for parallel processing intensifies.

Current approaches to address these challenges often involve either modifying memory chips to support in-memory computation or relying on specialized hardware accelerators that still require extensive data movement between memory and processors. Modifying DRAM or other memory technologies for computation has proven difficult to commercialize, as it necessitates changes to well-established manufacturing processes and standards, leading to increased costs and compatibility issues. Additionally, even with advanced accelerators, the fundamental problem of data transfer overhead remains, as data must still traverse the memory bus, consuming power and introducing delays. These limitations have hindered the widespread adoption of in-memory computing solutions and have left a gap in achieving truly high-throughput, low-power computation directly within existing memory infrastructures.

Technology

This technology is a specialized memory controller that enables in-memory computation by orchestrating rapid, back-to-back row activations within standard memory chips such as DRAM, PCM, flash, or RRAM. The controller, which can be implemented as an FPGA or ASIC, uses precisely timed command sequences to hold multiple memory rows open simultaneously, leveraging the natural charge-sharing on bit-lines to perform logical operations like AND, OR, and row copying directly within the memory array. This approach allows for massively parallel computation without requiring any modifications to the memory chips themselves. The controller also incorporates mechanisms to detect and bypass faulty memory columns, ensuring reliable operation. It can be integrated as a GPU or machine learning co-processor, an accelerator card, or as a modified main memory controller, thereby providing high-throughput, low-power computation by eliminating the need for off-chip data transfers.

What differentiates this technology is its ability to perform computation directly within unmodified, commercially available memory chips, a significant departure from previous approaches that required custom memory modifications. By shifting computational tasks from the CPU or GPU into the memory itself, the system dramatically reduces data movement, leading to substantial power savings and increased performance, especially for data-intensive applications. The use of standard memory hardware means it can be adopted quickly and cost-effectively, without waiting for new memory designs or manufacturing processes. Furthermore, its adaptability to various memory types and integration options—ranging from standalone accelerator cards to enhancements in existing computing architectures—makes it a versatile solution for accelerating workloads in fields such as machine learning, scientific computing, and graphics processing. The ability to map out and avoid faulty columns further enhances its robustness and practical deployability in real-world systems.

Benefits

• Enables massively parallel in-memory computation without modifying standard DRAM chips.
• Supports multiple memory technologies including DRAM, PCM, flash, and RRAM.
• Performs logical operations (AND, OR) and row copy directly within memory arrays.
• Reduces power consumption by eliminating off-chip data transfers.
• Can be integrated as a GPU or machine-learning co-processor, accelerator card, or modified memory controller.
• Detects and bypasses faulty memory columns to maintain reliable computation.
• Compatible with commercially available memory, facilitating easy adoption in existing systems.
• Improves throughput and efficiency by leveraging specialized memory controller hardware and precise command timing.

Applications

• Machine learning acceleration
• Database query processing
• Real-time analytics engines
• High-performance scientific computing
• In-memory data encryption

Additional info

A specialized memory controller orchestrates rapid row activations, simultaneously opening multiple memory rows. This leverages bit-line charge-sharing to execute logical AND/OR operations or row copies directly within the memory array. This hardware enables massively parallel, low-power, high-throughput in-memory computation without modifying standard memory chips, eliminating off-chip data transfers. It applies to various memory technologies.