With the growth of computing and communication technologies, the information processing paradigm of the healthcare environment is evolving. The patient information is stored electronically, making it convenient to store and retrieve patient information remotely when needed. However, evolving the healthcare systems into smart healthcare environments comes with challenges and additional pressures. Internet of Things (IoT) connects things, such as computing devices, through wired or wireless mediums to form a network. There are numerous security vulnerabilities and risks in the existing IoT-based systems due to the lack of intrinsic security technologies. For example, patient medical data, data privacy, data sharing, and convenience are considered imperative for collecting and storing electronic health records (EHR). However, the traditional IoT-based EHR systems cannot deal with these paradigms because of inconsistent security policies and data access structures. Blockchain (BC) technology is a decentralized and distributed ledger that comes in handy in storing patient data and encountering data integrity and confidentiality challenges. Therefore, it is a viable solution for addressing existing IoT data security and privacy challenges. BC paves a tremendous path to revolutionize traditional IoT systems by enhancing data security, privacy, and transparency. The scientific community has shown a variety of healthcare applications based on artificial intelligence (AI) that improve health diagnosis and monitoring practices. Moreover, technology companies and startups are revolutionizing healthcare with AI and related technologies. This study illustrates the implication of integrated technologies based on BC, IoT, and AI to meet growing healthcare challenges. This research study examines the integration of BC technology with IoT and analyzes the advancements of these innovative paradigms in the healthcare sector. In addition, our research study presents a detailed survey on enabling technologies for the futuristic, intelligent, and secure internet of health things (IoHT). Furthermore, this study comprehensively studies the peculiarities of the IoHT environment and the security, performance, and progression of the enabling technologies. First, the research gaps are identified by mapping security and performance benefits inferred by the BC technologies. Secondly, practical issues related to the integration process of BC and IoT devices are discussed. Third, the healthcare applications integrating IoT, BC, and ML in healthcare environments are discussed. Finally, the research gaps, future directions, and limitations of the enabling technologies are discussed. Full article
If you want to learn more, please visit our website Xieyuan Electronic Technology Co., Ltd..
The rest of this article is structured as follows: Section 2 summarizes the metrics used for the classification, the overview of different classes, namely, Computation-In Memory Array (CIM-A), Computation-In Memory Periphery (CIM-P) , and Computation-Out-of-Memory (COM) . Section 3 presents summary of architectures in the CIM-A class. Similarly, Section 4 presents architectures in CIM-P class followed by the COM class architectures in Section 5 . Each section describes and evaluates the architectures in their respective classes qualitatively. Section 6 discusses the prospect and challenges of memory-centric architectures. Section 7 concludes the article.
Comprehensive survey and comparison of different existing architectures and provide an outlook on the characteristics of the future architectures.
To assess the developments in Computation-In Memory (a.k.a. memory-centric architectures) and determine the innovation potentials, it is essential to classify and evaluate the existing memory-centric architectures. In this regard, our previous work presented a classification methodology and broad overview of memory-centric computing [ 140 ]. As follow-up of that earlier work, this article refines the classification metrics and presents a comprehensive set of classification metrics for memory-centric architectures. The metrics are then used to classify the existing memory-centric architectures. Then, for each class, based on the classification metric, a comprehensive survey on existing memory-centric architectures available in the literature is provided. Therefore, the main contribution of this article is a comprehensive survey on existing memory-centric architectures by thoroughly discussing and evaluating the advantages and disadvantages of different memory-centric architectures belonging to the main classes of the classification. Thus, the article contributions are summarized as follows:
Since the first Von-Neumann architecture in early 1950s, computer architectures have been evolved to various complex organizations, including pipelined, superscalar, multicore, and so on [ 69 , 80 , 96 ]. Since the energy and performance costs to move data between the memory subsystem and the CPU dominated the total costs of computation, architects and designers are forced to find breakthrough in computers architecture. Memory-centric computing paradigms such as Processing-in-memory (PIM) have evolved as promising solution to circumvent the aforementioned challenges [ 102 , 135 ]. PIM, a concept of integrating memory and processing units, so-called Logic-in-Memory (LIM) , was invented in 1970 [ 178 ]; however, it was only applied to cache/on-chip memory and was soon abandoned due to the reduced performance during non-local memory accesses [ 150 ]. Since 2000, big data and embedded applications have been on the rise and demand new computing systems with not only higher performance but also energy efficiency. To fulfill these requirements, several architectures were explored with the concept of LIM applied for main memories, and regarded as PIM architectures (i.e., FlexRAM [ 99 ], DIVA [ 47 ], intelligent RAM [ 105 ] etc.). However, the PIM architectures were also shortly dismissed due to the limitations of embedded DRAM technology [ 91 , 100 , 101 ]. From the year 2008, emerging non-volatile memory technologies (e.g., memristor) have revived the concept of Processing-In-Memory under the new name In-Memory Computing [ 43 ] or Computation-In-Memory [ 76 ]. The novel architectures together with new memory technologies promise a lot of potential in terms of area, performance, and energy-efficiency improvements [ 44 , 74 , 76 , 125 , 169 , 170 , 171 ]. In this regard, there are several works that can be labeled as In-memory computing, such as Computation-In-Memory (CIM) [ 51 , 60 ], ReVAMP [ 17 ], Pinatubo [ 117 ], and so on. All these architectures have common and distinct features, and these properties were addressed at least partly in the community. This leads to confusion in differentiating these architectures and hence, limiting the exploration potential of novel architectures.
For several decades, CMOS down-scaling and architecture improvements have doubled computer performance following Moore’s law [ 28 , 80 , 90 ]. However, existing technology suffers from three main walls: leakage wall, reliability wall, and cost wall [ 75 , 189 ], while computer architectures also face three walls: memory wall, power wall, and instruction-level parallelism (ILP) wall [ 62 , 152 , 192 ]. To address these walls, a lot of novel technologies and architectures are under research to improve the performance [ 89 , 123 , 141 ]. As a result, an enormous amount of architectures have been proposed so far. Therefore, a comprehensive survey on those architectures is needed to maintain a systematic view on these architectures.
In the following sections, we will discuss the existing architectures classified based on their computation position. Please note that each computation position-based classes are consisting of architectures implemented using different memory technologies.
Based on the above discussed metrics, 36 classes can be differentiated by combining computation location with memory technology as shown in Figure 1 (b); among those classes, 11 classes are occupied by the existing architectures, which are located in red and pink planes (see Figure 1 (b)). The red plane demonstrates that a lot of work has been done for that particular class. The pink plane demonstrates a moderate number of work has been done. The cyan plane demonstrates either unexplored classes due to the lack of attention from the research community or non-existing due to current restrictions of memory technologies. The developments in memory-centric computing are shown in the timeline of Figure 2 ; this shows the trend of computing moving from processor-centric to memory-centric architectures (CIM-A, and CIM-P). In the figure, a larger circle indicates that more work has been proposed in that year. As it can be seen from the figure, the concept of merging computation and memory was introduced back in 1970.
Computation parallelism defines the level of parallelism that can be exploited in a computer system; i.e., task-, data-, and/or instruction-level parallelism. In task-level parallelism, a system has multiple independent control units and data memories; examples include multi-threading [ 54 , 185 ] and multicore systems [ 69 ]. In data parallelism, a system has a single control unit used to apply the same instruction concurrently on a collection of data elements; examples include data elements with constant sizes (e.g., vector and array processor [ 52 , 61 ]) and varying sub-word sizes (e.g., SWAR (SIMD Within A Register) processor [ 154 ]). In instruction-level parallelism, a system has a single control unit used to execute various instructions concurrently; examples include intra-instruction (e.g., pipe-lined processor [ 184 ]) and inter-instruction (e.g., VLIW processor [ 198 ]) parallelism. Additionally, they can be combined together as in speculative processor [ 124 ].
The unique characteristics associated with each memory technology are illustrated in Table 1 . Besides to the regular memory operations, a range of in-memory logic and arithmetic operations can also be performed using these memory technologies [ 3 , 58 , 76 , 172 ].
PCRAM exploits the large resistance contrast between the amorphous and crystalline states of a chalcogenide glass. The change from a high resistive amorphous phase to less resistive crystalline phase is induced by heating the material above its crystallization temperature for a certain duration. The reverse switching can be realized by melting and quenching the material using a reset electrical pulse. Due to these switching mechanisms, PCRAM devices have the capability to achieve a number of intermediate distinct states, enabling the feature of multi-bit storage similar to RRAM.
In MRAM technologies, the value is stored in a Magnetic Tunnel Junction (MTJ) cell that consists of an oxide layer that is sandwiched between two ferromagnetic layers. Out of these two ferromagnetic layers, one is reference layer in which the magnetization is always fixed. The other one is known as free layer, whose magnetic orientation can be freely rotated, depending on the direction of current flowing through it. When the magnetic orientation of these two layers are parallel and anti-parallel to each other, the cell exhibits low and high resistance states, respectively. Reading a value from the MTJ cell applies the principle of Tunneling Magneto-Resistance effect.
RRAM, MRAM, and PCAM store information in the form of resistance states; these devices are thus also termed as memristors. These devices can be programmed in high resistance or low resistance states using reset or set electrical pulses. The RRAM cell consists of a top-electrode and a bottom-electrode that is separated by an oxide layer. Based on the formation/ disruption of a Conductive Filament (CF) , the resistive switching of RRAM devices takes place. The size of the CF determines the resistance state of the device. When a suitable positive voltage is applied, the breakage of ionic bonds increases the size of the CF, leading to a low resistance state of the device. However, when a suitable negative voltage is applied, some ions move back into the oxide region, thus reducing the size of the CF, resulting in a high resistance state. RRAMs are capable to perform multi-level bit storage.
The SRAM and DRAM memories are largely adopted by the semiconductor industries. Both of these memories are volatile in nature, which means they require power supply to maintain their state. A six transistor bit-cell design is commonly used in SRAM, whereas DRAM bit-cell comprises a capacitor and a transistor. Although SRAM has faster accesses, its bit-cell size is much larger and consumes more leakage than DRAM. Despite the fact that DRAM has a significant advantage in terms of density, it requires periodic refresh to retain its data. Due to their volatility, both of these memories are facing serious power dissipation problems. On the contrary, Flash is a non-volatile memory that uses a floating gate transistor that has a charge trapping mechanism. Since flash uses only a single transistor, its density is significantly higher than that of DRAM. However, it requires high voltage and considerably long duration to write a value. Moreover, it has limited endurance due to gate oxide degradation under strong electric field, meaning it can be only employed for applications where a few write operations are required.
Memory technologies can be classified as charge-based memories and non-charge-based memories. In charge-based memories such as Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM) , and Flash, information is stored through the presence of charge [ 67 , 121 , 122 , 134 ]. Whereas, the non-charge-based memories include different types of storage elements distinguished by their physical mechanism; these include resistive [ 16 , 23 , 59 , 159 , 196 ], “magnetic” memories [ 18 , 20 , 21 , 22 , 37 , 68 , 138 , 159 ] and other types of memories such as molecular memories [ 71 , 114 , 115 , 155 ] and mechanical memories [ 31 , 73 ]) that can currently not be used for computing and are not discussed further in this classification.
COM: In Computation-Out-of-Memory (COM) architectures, the computation is performed in the extra logic circuit available inside the memory SiP (noted as position 3 in Figure 1 (a)). If the computation is performed by off-memory computational cores (noted as position 4 in Figure 1 (a)), then the architecture is similar to the classical Von-Neumann architecture and hence, it is not discussed in this article, as the article focuses on memory-centric architectures.
CIM-P: In CIM Periphery (CIM-P) architectures, the computation result is produced within the peripheral circuitry (noted as position 2 in Figure 1 (a)). Typical examples of CIM-P architectures contain logical operations and vector-matrix multiplications [ 39 , 117 ]. CIM-P architectures typically contain dedicated peripheral circuits such as DACs and/or ADCs [ 63 , 164 ] and customized sense amplifiers [ 117 , 203 ]. Note that more radical changes in the peripheral circuit can be made in the future (e.g., changing in control voltages leads to radical changes in voltage drivers and sense amplifiers, or including a full functional processor inside memory banks). Even though the computational results are produced in the peripheral circuits for CIM-P, the memory array is a substantial component in the computations. As the peripheral circuits are modified, the currents/voltages applied to the memory array are typically different than in the conventional memory. Hence, similar to the CIM-A sub-class, the CIM-P architectures are also further divided into two groups: (1) basic CIM-P, where only change in the peripheral is required, which means the current levels should not be affected. An example of basic CIM-P is Pinatubo logic [ 117 ]; (2) hybrid CIM-P, where the majority of the changes take place in the peripheral circuit and minimal to medium changes in the memory array. An example of hybrid CIM-P is ISAAC [ 164 ]. ISAAC activates all rows of a memory array at the same time during read operations to perform a matrix vector multiplication using an ADC readout circuit. This architecture accumulates currents in the bitline that impose higher electrical loading in the memory array; hence, not only is the periphery circuit heavily modified but also the cell requires changes due to the high bitline current.
CIM-A: In CIM-A, the computation result is produced within the memory array (noted as position 1 in Figure 1 (a)). Note that this is different from a standard write operation. Typical examples of CIM-A architectures use memristive logic designs such as MAGIC and imply [ 104 , 108 ]. CIM-A architectures mostly require a modification at the cell-level to support such logic design, as the conventional memory cell dimensions and their embedding in the bit- and wordline structure do not allow them to be used for logic. In addition, modifications in the periphery are sometimes needed to support the changes in the cell. Therefore, CIM-A architectures can be further subdivided into two groups: (1) basic CIM-A, where only changes inside the memory array are required. An example of basic CIM-A is an architecture that performs computations using implication logic [ 112 ]; (2) hybrid CIM-A, where, in addition to major changes in the memory array, minimal to medium changes are required in the peripheral circuit. An example of hybrid CIM-A is an architecture that performs computations using MAGIC [ 108 ]. In this case, multiple memory rows are written simultaneously; due to the high write currents, modifications are required to the cell and medium changes in the peripheral circuits are needed to activate the multiple rows.
Computation position defines where the result of the computation is produced. A computation includes a primitive logic function (e.g., logical operations) or arithmetic operation (e.g., addition, multiplication). The possibilities of computation position can be seen at the four circles in Figure 1 (a). If the result is produced within the memory core (labeled as 1 in Figure 1 (a)), then the computer architecture is referred to as Computation-In-Memory Array (CIM-A) ; if the result is produced within the memory array periphery (labeled as 2 in Figure 1 (a)), then the architecture is referred to as Computation-In-Memory Periphery (CIM-P) ; otherwise, the result is produced outside the memory core (labels 3 and 4 in Figure 1 (a)) and the architecture is referred to as Computation-Out-of-Memory (COM) .
Evaluation and classification of modern computing systems is a complex process, as several metrics can be used to classify and evaluate computing systems [ 165 ]. Among the various metrics, performance, computing power, and resource utilization-based classification and evaluation of computing systems has been widely used in the literature [ 85 , 165 ]. However, classification of computing systems based on computation location and computing resource technology, such as memory technology, has not been explored so far due to the universal adoption of Von-Neumann architecture and CMOS technology dominance. Memory is the main storage unit in any computing platform, as shown in Figure 1 (a); it can include only memory core with memory arrays and its supporting peripheral circuits, or memory core with extra logic circuits, which is called memory System-in-Packages (SiP) . The computations are performed traditionally using computation cores, however, they can also be performed using extra logic circuits, peripheral circuits, and memory array of the memory SiP. With the abundance of Memory-centric architectures, it is becoming increasingly important to define efficient classification and evaluation metrics that can be used to analyze different memory-centric architectures. For this purpose, in our earlier paper, we developed two classification metrics, namely, computation position/location and memory technology. These classification metrics are summarized in the following subsections.
The architecture is simulated and evaluated using EPFL benchmarks [ 10 ] and compared against PLiM [ 65 ], which is based on a resistive memory with the same logic style. The compiler level of execution for such architecture can be performed as demonstrated in Reference [ 175 ].
The EX stage is complex, as it integrates control signals for memory and computations. Therefore, it is not easy to pipeline this architecture, as the EX stage will consume more time than the other stages; i.e., the stages IF, ID, and EX are not balanced.
The architecture has to deal with sneak path currents. Possible solutions to alleviate the problem consist of isolating each tile/crossbar, or using a transistor-memristor (1T1R) structure to actively control each memristor using a transistor [ 126 , 204 ], or using isolated/half select voltages [ 27 , 190 , 201 ].
The latency of majority primitive functions varies, depending on the functional complexity; in addition, before any operations are applied to the cells, these cells first have to be read-out to determine the appropriate control voltages.
However, it also has the following limitations:
The crossbar is based on only one device per cell, resulting in a more compact architecture as compared with other architectures that make use of two devices per cell (i.e., Complementary Resistive Switch (CRS) [ 167 ]).
The data transfer may include direct (within the crossbar based on copying resistance values) and indirect (based on read-out/write-back) schemes.
The IF and ID stages are similar to those of the traditional five-pipelined RISC architectures. The IF stage includes an Instruction Memory (IM) and a Program Counter (PC) . The ID stage contains registers (IR and Primary Inputs) and an Instruction Decode and Control Signal Generation. The EX stage consists of several registers (i.e., Data Memory Register (DMR), Primary Input Register (PIR), Mux control (M \( _c \)) register, Control (C \( _c \)) register, Wordline (W \( _c \)) register), as well as a crossbar interconnect, wordline select multiplexer, data Source Select multiplexer, and a Write circuit to control the crossbar that stores data. Once an instruction is fetched and decoded in IF and ID, respectively, the control registers in EX stage are filled with suitable values. These values control the multiplexers that are responsible for applying the right control signals to the crossbar. Depending on the operation, primary inputs from PIR or data retrieved from the crossbar stored in DMR can be used for the next operation. The crossbar interconnect permutes the inputs and control signals (indicated by C \( _c \)) to generate the voltages that need to be applied to the memory crossbar. The Write circuit applies these voltages to the targeted wordline address (indicated by W \( _c \)). In addition to the general advantages of CIM-A architectures, ReVAMP has the following advantages:
ReVAMP was proposed in 2017 by D. Bhattacharjeeet al. from Nanyang Technological University [ 17 ]. It is a memristor-based architecture that exploits data parallelism using majority logic. The architecture consists of an Instruction Fetch (IF), Instruction Decode (ID) , and Execute (EX) stage. The IF block fetches instructions from the Instruction Memory using the program counter (PC) as address and stores it in the Instruction Register (IR) . The ID block decodes the instruction and generates control signals that are placed in the control registers of the EX block. The EX stage finally executes the instruction.
The potential of the architecture is demonstrated by performing a logical bit-wise OR operation of two 8-bit vectors in 20 steps. The latest research has shown that MAGIC can be used for several arithmetic operations, such as addition, multiplication, and so on [ 72 , 87 , 109 ]. In this context, there is an architecture named SIMPLE MAGIC that automatically generates a defined sequence of atomic memristor-aided logic NOR operations [ 84 ]. In the same line, a SIMPLER flow is developed that additionally optimizes the mapping executions that reduce complexity and improve throughput [ 15 ]. Note, a lot of research on cell minimization and cell reuse has been done in the context of MAGIC and other basic cells.
The control voltages used in MAGIC have to satisfy the constraint \( 2V_{reset} \) \( \lt \) \( V_w \) \( \lt \) \( V_{set} \), where \( V_{set} \) is the minimum voltage required to switch a memristor from \( R_H \) to \( R_L \), and \( V_{eset} \) is the minimum voltage required to switch a memristor from \( R_L \) to \( R_H \); in other words, it requires that the memristors have a higher \( V_{set} \) than \( V_{reset} \), leading to an unbalanced hysteresis loop. This limits the types of memristors that can be used for MAGIC.
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3 .
In spite of the above-mentioned benefits, MPU also has its own set of limitations, which include:
MPU is compatible with the conventional Von-Neumann machines, meaning it can perform load/store/compute) operations.
MPU can perform many MAGIC gate operations in parallel, which makes it very efficient in terms of throughput.
The write energy of a single MAGIC cell is smaller in comparison to those of a CRS cell.
The MPU controller is simpler than for the CRS architecture, as each operation consists of a fixed number of steps while less controlling mechanisms are used.
The data transfer may include direct (based on copying) and indirect (based on read-out/write-back) schemes.
The MPU controller uses the Processor-In unit to divert the instructions to specific blocks (such as arithmetic, read and write blocks) responsible for the execution of those operations. Each block determines the appropriate voltages that have to be applied to the memristive memory. The set/reset, read and write block have a latency of 1 cycle, while the arithmetic block requires multiple cycles to execute a vector operations using MAGIC logic [ 108 ]. Data movements in the crossbar are performed directly using copy (double negation) or single NOT operations. In addition to the general advantages of CIM-A architectures, MPU has the following advantages:
MPU was proposed in 2016 by R. Ben Hur et al. from Technion-Israel Institue of Technology [ 83 ]. Similar to the DRAM-based architecture proposed in Reference [ 66 ] and SRAM-based solutions in References [ 1 , 53 , 64 , 130 , 156 , 194 ], MPU is a memristor-based architecture that exploits data-level parallelism using Memristive-Aid loGIC (MAGIC) [ 108 ]. The architecture consists of a conventional processor, MPU controller, and a memristive memory as shown in Figure 6 . The processor contains a control unit, an arithmetic and logic unit, and a memory controller. The MPU controller includes a Processor-In/Out block to interface to the conventional CPU, control blocks to execute specific commands (arithmetic, set/reset, read and write block) and an output multiplexer to apply appropriate voltages to specific rows/columns of the memristive memory. The conventional processor sends an instruction to the memristive memory using its own memory controller and the MPU controller. The memory controller of the processor recognizes the memristive memory instructions in a similar manner as conventional memory operations, while the MPU controller determines whether to treat the memristive memory as a storage element or a processing element. Based on that, the MPU controller applies read/write signals or a sequence of signals to perform logical or arithmetic operations.
The architecture is evaluated with a PRESENT Block Cipher algorithm [ 24 ], which encrypts a 64-bit plain text with an 80 or 128-bit key. The algorithm is compiled into a sequence of majority logic gates and executed on PLiM. Unfortunately, the results show that PLiM’s performance is almost a factor of two slower than a 180 nm FPGA implementation [ 24 ].
The LiM controller is complex, as it has to determine the control voltage values based on the operands’ values.
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3 .
The latency of majority primitive functions varies, depending on the functional complexity, and some read-outs are required to determine the voltage values to be applied.
Similarly, it also has its own limitations that are stated as follows:
The write energy and area of a memristor cell is smaller as compared to a CRS cell.
The LiM controller operates in two modes: conventional memory read/write mode and in-memory instruction mode. In the read/write mode, the FSM is deactivated, and the memory array is read or written in the same manner as a standard memory. In the in-memory instruction mode, FSM is activated, and an instruction is performed using majority logic gates inside the memory. Once the FSM is enabled, the following operations are performed: First, the FSM resets all registers in the LiM controller. Second, an instruction is read from the address in the program counter (PC) and decoded to obtain the addresses of the two operands and output; the addresses of the two operands are stored in registers @A and @B, while the output address is stored in register @Z. Third, the values of the two operands are read using the addresses in registers @A and @B; the obtained values are stored in registers @A and @B, respectively. Fourth, depending on the logic values (0 or 1) of the operands, appropriate voltages are applied to the crossbar at address @Z to perform a majority logic gate. Finally, the PC is incremented by one. On top of the general advantages of CIM-A architectures, PLiM has the following additional advantages:
Similar to the DRAM-based bit-serial addition using majority logic [ 5 ], PLiM was proposed in 2016 by P. Gaillardona et al. from EPFL [ 65 ]. It is a memristor-based architecture that exploits data parallelism using majority logic [ 166 ] to perform elementary Boolean logic such as OR and AND operations within the memory array. The architecture consists of a resistive memory organized in banks and a Logic-in-Memory (LiM) controller block as shown in Figure 5 . The memory is a memristive crossbar that stores both instruction and data. The LiM controller is composed of a number of registers and a finite state machine (FSM) . The controller functions as a simple processor; it fetches instructions from the memory array, decodes and executes the operation inside the memory.
The potentials of the architecture are demonstrated using a case study of a binary-tree-based parallel adder and multiplier [ 51 , 77 ]; the architecture is compared with a conventional multicore architecture. CIM architecture achieves at least one order of magnitude improvement in terms of delay, energy, and area.
In case CRS cells are used, the same drawbacks of CRS architecture apply, i.e., a larger cell area, the control unit imposes a high overhead, as the controllers are responsible for both controlling the crossbar and transferring data, and it requires additional compiling techniques and tools to convert conventional Boolean logic functions to implication logic.
In case FBL is used, typically a lot of cells are required due to LUT-based computing.
The architecture has to deal with sneak path currents in case a single RRAM cell (0T1R) is used, as multiple rows and columns are activated simultaneously. Possible solutions to alleviate the problem consist of isolating each FBL circuit, or the usage of a transistor-memristor (1T1R) structure to actively control each memristor using a transistor [ 126 , 204 ], or isolated/ half select voltages [ 27 , 201 ].
However, it also has the following limitations:
Compared to CRS architecture, CIM architecture has significant area and write energy advantage, as these values are much less at a single respective cell level.
The control block of FBL is less complex than the control block of implication logic due to a fixed number of write steps and a simpler control voltage scheme [ 202 ].
The data transfer using both direct and indirect schemes has been intensively explored in References [ 49 , 200 ].
In case FBL is used, the latency of primitive functions (i.e., addition, multiplication) is a constant number.
The architecture can accommodate any type of memristor logic design due to the flexibility of the control and communication block.
The architecture uses state machines stored in the control and communication block to compute and transfer data in the crossbar. Once triggered, the state machine applies an appropriate sequence of control voltages to the rows and columns of the memristor crossbar. Depending on the memristor types, the data transfer occurs directly inside the crossbar (for single RRAM cells) or indirectly through the control and communication block outside the crossbar (for CRS cells) using the CRS read-out/write-back scheme. In addition to the general advantages of CIM-A architectures, CIM comes with the following set of advantages:
CIM was proposed in 2015 by H. A. Du Nguyen et al. from Delft University of Technology [ 51 , 74 , 76 ]. It is a memristor-based architecture that exploits data-level parallelism by using any memristor logic style; the authors have showed the potential of this architecture using Fast Boolean Logic (FBL) [ 202 ] and implication logic [ 167 ]. The architecture consists of a memristor crossbar and a control and communication block as shown in Figure 4 [ 74 ]. The memristor crossbar stores data and performs computations. The control and communication block applies appropriate voltages to the memristor crossbar.
This architecture was only evaluated at circuit level using adders. Therefore, it is hard to make general conclusions on the performance and the applicability of this architecture.
The area of CRS cells is larger than those based on single memristor cells.
The control unit imposes a high overhead, as it is responsible for both controlling the crossbar (requiring a large number of states) and transferring data (which involves the usage of buffers/registers to store temporary values).
The data transfer method is indirect, as it is based on the read-out and write-back scheme. As all cells have high resistance, direct copying of cells in the crossbar is not applicable.
The RO operation is destructive, hence, a WB operation is required after each RO operation, which increases the latency and energy of computations.
The latency of the primitive functions varies and requires extra read-out instructions to determine the voltages that have to be applied.
However, it also has the following limitations:
It is possible to use sufficiently large crossbar arrays, meaning effective overhead due to control circuits can be minimized.
CRS logic requires fewer cells to perform computations than Fast Boolean Logic (FBL) , which is required to express the sum-of-product format.
It is less impacted by the sneak path currents due to the usage of CRS cells. The cell’s resistance is always equivalent to high resistance, hence, sneak path currents are eliminated. However, variations in resistances will make such paths practically unavoidable unless a 1T2R cell is used.
The crossbar is controlled by a sequence of operations including: write-in (WI), read-out (RO), write-back (WB) , and compute (CP) . Before the operations can be performed, the crossbar part used for computation is entirely reset to a logic value 0. The WI operation writes a logic value into a memristor. The RO operation reads a logic value from a cell; the logic output value is determined by the sense amplifier. The RO operation is destructive and changes the value of the memristor to logic value 1. The task of the WB operation is to recover the destroyed value. Finally, the CP instructions are used to execute the implication logic gates [ 118 , 167 ]. The data transfer between CRS cells is carried out through the control unit using a RO and WB operations; in other words, the control unit reads a value of the source CRS cell and writes it into the destination cell. In addition to the general advantages of CIM-A architectures, CRS has the following advantages:
Complementary Resistive Switch (CRS) architecture was proposed in 2014 by A. Siemon et al. from RWTH Aachen University [ 167 ]. It is a memristor-based architecture that exploits data-level parallelism using implication logic. The architecture shown in Figure 7 consists of multiple crossbars and a control unit. The crossbar stores and performs logic operations using CRS cells; a CRS cell consists of two resistive switches or resistive RAMs. The control unit distributes signals to the intended addresses (wordlines and bitlines) to perform operations on the crossbars.
The architecture is simulated and evaluated against GPU TITAN X [ 146 ] using four CNN applications, including 8-layer AlexNet [ 106 ], 16-layer VGG-16 [ 168 ], 19-layer VGG-19, and 152-layer ResNet-152 [ 79 ].
The architecture uses DRAM technology, which suffers from low performance, high energy consumption, large footprint, and is difficult to scale down.
The latency of complex functions varies, depending on the functional complexity, as each function needs to be converted into multiple NOR gates.
In spite of those advantages, DRISA-3T1C also has its own set of limitations that impact its effectiveness. The limitations of DRISA-3T1C include:
The architecture uses DRAM technology, which has several benefits, such as: high maturity and endurance, no sneak path currents, and the accessibility to optimized architectures, technology, and tools.
The controller is simpler than for the CRS architecture, as each operation consists of a fixed number of steps while fewer control voltage values are used.
The architecture does not suffer destructive read as in the case of CRS architecture [ 167 ], hence the write energy might be less due to the absence of write-after-read.
The memory mats consist of cell regions for both data and computations and peripheral circuits including calc-SA (Sense Amplifier) , intra/inter-lance shifter, and lane forwarding unit. The cell regions contains multiple DRAM cells that consist of three transistors connected to form a NOR gate and one capacitor to store the data value (see Figure 3 (d)). To perform computations, two DRAM cells (Rs and Rt) to be activated simultaneously and one DRAM cell (Rr) to store the computation result (as shown in Figure 3 ). Read voltages are applied to the sources DRAM cells (Rs and Rt) through the wordline (rWL), while write voltage is applied to the result DRAM cell (Rr). The voltage collected by the sense amplifier (SA) is used to control the transistor in the Rr DRAM cell as shown in Figure 3 (d). Due to the NOR organization of these transistors, a NOR operation is realized and produces results in DRAM cells. The SA (also called calc-SA) cooperates with extra logic circuitry such as SHF and FWD to perform complex functions such as addition, copy, and inner product. DRISA-3T1C has the following advantages on top of the general advantages of CIM-A architectures:
DRISA-3T1C was proposed in 2016 by S. Li et al. from University of California [ 116 ]. It is a DRAM-based architecture that exploits data parallelism by performing NOR gate inside DRAM cells [ 166 ]. The architecture consists of a DRAM memory organized in a hierarchy of banks, sub-arrays, and mat; each levels is controlled by their corresponding controllers as shown in Figures 3 (a), (b), and (c). The banks are connected through global bus (gbus) , while communication among subarrays is carried out using bank buffers (bBuf) . The mats perform both data storage and computations.
The following subsections discuss the details and characteristics of each architecture. In this subsection, DRAM-based architecture is presented first, and the remaining RRAM-based architectures are discussed in a chronological order of their date of publication. This ordering technique is also reflected in Table 2 .
The cell designs are mostly modified to make the computing feasible.
Higher endurance requirement due to the need of multiple write steps to perform Boolean functions.
High computing latency per access due to the high latency of writing memristors and the need of multiple write steps to perform Boolean functions. Note that despite a high computing latency, the performance can be still high when sufficient parallelism is exploited.
On the other hand, they all share several limitations:
Low leakage due to the usage of non-volatile memory technology and small footprint when compared to conventional memory technologies, but only in the case of very large memory array.
On one hand, all these architectures have several common advantages:
The CIM-A class contains mostly resistive computing architectures that use memristive-based logic circuits [ 50 ] to perform computations and resistive RAM (RRAM) as memory technology. Few architectures have been proposed in this category. Table 2 shows a brief comparison among the architectures that will be explained in each subsection.
Skip 4COMPUTATION-IN-MEMORY - PERIPHERALS (CIM-P) Section4
COMPUTATION-IN-MEMORY - PERIPHERALS (CIM-P)The CIM-P class consists of architectures that perform computations during read-out operations (i.e., two or more word lines are activated simultaneously) using special peripheral circuitry. Earlier memory-centric works explored the potential of shifting data-intensive computations to the memory system with/without reconfigurable logic includes Active Pages [26, 95, 149, 180]. These works utilized conventional DRAM and magnetic bubble memories to realize memory-centric architecture. However, as there are less restrictions on the functionality of the cell, various memory technologies can be used in this category, such as DRAM, SRAM, and non-volatile memory technologies. For instance, a medium number of architectures have been proposed in this category. Table 3 shows a brief comparison among the architectures that will be explained in each subsection.
Table 3.
Table 3. Comparison among Architectures of CIM-P Classes
On one hand, these architectures have several common advantages:
Low memory access/bandwidth bottleneck, as the results are produced in the peripheral circuitry that is connected directly to the memory array.
High parallelism due to the the possibility of performing multiple concurrent operations.
High performance, as computations are performed in a single read step.
Relatively simple controllers, as the operations are constructed in a similar manner as for conventional memory (read/write) operations.
Higher compatibility with available memory technologies, because redesigning cells would induce a huge cost for the vendors.
Lower endurance requirement, as operations are based on reading instead of writing [203].
On the other hand, they all share the following limitations:
Overhead to align data; note that each operation requires the data to be aligned in the memory. Therefore, if the operands are not located in the same crossbar, then data transfer operations are required.
Additional write overhead when the results have to be stored back into the memory. Note that the outputs are produced as voltages in the peripheral circuit, and therefore, if the results have to be stored back in the memory, then extra write operations would be necessary.
Parallelism is possible, but it is achieved at the cost of area and power overhead.
Similar to CIM-A architectures, the following subsections discuss the details and characteristics of each CIM-P architecture. In this section, architectures that utilize charge-based memory technology (DRAM and SRAM) are presented first, followed by the discussion of architectures based on non-charge-based memory technology (e.g., Resistive, Magnetic, etc.). Both charge and non-charge-based memory technology architectures are discussed in chronological order of their date of publication. This ordering technique is also reflected in Table 3.
4.1
Ambit: In-memory Accelerator for Bulk Bitwise Operations Using Commodity DRAM TechnologyAmbit was proposed in 2017 by V. Seshadri et al. from Carnegie Mellon University [163]. Ambit is a DRAM-based architecture that performs in-memory instructions using modified peripheral circuits to exploit data-level parallelism, which is achieved by computing all bits in a row(s) in parallel. This architecture can be plugged into a computer system as an accelerator in a similar manner as a GPU. The architecture consists of an Ambit controller and a 3D-stacked DRAM memory with modified sense amplifiers as shown in Figure 8. After receiving an instruction from the host processor, Ambit determines whether a normal memory operation or an in-memory instruction should be performed. After performing the required operations, the results are transferred back to the host processor for further processing. Depending on the type of operation, the Ambit controller activates single or multiple rows in the DRAM memory. AMBIT architecture can perform any bit-wise operations at column level using basic set of operations such as AND, OR, and NOT. The currents are summed up based on the values stored in the DRAM cells and converted into a digital value using the modified sense amplifiers. To transfer data, Ambit enables row copy (RowClone [162]) operations to directly move data inside DRAM memory. Moreover, an indirect scheme can be used as well by having the Ambit controller performing read and write operations. In addition to higher throughput, Ambit has the following advantages on top of the generic CIM-P architecture advantages:
Fig. 8.
Fig. 8. In-memory Accelerator for Bulk Bitwise Operations Using Commodity DRAM Technology (Ambit) [163].
Data can be transferred in the memory using both direct and indirect schemes.
The architecture uses DRAM technology, which has several benefits, such as: high maturity and endurance, no sneak path currents, and the accessibility to optimized architectures, technology, and tools.
Since the pitch of the SRAM bit-cell is more, it can easily accommodate the modified version of the sense amplifier in a column.
However, it also has the following limitations:
Computations are currently limited to logical operations. More research is required to map complex functions on the architecture.
The architecture uses DRAM technology, which suffers from low performance, high energy consumption, large footprint, and is difficult to scale down.
The architecture is simulated by Rambus simulator and evaluated against the implementations on multicore Intel Skylake CPU [88], NVIDIA GeForce GTX 745 GPU [145], and HMC 2.0 [127] using logical vector operations and bitmap index application [34, 177]. The simulation results demonstrated that Ambit can deliver higher throughput than Skylake, GTX, and HMC 2.0 architectures.
4.2
S-AP: Cache AutomatonS-AP shown in Figure 9 was proposed in 2017 by A. Subramaniyan et al. from University of Michigan [181]. The architecture targets an automata processor that exploits data-level parallelism by performing computations using state machines. An automata processor contains two main components: the State Transition Elements (STEs) and the routing matrix; the STE stores the accepting states, while the routing matrix stores the state transitions as shown in Figure 10. The automata processor accepts one input symbol at a time, generates next active states, and decides whether a complete input string is accepted or not.
Fig. 9.
Fig. 9. SRAM Automata Processor(S-AP) [181].
Fig. 10.
Fig. 10. General architecture for automata processor [206].
The architecture consists of STEs and a routing matrix that are implemented using SRAM technology. Each SRAM column corresponds to an STE that stores the accepting states in SRAM cells. The input symbol is fed to all the STEs simultaneously. The sense amplifiers collect a dot-product results of a vector-matrix multiplication. The output of the STE together with the routing matrix are used to determine the next active states; this process is carried on until all input symbols are processed. In case the one or more final active states are part of the acceptance states, it means that the input string has been matched with the corresponding pattern of the acceptance state. Note that data transfer inside the automata processor is carried out using the routing matrix. In addition to the general advantages of CIM-P architectures, S-AP has the following advantages:
Computations may include logical and arithmetic operations using automata processing.
Data can be transferred using both direct and indirect schemes.
The architecture uses SRAM technology, which has several benefits, such as maturity, high endurance, no sneak path currents, and may benefit for the the existing optimizing techniques and tools.
Since the pitch of the SRAM bit-cell is more, it can easily accommodate the modified version of the sense amplifier in a column.
The automata processing techniques and tooling are quite mature, hence it is feasible to explore many applications using automata processing.
However, it also has the following limitations:
The architecture uses SRAM technology, which suffers from high energy consumption, low scalability, and large footprint.
The architecture requires additional compiling techniques and tools to perform conventional Boolean logic functions using automata processing.
On one hand, S-AP has potential in reducing non-memory components required to implement the automata processor. The D-AP requires most of its resources for routing matrix and other logic, while the S-AP can be implemented on processor, which has advantages in realizing logic functions. On the other hand, S-AP suffers from low frequency, density, and latency due to SRAM intrinsic properties. The S-AP is simulated using VASim [191] and evaluated against DRAM-AP and x86 CPU using ANMLZoo [191] and the Regex [13] benchmark suites.
4.3
ReAP: Resistive Associative ProcessorReAP was proposed in 2014 by L. Yavits et al. from Technion-Israel Institute of Technology [205]. ReAP is a RRAM-based architecture that exploits data parallelism using LUTs implemented with Contend Addressable Memories (CAMs) to perform computations. The architecture consists of a crossbar of resistive CAM cells and peripheral circuits including sense amplifiers and registers (as shown in Figure 11). In this architecture, multiple CAM cells are employed to create a look-up-table (LUTs); together, they implement a specific logic function. A single CAM cell comprises two resistive RAM cells that store the true and the complementary value of a single bit. When it is activated, it performs a compare operation with the inputs stored in register KEY to the LUT words and produce in case a match occurs the corresponding outputs in TAG registers.
Fig. 11.
Fig. 11. Resistive Associative Processor (ReAP) [205].
The compare operation is performed in a similar manner as in conventional CAMs. First, the Match/Word line is pre-charged. Thereafter, the values in KEY are applied to the bit-lines, depending on the MASK value; if a bit is masked, then it is kept floating. If the KEY matches the content on a particular wordline, then the TAG will generate the value “1” at the output, otherwise “0.” For example, in case a key bit is 1, both the true (i.e., low resistance) and complement value (i.e., high resistance) will keep the floating word line high in case a 1 is stored. In case the cell holds a 0, i.e., the true memristor has a high resistance and complement a low resistance value, the complement path will discharge the Match/Word line. Similar conclusions can be drawn in case the key bit is 0. To execute a more complex function, LUTs can be reconfigured. In such cases, the output of the LUT is fed back to input of the same LUT but with a different configuration. Another option is to implement the function using multiple LUTs. ReAP architecture has additional advantages on top of the generic CIM-P architecture advantages. The incremental advantages of ReAP are summarized as follows:
The architecture is used as an accelerator, which has a positive impact on the endurance due to infrequent use [35, 193]. In contrast, some CIM-P architectures are used as main memory and they require a much higher endurance.
Computing based on LUTs is quite mature (e.g., in FPGAs) and can benefit from existing techniques and tools.
The architecture uses non-volatile memory, hence consumes a low amount of energy and has a small footprint.
However, it should also be noted that ReAP has several limitations that can severely affect its applicability. The limitations of ReAP include:
Computations using LUTs can be inefficient if the number of inputs per LUT is large. If multiple smaller LUTs are used, then the latency becomes higher.
The data transfer consists of an indirect read-out scheme.
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
The write operations of this architecture may suffer from high energy consumption, as two memristors are written per CAM cell.
The architecture might not exploit the full memory bandwidth, as it is challenging to fit all sense amplifiers into the memory core.
The architecture requires additional compiling techniques and tools to convert conventional Boolean logic functions to CAM-based LUTs.
The architecture was evaluated analytically using several benchmarks [40] such as N-pairs Black-Scholes option pricing, N-point Fast Fourier Transform, and Dense Matrix Multiplications. They compared the results of these benchmarks on ReAP with two other platforms: a CMOS equivalent of ReAP denoted by CMOS-AP and GTX480 GPU. The results show that ReAP outperforms GTX480 in terms of performance (8 GFLOPs/W for ReAP versus 5 GLOPs/W for GTX480), while it is being outperformed by CMOS AP (18 GFLOPs/W).
4.4
Pinatubo: A Processing-in-Memory Architecture for Bulk Bitwise OperationsPinatubo was proposed in 2016 by S. Li et al. from University of California [117]. Pinatubo is a non-volatile memory-based architecture that exploits data-level parallelism by performing bulk bitwise operations using modified sense amplifiers. The architecture consists of a processor with caches, a non-volatile main memory, and modified sense amplifiers (as shown in Figure 12). The processor sends in-memory instructions to the main memory and also handles the operations that cannot be performed on the main memory. After an instruction is sent to the main memory, single or multiple rows of the memory are activated simultaneously, depending on the type of instructions (i.e., normal read or in-memory instructions). The modified sense amplifiers thereafter perform a read-out operation to produce the results, which can be a normal read or a bitwise vector operation. In case needed, the results are transferred back to the processor for further processing.
Fig. 12.
Fig. 12. The Processing-in-Memory architecture for bulk bitwise operations (Pinatubo) [117].
The main memory architecture is shown in Figure 13; it consists of multiple banks that are further divided into mats. Note that the modified sense amplifiers can only perform bitwise vector operations on data residing in the same mat. For operations where the data resides in different mats whether on the same bank or not, extra logic gates (e.g., AND, OR) are used to perform the operations. Communication can be performed by enabling two memory rows for direct copy operations or using the buffers and read-out operations for indirect data transfer. In addition to the general advantages of CIM-P architectures, Pinatubo has the following advantages:
Fig. 13.
Fig. 13. Main memory of Pinatubo [117].
Data can be transferred in the memory using both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes low energy and has a small footprint.
However, Pinatubo has its own limitations, which are listed as follows:
The architecture uses non-volatile memory as main memory, which may impact the lifetime due to limited endurance [35, 193].
Computations currently include only logical operations. More research is required to map complex functions on the architecture.
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
Efforts are required to use this architecture with a host processor. For example, the instruction set of the host processor has to be adapted and additional software support is needed to maximally exploit the performance.
The architecture is simulated using an in-house cycle accurate simulator modified from Sniper [32] and evaluated using three applications: vector OR operations, bitmap-based BFS for graph processing [12], and bitmap-based database using Fastbit [199].
4.5
ISAAC: A Convolutional Neural Network Accelerator with In Situ Analog ArithmeticISAAC was proposed in 2016 by Ali Shafiee et al. from University of Utah [164]. ISAAC is a memristor-based architecture that performs dot-product computations using the memristor crossbar and CMOS peripheral circuitry to exploit instruction-level parallelism. The architecture consists of multiple tiles connected through an on-chip concentrated mesh and an I/O interface, as shown in the left part of Figure 14. The architecture is only used during the inference phase of machine learning applications, i.e., the phase after training; the inference phase consists of dot product operations to compute convolutions, shift and add operations, and sigmoid operations. ISAAC processes inputs from the I/O interface in multiple tiles. After processing, the outputs are communicated through the I/O interface to the outside world or a different ISAAC chip. Each tile of ISAAC contains multiple In Situ Multiply Accumulate (IMA) units that are connected through a bus, an eDRAM buffer, output register (OR), and computation units (max-pool, sigmoid, and Shift-and-Add (S+A)). Each IMA contains multiple memristor arrays with their DAC and Sample-and-Hold (S+H) units, an Input and Output Register (IR, OR), S+A, and multiple ADC units. Inputs from the I/O interface are delivered to the memristor arrays and are used to perform a dot product computation with the weights that are already stored in the memristor array. The results thereafter go through the S+H units (to temporarily store data before feeding them to ADCs) and S+A units (to accumulate data) if applicable. Finally, if multiple inputs are fetched, then a pipeline is created using IR and OR of the IMAs and tiles. To transfer data within a single memory array, a controller can be used to apply appropriate voltages to move data directly inside the memory crossbar or use read-out and write-back schemes. The additional merits of ISAAC over the generic CIM-P architecture benefits can be summarized as follows:
Fig. 14.
Fig. 14. A convolutional neural network accelerator with In Situ Analog Arithmetic (ISAAC) [164].
The architecture is used as an accelerator, which has a positive impact on the endurance due to infrequent use [35, 193]. In contrast, some CIM-P architectures are used as main memory and therefore require a much higher endurance.
The computations for neural networks are quite mature and can benefit from existing neural network techniques and tools.
The computations for neural networks do not require a high precision; hence, they are more resilient against device variation.
Data can be transferred in the crossbar using both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes low energy and has a small footprint.
However, ISAAC also has different limitations that need to be addressed properly. Some of the limitations are:
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
The architecture might suffer from a high overhead due to the need of ADC and DAC converters.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
In case general purpose computing is desired, the architecture requires additional compiling techniques and tools to perform conventional Boolean logic functions using neural network computations.
The architecture is evaluated analytically and compared against DaDianNao architecture (which is an ASIC design with embedded DRAM) using a suite of CNN [78, 168] and DNN workloads [86, 183]. Their analytical result demonstrated that it potentially outperforms DaDianNao in terms of throughput, energy, and computational density. NEWTON introduced techniques to improve the energy efficiency by adapting ADC precision to improve the energy efficiency by adapting sub-block based ADC precision as well as an algorithm to reduce computations [136].
4.6
PRIME: A Processing-in-Memory Architecture for Neural Network Computation in ReRAM-based Main MemoryPRIME was proposed in 2016 by C. Pinga et al. from University of California [39]. PRIME is a resistive RAM-based architecture that exploits data-level parallelism to perform computations for neural networks (i.e., weighted vector-matrix multiplication) using high-precision multi-level sense amplifiers and some extra logic circuits. The architecture consists of a CPU and multiple RRAM banks; each RRAM bank contains multiple memory crossbars (mem subarrays), full function (FF), and buffer subarrays, as shown in Figure 15(a). The CPU sends instructions to the resistive RAM banks; an instruction is either a memory operation (read/write) or a neural network computation. The memory bank performs the request without blocking the CPU, i.e., the CPU continues executing (different) instructions simultaneously. The results are returned to the CPU for further processing.
Fig. 15.
Fig. 15. A Processing-in-Memory architecture for neural network computation (PRIME) [39] (a) PRIME banks organization, (b) architecture of PRIME bank.
In the resistive RAM banks, the memory crossbars store data in multiple mats, while the FF and buffer subarrays serve for computation. Special subarray structures are used to enable both neural network computations and memory operations (blue blocks in Figure 15(b)) feasible. The neural network computations are mainly performed in the FF subarray, while the buffer subarray stores temporary data that needs to be processed; this enables a parallel execution between CPU and FF subarrays. Neural network computations are performed using a vector matrix multiplication between a weighted matrix stored in the FF subarray and a vector stored in the buffer subarray. Additional logic gates such as subtraction and sigmoid units are used to compute negative weights and sigmoid activation functions before the results are sensed by the multi-level sense amplifiers. To communicate between the memories, a controller can be used to apply appropriate voltages to the crossbar to move data directly inside it or use read-out and write-back schemes. On top of the generic advantages of CIM-P architectures, PRIME has the following set of unique advantages:
The computations for neural networks are quite mature and can benefit from existing neural network techniques and tools.
The computations for neural networks do not require a high precision; hence, they are more resilient against device variations.
Data can be transferred in the crossbar using both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes a low energy and has a small footprint.
However, the above-mentioned merits of PRIME architecture also come with their own challenges and limitations. The limitations of PRIME include:
The architecture uses non-volatile memory as main memory, which may impact the lifetime due to limited endurance [35, 193].
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
In case general purpose computing is desired, the architecture requires additional compiling techniques and tools to perform conventional Boolean logic functions using neural network computations.
The architecture is synthesized using TSMC CMOS library 60 nm and modeled using NVSIM, CACTI-3D and CACTI-IO. It is evaluated using MlBench benchmarks [113] and compared against a CPU-only solution. Their comparison shows that the architecture achieves significant improvements in terms of performance and energy consumption over CPU-only-based solution.
4.7
ReGP: Resistive GP-SIMDReGP was proposed in 2016 by A. Morad et al. from Technion-Israel Institute of Technology [131]. ReGP is a RRAM memory-based architecture that exploits data parallelism by attaching a SIMD-like processing unit to the resistive memory. The architecture consists of a sequential or conventional processor, its L1 and LLC cache, shared memory array, and SIMD processor. The sequential processor executes traditional code and controls the SIMD processor in a master-slave mode. The SIMD processor executes parallel instructions on the data stored in the shared memory array.
The SIMD processor contains multiple processing units (PUs), a sequencer, and a Network on Chip (NoC) with reduction tree. Each PU contains registers, a single bit full-adder, and a function generator to perform arithmetic and logical operations. The sequencer receives instructions from the sequential processor and assigns them to PUs. The PUs load data from the shared memory array and perform the requested operations. If required, the NoC and reduction trees are used to perform more complex functions. In addition to the general advantages of CIM-P architectures, ReGP comes with the following advantages:
The parallelism is high due to multiple parallel processing units.
The architecture uses non-volatile memory, hence consumes low amount of energy and has a small footprint.
The architecture can reuse compilers, programming languages, and tools from SIMD architectures.
However, the operations within the processing units are simple; complex functions such as floating point operations can cause a high overhead. Thus, scalability is the main limitation of ReGP architecture.
The architecture is evaluated analytically against CMOS GP-SIMD [130] using the dense matrix multiplication benchmark [129].
4.8
CIMA: Computation-in-Memory AcceleratorCIMA was proposed in 2017 by H. A. Du Nguyen et al. from Delft University of Technology [50]. CIMA is a resistive-based accelerator that exploits data-level parallelism by performing computations with custom sense amplifiers. The architecture consists of a conventional processor, caches, CIM accelerator, main memory DRAM, and external memory (as shown in Figure 16). The processor fetches, decodes, and executes non-intensive memory parts of an application and off-loads the memory intensive parts to the CIM accelerator. Similarly as in Pinatubo, the CIM accelerator performs operations by activating one or multiple wordlines. The difference with Pinatubo, however, is that CIMA is used as an accelerator and has more efficient sense amplifiers. In case needed, the results are transferred back to the processor for further processing.
Fig. 16.
Fig. 16. Computation-in-Memory Accelerator (CIMA) [50].
The CIM accelerator consists of a CIM controller, peripheral circuits (including decoder, voltage driver, and sense amplifiers), and a memristor crossbar. The CIM controller receives instructions from the processor and performs operations by sensing the current of two or more activated rows of the crossbar. Based on the type of operation, the customized sense amplifiers compute the output based on this current. CIMA uses scouting logic [203]; currently, it can only perform bitwise logical operations. However, more operations such as addition, vector-matrix multiplication, and matrix-matrix multiplication have demonstrated to be feasible [128]. The data transfer in the memory can be performed by enabling two memory rows using direct copy operations or using indirect read-out and write-back operations. In addition to the general advantages of CIM-P architectures, CIMA has the following advantages:
The architecture is used as an accelerator, which has a positive impact on the endurance due to infrequent use [35, 193].
Data can be transferred in the memory using both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes low energy and has a small footprint.
However, it also has the following limitations:
Computations currently include only logical operations. More research is required to map complex functions on the architecture.
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
Additional software support (i.e., profiling and extracting memory intensive kernels) is required to maximally exploit the accelerator performance.
The architecture is evaluated against a theoretical model of a conventional multicore architecture.
4.9
STT-CiM: Computing in Memory Spin-Transfer Torque Magnetic RAMSTT-CiM was proposed in 2017 by S. Jain et al. from Purdue University [92]. STT-CiM is a Spin-Transfer Torque Magnetic RAM-based architecture that exploits data-level parallelism by performing computations using both modified sense amplifiers and some additional CMOS gates. The architecture consists of a conventional architecture with an STT-MRAM used as a scratch-pad memory. This scratch-pad memory is equipped with the capability to perform in-memory instructions. These instructions are sent from the main processor.
The STT-CiM contains a CiM decoder, an array of memory cells, enhanced address decoder, and modified sensing circuitry to perform computations, as shown in Figure 17. Based on the in-memory instruction, the enhanced address decoder activates one (for normal read) or multiple rows (for computations) of the memory array. The CiM decoder determines simultaneously the reference currents of the sense amplifiers. For example, in case an addition is executed, the set of logic gates for addition is enabled. The results are captured by the modified sense amplifiers. Data transfer can be performed by enabling two memory rows for direct copy operations or using the buffers and read-out operations for indirect copy operations. In addition to the general advantages of CIM-P architectures, STT-CiM has the following advantages:
Fig. 17.
Fig. 17. Computing in Memory Spin-Transfer Torque Magnetic RAM (STT-CiM) [92].
The architecture is used as an accelerator (i.e., scratch-pad memory), which has a positive impact on the endurance due to infrequent use [35, 193].
Computations currently include both logical operations and addition.
The data transfer may include both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes low energy and has a small footprint.
However, it also has the following limitations:
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
Additional software support (i.e., profiling and extracting memory intensive kernels) is required to maximally exploit the accelerator performance.
The architecture is evaluated using the STT-CiM device-to-architecture evaluation framework [92] and a set of benchmarks including string matching, text processing, low-level graphics, data compression, bio-informatic, image processing, and cryptography.
4.10
DPP: Data Parallel ProcessorDPP was proposed in 2018 by D. Fujiki et al. from University of Michigan [63]. DPP is a RRAM-based architecture that exploits instruction and data-level parallelism by performing computations using a combination of RRAM-based dot-product operations and LUTs. The architecture consists of multiple RRAM tiles connected as an H-tree; each tile has multiple clusters and some logic units (as shown in Figure 18). Tiles and clusters form a SIMD-like processor that performs the parallel operations. The architecture is considered as a general purposed architecture, as it can perform all primitive functions such as logical, arithmetic, shift, and copy operations.
Fig. 18.
Fig. 18. Data Parallel Processor (DPP) [63].
In addition to clusters, each tile has several units to support the computations, including instruction buffer, Shift and Add (S+A), and router. Each cluster additionally has one or more computational units; they are S+A, Sample and Hold (S+H), DAC and ADC, a LUT and register file (as shown in the right part of Figure 18). While reading from the high latency RRAM, other units are simultaneously used for processing. Therefore, the S+H is used to read data (in the form of a current) from the RRAM array and temporarily store it. Once that data is needed, it is fed to an ADC to convert the analog value to a digital value. The S+A is used to perform carry propagation in a multiple-bit addition. DAC is used to apply a digital value to the RRAM array with an appropriate control voltage. Some complex functions that cannot be realized with these units are performed using LUTs and register file in each cluster. Data transfer can be performed by enabling two memory rows for direct copy operations or using the buffers and read-out operations for indirect copy operations. In addition to the general advantages of CIM-P architectures, DPP has the following advantages:
Computations include both logical operations and simple arithmetic operations (e.g., addition, multiplication).
Data can be transferred using both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes low energy and has a small footprint.
This architecture is claimed to be general purpose, hence it can exploit the existing instruction set, compiling techniques, and tools, as well as applications.
However, it also has the following limitations:
The architecture uses non-volatile memory as main memory, which may impact the lifetime due to limited endurance [35, 193].
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
The architecture potential was simulated and evaluated against CPU Intel Xeon E5-2697 using a subset of PARSEC benchmarks [19] and against GPU NVIDIA Titan XP using Rodinia benchmarks [36].
4.11
R-AP: Resistive RAM Automata ProcessorR-AP was proposed in 2018 by J. Yu et al. from Delft University of Technology [206]. R-AP is an automata processor that exploits data-level parallelism by performing computations similarly as mentioned in Section 4.2. The working principle of R-AP is similar to the S-AP. In contrast to S-AP, R-AP uses RRAM-based STEs and routing matrice, as shown in Figure 19. In addition to the general advantages of CIM-P architectures, R-AP has the following advantages:
Fig. 19.
Fig. 19. Resistive RAM Automata Processor (R-AP) [206].
The architecture is used as a read-favored accelerator, which has a positive impact on the endurance due to infrequent use [35, 193].
Automata processing can be used to perform both logical and arithmetic operations in general.
Data can be transferred using both direct and indirect schemes.
The architecture uses non-volatile memory, hence consumes low energy and has a small footprint.
The automata processing techniques and tooling are quite mature, hence it is feasible to explore many applications using automata processing.
However, it also has the following limitations:
The architecture has to deal with sneak path currents. Possible solutions are mentioned in Section 3.3.
As the sense amplifiers are complex, a tradeoff between area and bandwidth has to be made.
The architecture requires additional compiling techniques and tools to perform conventional Boolean logic functions using automata processing.
The architecture has been validated using circuit-level simulations and evaluated against S-AP.