{"id":15163,"date":"2024-07-09T09:45:07","date_gmt":"2024-07-09T06:45:07","guid":{"rendered":"https:\/\/www.weebit-nano.com\/?p=15163"},"modified":"2025-06-25T14:09:53","modified_gmt":"2025-06-25T11:09:53","slug":"innovative-memory-architectures-for-ai","status":"publish","type":"post","link":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/","title":{"rendered":"Innovative Memory Architectures for AI"},"content":{"rendered":"

\"\"<\/p>\n

 <\/p>\n

One of the biggest trends in the industry today is the shift towards AI computing at the edge. For many years the expectation was that the huge datacenters on the cloud would be the ones performing all the AI tasks, and the edge devices would only collect the raw data and send it to the cloud, potentially receiving the end directives after the analysis was done.<\/p>\n

 <\/p>\n

More recently, however, it has become more and more evident that this can\u2019t work. While the learning task is a strong fit for the cloud, performing inference on the cloud is less optimal.<\/p>\n

With the promise of lower latency, lower power and better security, we are seeing AI inference in a growing number of edge applications, from IoT and smart home devices all the way up to critical applications<\/a> like automotive, medical, and aerospace and defense.<\/p>\n

 <\/p>\n

Since edge devices are often small, battery-powered, and resource-constrained, edge AI computing resources must be low-power, high-performance, and low-cost. This is a challenge considering power-hungry AI workloads, which must rely on the storage of large amounts of data in memory and the ability to quickly access it. Some models have millions of parameters (e.g., weights and biases), which must be continually read from memory for processing. This creates a fundamental challenge in terms of power consumption and latency in computing hardware.<\/p>\n

 <\/p>\n

Data movement is a key contributor to power consumption. Within chips, significant power is consumed while accessing the memory arrays in which the data is stored and while transferring the data over the on-chip<\/a> interconnect. The memory access and speed of the interconnect also contribute to latency, which limits the speed of the AI computation. Speed and power both get significantly worse when the data needs to be moved between two separate chips.<\/p>\n

To keep edge computing resources low-power and low-latency, hardware must be designed so that memory is as close as possible to the computing resources.<\/p>\n

 <\/p>\n

The continuous move to smaller process geometries has helped to keep power consumption to a minimum and has also reduced latency for AI tasks. But while computing resources continually scale to more advanced nodes, Flash<\/a> memory hasn\u2019t been able to keep pace. Because of this, it isn\u2019t possible to integrate Flash and an AI inference engine in a single SoC<\/a> at 28nm and below for edge AI.<\/p>\n

 <\/p>\n

Today it\u2019s standard practice to implement a two-die solution, where one die at an advanced process node is used for computing, and another die at a more mature process node is used for memory. The two dies are then integrated in a single package or in two separate packages. A two-die solution is detrimental to AI performance because memory resides far away from compute, creating high levels of power consumption, latency and total system costs.<\/p>\n

 <\/p>\n

The ideal solution is a single die that hosts memory and compute, and embedded ReRAM<\/a> (or RRAM<\/a>) is the logical NVM<\/a> to use. Embedding ReRAM into an AI SoC would replace off-chip flash devices, and it can also be used to replace the large on-chip SRAM<\/a> to store the AI weights and CPU<\/a> firmware. Because ReRAM is non-volatile, there is no need to wait at boot time to load the AI model from external NVM.<\/p>\n

\"\"<\/p>\n

 <\/p>\n

Such ReRAM-based chips use less power and have lower latency than two-chip solutions. And, with a wider path to memory enabled by the width of the interface, the memory interface is no longer limited by the number of pins on the memory device. The result is faster access time, faster inference, and the potential for true real-time AI computing at the edge.<\/p>\n

 <\/p>\n

ReRAM is also much denser than SRAM which makes it less expensive than SRAM per bit<\/a>, so more memory can be integrated on-chip to support larger neural networks for the same die<\/a> size and cost. While on-chip SRAM will still be used for data storage, the array will be smaller and the total solution more cost-effective.<\/p>\n

 <\/p>\n

Finally, ReRAM-enabled chipsets are cheaper to manufacture, since they only require the fabrication of one die. This makes edge computing more affordable and consequently more accessible for a large array of applications<\/a>.<\/p>\n

\"\"<\/p>\n

Above: how the various memory technologies compare<\/em><\/p>\n

 <\/p>\n

You can see here<\/a> the slides on this topic I presented at the Design Automation Conference (DAC<\/a>) 2024.<\/p>\n","protected":false},"excerpt":{"rendered":"

  One of the biggest trends in the industry today is the shift towards AI computing at the edge. For many years the expectation was that the huge datacenters on the cloud would be the ones performing all the AI tasks, and the edge devices would only collect the raw data and send it to […]<\/p>\n","protected":false},"author":29,"featured_media":15162,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[56,1,64],"tags":[],"class_list":["post-15163","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-general","category-tech-insight"],"acf":[],"yoast_head":"\nInnovative Memory Architectures for AI | Weebit | A Quantum Leap In Data Storage<\/title>\n<meta name=\"description\" content=\"Read Weebit Blog: Technology updates, industry trends, NVM embedded ReRAM (RRAM) IP developments replace Flash (eFlash) memory for SoC semiconductor company and Innovative Memory Architectures for AI\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Innovative Memory Architectures for AI | Weebit | A Quantum Leap In Data Storage\" \/>\n<meta property=\"og:description\" content=\"Read Weebit Blog: Technology updates, industry trends, NVM embedded ReRAM (RRAM) IP developments replace Flash (eFlash) memory for SoC semiconductor company and Innovative Memory Architectures for AI\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\" \/>\n<meta property=\"og:site_name\" content=\"Weebit\" \/>\n<meta property=\"article:published_time\" content=\"2024-07-09T06:45:07+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-25T11:09:53+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash-1024x580.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"580\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Gideon Intrater\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gideon Intrater\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":[\"Article\",\"BlogPosting\"],\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\"},\"author\":{\"name\":\"Gideon Intrater\",\"@id\":\"https:\/\/www.weebit-nano.com\/#\/schema\/person\/a0990129dba5d6f172f0eba080c3619d\"},\"headline\":\"Innovative Memory Architectures for AI\",\"datePublished\":\"2024-07-09T06:45:07+00:00\",\"dateModified\":\"2025-06-25T11:09:53+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\"},\"wordCount\":732,\"publisher\":{\"@id\":\"https:\/\/www.weebit-nano.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png\",\"articleSection\":[\"AI\",\"General\",\"Tech Insight\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\",\"url\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\",\"name\":\"Innovative Memory Architectures for AI | Weebit | A Quantum Leap In Data Storage\",\"isPartOf\":{\"@id\":\"https:\/\/www.weebit-nano.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png\",\"datePublished\":\"2024-07-09T06:45:07+00:00\",\"dateModified\":\"2025-06-25T11:09:53+00:00\",\"description\":\"Read Weebit Blog: Technology updates, industry trends, NVM embedded ReRAM (RRAM) IP developments replace Flash (eFlash) memory for SoC semiconductor company and Innovative Memory Architectures for AI\",\"breadcrumb\":{\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage\",\"url\":\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png\",\"contentUrl\":\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png\",\"width\":2560,\"height\":1450},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.weebit-nano.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Innovative Memory Architectures for AI\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.weebit-nano.com\/#website\",\"url\":\"https:\/\/www.weebit-nano.com\/\",\"name\":\"Weebit\",\"description\":\"A Quantum Leap In Data Storage\",\"publisher\":{\"@id\":\"https:\/\/www.weebit-nano.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.weebit-nano.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.weebit-nano.com\/#organization\",\"name\":\"Weebit Nano\",\"url\":\"https:\/\/www.weebit-nano.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.weebit-nano.com\/#\/schema\/logo\/image\/\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Weebit Nano\"},\"image\":{\"@id\":\"https:\/\/www.weebit-nano.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.weebit-nano.com\/#\/schema\/person\/a0990129dba5d6f172f0eba080c3619d\",\"name\":\"Gideon Intrater\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.weebit-nano.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/cropped-Gideon-Intrater-96x96.jpg\",\"contentUrl\":\"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/cropped-Gideon-Intrater-96x96.jpg\",\"caption\":\"Gideon Intrater\"},\"description\":\"Strategic Advisor\",\"url\":\"https:\/\/www.weebit-nano.com\/author\/gideon-intrater\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Innovative Memory Architectures for AI | Weebit | A Quantum Leap In Data Storage","description":"Read Weebit Blog: Technology updates, industry trends, NVM embedded ReRAM (RRAM) IP developments replace Flash (eFlash) memory for SoC semiconductor company and Innovative Memory Architectures for AI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/","og_locale":"en_US","og_type":"article","og_title":"Innovative Memory Architectures for AI | Weebit | A Quantum Leap In Data Storage","og_description":"Read Weebit Blog: Technology updates, industry trends, NVM embedded ReRAM (RRAM) IP developments replace Flash (eFlash) memory for SoC semiconductor company and Innovative Memory Architectures for AI","og_url":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/","og_site_name":"Weebit","article_published_time":"2024-07-09T06:45:07+00:00","article_modified_time":"2025-06-25T11:09:53+00:00","og_image":[{"width":1024,"height":580,"url":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash-1024x580.png","type":"image\/png"}],"author":"Gideon Intrater","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Gideon Intrater","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":["Article","BlogPosting"],"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#article","isPartOf":{"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/"},"author":{"name":"Gideon Intrater","@id":"https:\/\/www.weebit-nano.com\/#\/schema\/person\/a0990129dba5d6f172f0eba080c3619d"},"headline":"Innovative Memory Architectures for AI","datePublished":"2024-07-09T06:45:07+00:00","dateModified":"2025-06-25T11:09:53+00:00","mainEntityOfPage":{"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/"},"wordCount":732,"publisher":{"@id":"https:\/\/www.weebit-nano.com\/#organization"},"image":{"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage"},"thumbnailUrl":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png","articleSection":["AI","General","Tech Insight"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/","url":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/","name":"Innovative Memory Architectures for AI | Weebit | A Quantum Leap In Data Storage","isPartOf":{"@id":"https:\/\/www.weebit-nano.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage"},"image":{"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage"},"thumbnailUrl":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png","datePublished":"2024-07-09T06:45:07+00:00","dateModified":"2025-06-25T11:09:53+00:00","description":"Read Weebit Blog: Technology updates, industry trends, NVM embedded ReRAM (RRAM) IP developments replace Flash (eFlash) memory for SoC semiconductor company and Innovative Memory Architectures for AI","breadcrumb":{"@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#primaryimage","url":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png","contentUrl":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/Pb_Memory-Architectures-AI-Weebit-technology-IP-RRAM-ReRAM-NVM-Flash.png","width":2560,"height":1450},{"@type":"BreadcrumbList","@id":"https:\/\/www.weebit-nano.com\/innovative-memory-architectures-for-ai\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.weebit-nano.com\/"},{"@type":"ListItem","position":2,"name":"Innovative Memory Architectures for AI"}]},{"@type":"WebSite","@id":"https:\/\/www.weebit-nano.com\/#website","url":"https:\/\/www.weebit-nano.com\/","name":"Weebit","description":"A Quantum Leap In Data Storage","publisher":{"@id":"https:\/\/www.weebit-nano.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.weebit-nano.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.weebit-nano.com\/#organization","name":"Weebit Nano","url":"https:\/\/www.weebit-nano.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.weebit-nano.com\/#\/schema\/logo\/image\/","url":"","contentUrl":"","caption":"Weebit Nano"},"image":{"@id":"https:\/\/www.weebit-nano.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.weebit-nano.com\/#\/schema\/person\/a0990129dba5d6f172f0eba080c3619d","name":"Gideon Intrater","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.weebit-nano.com\/#\/schema\/person\/image\/","url":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/cropped-Gideon-Intrater-96x96.jpg","contentUrl":"https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2024\/07\/cropped-Gideon-Intrater-96x96.jpg","caption":"Gideon Intrater"},"description":"Strategic Advisor","url":"https:\/\/www.weebit-nano.com\/author\/gideon-intrater\/"}]}},"_links":{"self":[{"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/posts\/15163","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/comments?post=15163"}],"version-history":[{"count":1,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/posts\/15163\/revisions"}],"predecessor-version":[{"id":16496,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/posts\/15163\/revisions\/16496"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/media\/15162"}],"wp:attachment":[{"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/media?parent=15163"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/categories?post=15163"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.weebit-nano.com\/wp-json\/wp\/v2\/tags?post=15163"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}