The Contribution of H3 to the AI Revolution

Since late 2023, generative AI models like ChatGPT have excelled across various fields. In 2024, we expect a surge in applications based on Large Language Models (LLMs), speeding up content creation in text, videos, images, and audio, even in unconventional areas. For the recent update as the example, StarCoder2 is coming out to generate code, and Now Assist for Telecommunications Service Management (TSM) by NVIDIA AI may powerfully enhance customer service in telecom companies. As firms seek cost savings and adopt AI, its widespread use will drive societal innovation. The evolution from hardware to applications will continue to be driven by Large Language Models, affecting AI chip production, accelerators, memory tech, and upper-layer applications.

Figure 1: Overview of the AI Technology Stack

What can H3 bring about 

What can H3 bring about in this technical revolution? As depicted in Figure 1, the AI tech ecosystem encompasses various partners within the industry. H3's contribution with GPU solution and memory pooling falls under the Compute Hardware layer, supporting the development of models built upon which as integral components. These infrastructure optimizations facilitate the development and deployment of applications, ultimately contributing to the transformative impact of AI on modern society.

In the processes of model training and data inference, which are fundamental to AI, GPU acceleration plays a crucial role. H3's PCIe Gen5 composable GPU solution stands out by accommodating up to 10 GPU modules within a single chassis and facilitating direct GPU-to-GPU communication, bypassing the need for CPU intervention. This innovative composable architecture maximizes the potential of GPU acceleration.

Addressing Memory Challenges in AI Infrastructure 

On the other hand, as the AI trend overturns the world, another problem to face is the obsolete system architecture. As the flushing in of data and application processing workload, the requirement of memory will surge, resulting in the running out of memory or spilling into the disk, which will seriously degrade performances due to low bandwidth and high latency.

Figure 2. Infrastructures Driven by LLM Applications (Source: Applied Materials) 

CXL memory technologies can be the way out of the new situational difficulties. As a solution, CXL memory tech makes up the gap between direct-attached memory and SSD to unlock the power of AI microprocessors in AI infrastructure. In the CXL memory pooling solution, CXL memory expanders are installed in the JBOM (Just-a-Bunch-of-Memory) as a pool, with the switch to create memory fabrics to enlarge memory capacities for the CPU. Having access to memory with low latency and high bandwidth, CPUs can eliminate the out-of-memory and spilling-to-disk issues running AI and Machine Learning applications. It is Memory Technology depicted in Figure 2, with H3 within it, that is capable of contributing to this trend.

• Learn more about CXL Memory Pooling

As generative AI models continue to advance, we expect to see a significant increase in the use of Large Language Models (LLMs) across various domains. This rise in LLM-based applications highlights the growing impact of AI on diverse sectors. As more and more firms adopt AI to drive cost savings and innovation, the transition from hardware to applications is being propelled by Large Language Models.

H3 can help in the ongoing technical revolution as it provides infrastructure optimizations within the Compute Hardware layer. It offers a GPU solution and memory pooling capabilities that support the development and deployment of AI applications while enhancing their efficiency and performance. Additionally, H3's PCIe Gen5 composable GPU solution maximizes the power of GPU acceleration, and CXL memory technologies address the growing demand for memory in AI infrastructure. By enabling access to low-latency and high-bandwidth memory, H3 helps overcome memory constraints, which ultimately drives the transformative impact of AI on modern society.