CS&E Colloquium: Marrying Application-Level Opportunities with Algorithm-Hardware Co-Design Towards Ubiquitous Edge Intelligence

The recording-breaking performance of artificial intelligence (AI) algorithms has motivated a growing demand for bringing powerful AI-powered intelligent functionalities onto edge devices, e.g., virtual reality/augmented reality (VR/AR) and embodied AI, towards ubiquitous edge intelligence. However, the powerful performance of AI algorithms comes with much increased computational complexity and memory storage requirements, which stand at odds with the limited compute/storage resources on edge devices. Additionally, the stringent application-specific requirements, including real-time response (i.e., high throughput/low latency), high energy efficiency, and small form factor, further aggravate the aforementioned gap.  

In this talk, I will introduce a holistic solution from energy- and latency-efficient architectures, to chips, and to integrated systems to closing the above-mentioned gap to enable more extensive AI-powered edge intelligence. Excitingly, my work shares the same underlying design insight, which is to advocate simultaneously harmonizing dedicated algorithms and hardware architectures via algorithm-hardware co-design while leveraging application-level opportunities to minimize redundancy within the processing pipeline and thus boost the achievable efficiency. 

First, I will introduce our algorithm-hardware co-design work, called SmartExchange, which trades higher-cost memory storage/accesses for lower-cost computations to boost energy- and latency-efficiency. Motivated by the promising efficiency achieved by SmartExchange, we further validated its co-designed architecture by designing an AI acceleration chip prototype, which minimizes both the chip area and control overhead. To demonstrate the real-world advantages of the above SmartExchange architecture and its chip prototype, we developed i-FlatCam, a first-of-its-kind real-time eye-tracking system towards next-generation VR/AR devices, where we leverage the application-level opportunities to reduce both spatial and temporal redundancy. After that, we went beyond to build a scaled-up eye-tracking system, called EyeCoD, which targets a more general eye-tracking solution at the cost of marginally increased chip area as compared with i-FlatCams. Finally, I will conclude my talk with exciting future directions.