High-Throughput Batch Rendering for Embodied AI
DescriptionIn this paper we study the problem of efficiently rendering images for embodied AI training workloads, where agent training involves rendering millions to billions of independent frames, often at low-resolutions and with simple (or no) lighting and shading, that serve as the agent's observations of the world. To enable high-throughput end-to-end training, we design, and provide a high-performance GPU implementation of, a frontend renderer interface that allows state-of-the-art GPU-accelerated batch world simulators to efficiently communicate with high-performance rendering backends for generating agent observations. Using this interface we architect and compare two high-performance renderers: one based on the GPU hardware-accelerated graphics pipeline and a second based on a GPU software implementation of ray tracing.To evaluate these renderers and encourage further research by the graphics community in this area, we build a rendering benchmark for this underexplored regime and find that the ray tracing based renderer outperforms the rasterization based solution across the benchmark on a datacenter class GPU, while also performing competitively in geometrically complex environments on a high-end consumer GPU. When tasked to render large batches of independent 128x128 images, the raytracer can exceed 100,000 frames per second per GPU for simple scenes, and exceed 10,000 frames per second per GPU on geometrically complex scenes from the HSSD dataset.
Event Type
Technical Papers
TimeThursday, 5 December 202411:13am - 11:27am JST
LocationHall B5 (2), B Block, Level 5
