Researchers at 91亚色 have developed a deep learning-based computer vision system for attentive puck tracking (APT) that is able to automatically track the play in a game of hockey and deliver a dynamic zoomed video feed similar to an NHL broadcast feed.
The project, which is a collaboration between 91亚色 PhD candidate Hemanth Pidaparthy and Professor at 91亚色鈥檚 ), will be presented on Jan. 10 at the in Hawaii.
While hockey involves a large playing surface, instantaneous play is typically localized to a smaller region of the ice. Live spectators attentively shift their gaze to follow play, and professional sports videographers pan their cameras to mimic this process. Unfortunately, manual videography is economically prohibitive below the elite level.
The APT system takes as input a wide-field 4K video feed and automatically tracks the play, allowing an HD 3x-zoomed video feed to be dynamically cropped and re-targeted to a spectator's display device (e.g. an iPad). This allows family and friends to enjoy high-quality video recordings of amateur games in the comfort of their homes and provides coaches and players with the ability to review play.
Original 4K video. Green dot: automatically estimated puck location. Blue rectangle: instantaneous region of interest
HD 3x-zoomed view, re-targeted for a standard display device
鈥淲ithout this technology, it is really not possible to enjoy watching a game of amateur hockey on a standard display, as the puck is too small to be visible,鈥 explained Pidaparthy.
A prototype of the system is currently operating at the Canlan Ice Sports facility at 91亚色's Keele Campus, where it is being used to record the varsity home games for the 2018-19 season. A provisional patent application has been filed, and and 91亚色鈥檚 program are supporting the researchers to commercialize the technology. The project was awarded funding from the the VISTA Prototyping Fund to accelerate the development.
鈥淭his is a good example of how AI (artificial intelligence) can be used to simplify and enrich our lives. In this case, greater online access to community sporting events can cut down on vehicle traffic while keeping us socially engaged,鈥 said Elder, who holds the 91亚色 Research Chair in Human and Computer Vision and is jointly appointed to both the and the at 91亚色.
The research was supported by the Natural Sciences & Engineering Research Council (NSERC)聽program and by the NSERC聽, led at 91亚色 by Elder.
The researchers are currently enhancing their system to automatically extract video summaries of game highlights, and plan to extend their research to other sports such as soccer in the spring.