MnRI Faculty and grad student win honorable mention at CVPR 2021

Congratulations to CS&E Ph.D. student Yasamin Jafarian and assistant professor Hyun Soo Park for receiving the best paper honorable mention at the 2021 Conference on Computer Vision and Pattern Recognition (CVPR).

This award recognizes outstanding work appearing at the annual conference, and winners are picked by a committee delegated by the program chairs. Over 7,000 papers were submitted to the conference, with a 23.5% acceptance rate. The committee identified 32 best paper award nominees from among the accepted papers, and ultimately selected seven winners across four award categories.

The winning work by Jafarian and Park, "Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos", addresses the challenge of learning the geometry of dressed humans due to the limited availability of the ground truth data (e.g., 3D scanned models) by leveraging a new data resource: a number of social media dance videos that span diverse appearance, clothing styles, performances, and identities.

Each video depicts dynamic movements of the body and clothes of a single person while lacking the 3D ground truth geometry. To utilize these videos, the researchers present a new method to use the local transformation that warps the predicted local geometry of the person from an image to that of the other image at a different time instant. With the transformation, the predicted geometry can be self-supervised by the warped geometry from the other image. Their method is end-to-end trainable, resulting in high fidelity depth estimation that predicts fine geometry faithful to the input real image. They also demonstrate that their method outperforms the state-of-the-art human depth estimation and human shape recovery approaches on both real and rendered images.