{"componentChunkName":"component---src-templates-publication-js","path":"/research/handar-iccv2021/","result":{"pageContext":{"publication":{"id":"handar-iccv2021","title":"Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction","authors":["Xiao Tang","Tianyu Wang","Chi-Wing Fu"],"highlightAuthor":"Tianyu Wang","venue":"ICCV","year":2021,"tldr":"Real-time 3D hand mesh reconstruction with improved alignment accuracy.","links":{"paper":"https://arxiv.org/abs/2109.01723","code":"https://github.com/wbstx/handAR"},"insight":"Achieves accurate mesh-image alignment for real-time 3D hand reconstruction through a three-stage pipeline of joint prediction, mesh generation, and offset-based refinement, enabling practical AR applications.","teaser":"/images/information/handar.gif","bibtex":"@inproceedings{tang2021towards,\n  title={Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction},\n  author={Tang, Xiao and Wang, Tianyu and Fu, Chi-Wing},\n  booktitle={International Conference on Computer Vision (ICCV)},\n  pages={11698--11707},\n  year={2021}\n}","abstract":"3D hand-mesh reconstruction from RGB images facilitates many applications, including augmented reality (AR). However, this requires not only real-time speed and accurate hand pose and shape but also plausible mesh-image alignment. While existing works already achieve promising results, meeting all three requirements is very challenging. This paper presents a novel pipeline by decoupling the hand-mesh reconstruction task into three stages: a joint stage to predict hand joints and segmentation; a mesh stage to predict a rough hand mesh; and a refine stage to fine-tune it with an offset mesh for mesh-image alignment. With careful design in the network structure and in the loss functions, we can promote high-quality finger-level mesh-image alignment and drive the models together to deliver real-time predictions. Extensive quantitative and qualitative results on benchmark datasets demonstrate that the quality of our results outperforms the state-of-the-art methods on hand-mesh/pose precision and hand-image alignment. In the end, we also showcase several real-time AR scenarios.","abstractSource":"https://arxiv.org/abs/2109.01723","version":"arXiv version linked; publication venue as listed above"}}},"staticQueryHashes":["63159454"]}