{"componentChunkName":"component---src-templates-publication-js","path":"/research/h2onet-cvpr2023/","result":{"pageContext":{"publication":{"id":"h2onet-cvpr2023","title":"H2ONet: Hand-Occlusion-and-Orientation-aware Network for Real-time 3D Hand Mesh Reconstruction","authors":["Hao Xu","Tianyu Wang","Xiao Tang","Chi-Wing Fu"],"highlightAuthor":"Tianyu Wang","venue":"CVPR","year":2023,"tldr":"A network for real-time 3D hand mesh reconstruction that handles hand occlusion and orientation challenges.","links":{"paper":"https://openaccess.thecvf.com/content/CVPR2023/papers/Xu_H2ONet_Hand-Occlusion-and-Orientation-Aware_Network_for_Real-Time_3D_Hand_Mesh_Reconstruction_CVPR_2023_paper.pdf","video":"https://www.youtube.com/watch?v=JN-G8ePC3Mk","code":"https://github.com/hxwork/H2ONet_Pytorch"},"insight":"Addresses the overlooked challenges of hand occlusion and orientation in 3D hand mesh reconstruction, achieving real-time performance while significantly improving accuracy on occluded and rotated hand poses.","teaser":"/images/information/h2onet.png","bibtex":"@inproceedings{xu2023h2onet,\n  title={H2ONet: Hand-Occlusion-and-Orientation-aware Network for Real-time 3D Hand Mesh Reconstruction},\n  author={Xu, Hao and Wang, Tianyu and Tang, Xiao and Fu, Chi-Wing},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n  year={2023}\n}","teaserGif":"/videos/previews/h2onet-cvpr2023.gif","abstract":"Real-time 3D hand mesh reconstruction is challenging, especially when the hand is holding some object. Beyond the previous methods, we design H2ONet to fully exploit non-occluded information from multiple frames to boost the reconstruction quality. First, we decouple hand mesh reconstruction into two branches, one to exploit finger-level non-occluded information and the other to exploit global hand orientation, with lightweight structures to promote real-time inference. Second, we propose finger-level occlusion-aware feature fusion, leveraging predicted finger-level occlusion information as guidance to fuse finger-level information across time frames. Further, we design hand-level occlusion-aware feature fusion to fetch non-occluded information from nearby time frames. We conduct experiments on the Dex-YCB and HO3D-v2 datasets with challenging hand-object occlusion cases, manifesting that H2ONet is able to run in real-time and achieves state-of-the-art performance on both the hand mesh and pose precision. The code will be released on GitHub.","abstractSource":"https://openaccess.thecvf.com/content/CVPR2023/html/Xu_H2ONet_Hand-Occlusion-and-Orientation-Aware_Network_for_Real-Time_3D_Hand_Mesh_Reconstruction_CVPR_2023_paper.html"}}},"staticQueryHashes":["63159454"]}