Gaussian Haircut: Human Hair Reconstruction with Strand-Aligned 3D Gaussians

This repository contains the official implementation of Gaussian Haircut, a strand-based human hair reconstruction method from monocular video.

Paper | Project Page

Overview

The reconstruction process includes the following main stages:

Preprocessing Stage
- Video frame extraction and organization
- COLMAP camera reconstruction
- Hair and body segmentation
- Image quality assessment and filtering
- Orientation map calculation
- Facial keypoint detection
- FLAME head model fitting
Reconstruction Stage
- 3D Gaussian reconstruction
- FLAME mesh fitting
- Scene cropping and optimization
- Hair strand reconstruction
Visualization Stage
- Export reconstructed strands
- Blender rendering visualization
- Generate result video

Expected output:

[your_scene_folder]/
├── raw.mp4                    # Input video
├── 3d_gaussian_splatting/     # 3D Gaussian reconstruction results
├── flame_fitting/             # FLAME head model fitting results
├── strands_reconstruction/    # Hair strand reconstruction intermediate results
├── curves_reconstruction/     # Final hair strand results
└── visualization/            # Rendering results and video

Required resource structure:

resource/
├── NeuralHaircut/
│   ├── pretrained_models/
│   │   ├── diffusion_prior/
│   │   │   └── dif_ckpt.pt          # Diffusion prior model
│   │   └── strand_prior/
│   │       └── strand_ckpt.pt       # Strand prior model
│   └── PIXIE/
│       └── pixie_data/
│           └── pixie_data/          # PIXIE model data
├── Matte-Anything/
│   ├── model.pth                    # Matte-Anything model
│   ├── sam_vit_h_4b8939.pth        # SAM model
│   └── groundingdino_swint_ogc.pth  # GroundingDINO model
├── openpose/
│   └── models/
│       └── pose/
│           └── coco/
│               └── pose_iter_584000.caffemodel  # OpenPose pose model
└── hyperIQA/
    └── pretrained/
        └── hyperIQA.pth             # Image quality assessment model

Getting Started

Linux Platform

Install CUDA 11.8

按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。

确保：
- PATH 包含 <CUDA_DIR>/bin
- LD_LIBRARY_PATH 包含 <CUDA_DIR>/lib64
该环境仅在此 CUDA 版本下进行了测试。
Install Blender 3.6 以创建股线可视化

按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。

Clone repository and run installation script

git clone git@github.com:eth-ait/GaussianHaircut.git
cd GaussianHaircut
chmod +x ./install.sh
./install.sh

Windows Platform

Install CUDA 11.8
- 从 https://developer.nvidia.com/cuda-11-8-0-download-archive 下载并安装
- 默认安装路径：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
- 确保CUDA版本与系统兼容
Install Blender 3.6
- 从 https://www.blender.org/download/lts/3-6 下载并安装
- 默认安装路径：C:\Program Files\Blender Foundation\Blender 3.6
Install Visual Studio 2019 Build Tools
- 从 https://visualstudio.microsoft.com/vs/older-downloads/ 下载并安装
- 选择"C++构建工具"工作负载
- 默认安装路径：C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
Install COLMAP
- 从 https://github.com/colmap/colmap/releases 下载并安装
- 下载CUDA版本的COLMAP (例如：COLMAP-3.8-windows-cuda.zip)
- 解压到不含空格的路径 (例如：C:\COLMAP)
- 将COLMAP目录添加到系统PATH:
  1. 打开"系统属性" > "环境变量"
  2. 在"系统变量"中找到"Path"
  3. 点击"编辑" > "新建"
  4. 添加COLMAP目录路径
  5. 点击"确定"保存
- 重启终端使PATH生效
Install 7-Zip
- 从 https://7-zip.org/ 下载并安装
- 将7-Zip安装目录添加到系统PATH:
  1. 打开"系统属性" > "环境变量"
  2. 在"系统变量"中找到"Path"
  3. 点击"编辑" > "新建"
  4. 添加7-Zip安装目录(默认为C:\Program Files\7-Zip)
  5. 点击"确定"保存
- 重启终端使PATH生效
Download pre-trained models and resources
```
git clone https://github.com/Jeffreytsai1004/GaussianHairCut
cd GaussianHairCut
# 在PowerShell中运行:
# 脚本会自动安装gdown并下载所需资源
.\download_resource.bat
```
注意：
- 下载过程可能需要几分钟到几十分钟，取决于网络速度
- 如果下载失败，可以重新运行脚本
- 确保有稳定的网络连接

Clone repository and run installation script

git clone https://github.com/Jeffreytsai1004/GaussianHairCut
cd GaussianHairCut
# 先下载所需要的资源
.\download_resource.bat
# 运行安装脚本
.\install.bat
# 运行重建脚本
.\run.bat

Usage

Record Monocular Video
- Reference example videos on the project page
- Recording requirements:
  - Subject should rotate head slowly to capture all angles
  - Keep hair and face clearly visible
  - Avoid motion blur from fast movements
  - Maintain stable lighting conditions
  - Recommended length: 10-20 seconds
  - Recommended resolution: 1920x1080 or higher

Setup Scene Directory

# In CMD:
set PROJECT_DIR=[path\to\]GaussianHaircut
set DATA_PATH=[path\to\scene\folder]
run.bat

# Or in PowerShell:
$env:PROJECT_DIR="[path\to\]GaussianHaircut"
$env:DATA_PATH="[path\to\scene\folder]"
.\run.bat

Note:

DATA_PATH should point to directory containing raw.mp4
Directory paths should not contain spaces or special characters
Ensure sufficient disk space (at least 20GB recommended)

License

This code is based on the 3D Gaussian Splatting project. See LICENSE_3DGS for terms and conditions. The rest of the code is distributed under CC BY-NC-SA 4.0.

Citation

If you find this code helpful for your research, please cite our paper:

@inproceedings{zakharov2024gh,
   title = {Human Hair Reconstruction with Strand-Aligned 3D Gaussians},
   author = {Zakharov, Egor and Sklyarova, Vanessa and Black, Michael J and Nam, Giljoo and Thies, Justus and Hilliges, Otmar},
   booktitle = {European Conference of Computer Vision (ECCV)},
   year = {2024}
}

3D Gaussian Splatting
Neural Haircut: FLAME fitting pipeline, strand prior and hairstyle diffusion prior
HAAR: Hair upsampling
Matte-Anything: Hair and body segmentation
PIXIE: FLAME fitting initialization
Face-Alignment, OpenPose: Keypoint detection for FLAME fitting

7.3 KiB Raw Blame History Unescape Escape