Gaussian Haircut: Human Hair Reconstruction with Strand-Aligned 3D Gaussians

This repository contains the official implementation of Gaussian Haircut, a strand-based human hair reconstruction method from monocular video.

Paper | Project Page

Overview

The reconstruction process includes the following main stages:

Preprocessing Stage
- Video frame extraction and organization
- COLMAP camera reconstruction
- Hair and body segmentation
- Image quality assessment and filtering
- Orientation map calculation
- Facial keypoint detection
- FLAME head model fitting
Reconstruction Stage
- 3D Gaussian reconstruction
- FLAME mesh fitting
- Scene cropping and optimization
- Hair strand reconstruction
Visualization Stage
- Export reconstructed strands
- Blender rendering visualization
- Generate result video

Expected output:

[your_scene_folder]/
├── raw.mp4                    # Input video
├── 3d_gaussian_splatting/     # 3D Gaussian reconstruction results
├── flame_fitting/             # FLAME head model fitting results
├── strands_reconstruction/    # Hair strand reconstruction intermediate results
├── curves_reconstruction/     # Final hair strand results
└── visualization/            # Rendering results and video

Directory Structure

├── cache/                      # Cache directory
│   ├── gdown/                 # gdown cache
│   ├── torch/                 # PyTorch cache
│   └── huggingface/          # Hugging Face cache
├── ext/                       # External dependencies
│   ├── NeuralHaircut/        # NeuralHaircut repository
│   ├── Matte-Anything/       # Matte-Anything repository
│   ├── openpose/             # OpenPose repository
│   ├── pytorch3d/            # PyTorch3D repository
│   ├── simple-knn/           # Simple KNN repository
│   ├── kaolin/               # Kaolin repository
│   └── hyperIQA/             # HyperIQA repository
├── resource/                  # Resource files
│   ├── NeuralHaircut/        # NeuralHaircut models
│   ├── Matte-Anything/       # Matte-Anything models
│   ├── openpose/             # OpenPose models
│   ├── PIXIE/                # PIXIE models
│   └── hyperIQA/             # HyperIQA models
├── src/                      # Source code
├── micromamba/               # Micromamba installation
├── micromamba.exe           # Micromamba executable
├── install.bat              # Installation script
├── download_resource.bat    # Resource download script
└── run.bat                  # Execution script

Environment Variables

Required environment variables:

set "PROJECT_DIR=C:\path\to\project"              # Project root directory
set "CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
set "BLENDER_DIR=C:\Program Files\Blender Foundation\Blender 3.6"
set "VS_DIR=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"

Getting Started

Linux Platform

Install CUDA 11.8

Follow instructions at https://developer.nvidia.com/cuda-11-8-0-download-archive

Make sure:
- PATH includes <CUDA_DIR>/bin
- LD_LIBRARY_PATH includes <CUDA_DIR>/lib64
The environment was tested only with this CUDA version.
Install Blender 3.6 for strand visualization

Follow instructions at https://www.blender.org/download/lts/3-6

Clone repository and run installation script

git clone git@github.com:eth-ait/GaussianHaircut.git
cd GaussianHaircut
chmod +x ./install.sh
./install.sh

Windows Platform

Install CUDA 11.8
- Download and install from https://developer.nvidia.com/cuda-11-8-0-download-archive
- Default installation path: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
- Ensure CUDA version is compatible with your system
Install Blender 3.6
- Download and install from https://www.blender.org/download/lts/3-6
- Default installation path: C:\Program Files\Blender Foundation\Blender 3.6
Install Visual Studio 2019 Build Tools
- Download and install from https://visualstudio.microsoft.com/vs/older-downloads/
- Select "C++ Build Tools" workload
- Default installation path: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
Install COLMAP
- Download from https://github.com/colmap/colmap/releases
- Download CUDA version of COLMAP (e.g., COLMAP-3.8-windows-cuda.zip)
- Extract to a path without spaces (e.g., C:\COLMAP)
- Add COLMAP directory to system PATH:
  1. Open "System Properties" > "Environment Variables"
  2. Under "System Variables", find "Path"
  3. Click "Edit" > "New"
  4. Add COLMAP directory path
  5. Click "OK" to save
- Restart terminal for PATH changes to take effect
Install 7-Zip
- Download and install from https://7-zip.org/
- Add 7-Zip installation directory to system PATH:
  1. Open "System Properties" > "Environment Variables"
  2. Under "System Variables", find "Path"
  3. Click "Edit" > "New"
  4. Add 7-Zip installation directory (default: C:\Program Files\7-Zip)
  5. Click "OK" to save
- Restart terminal for PATH changes to take effect

Download pre-trained models and resources

git clone https://github.com/Jeffreytsai1004/GaussianHairCut
cd GaussianHairCut
# Run in PowerShell:
# The script will automatically install gdown and download required resources
.\download_resource.bat

Note:

Download time varies from minutes to tens of minutes depending on network speed
If download fails, you can rerun the script
Ensure stable network connection

Usage

Record Monocular Video
- Reference example videos on the project page
- Recording requirements:
  - Subject should rotate head slowly to capture all angles
  - Keep hair and face clearly visible
  - Avoid motion blur from fast movements
  - Maintain stable lighting conditions
  - Recommended length: 10-20 seconds
  - Recommended resolution: 1920x1080 or higher Note:
- DATA_PATH should point to directory containing raw.mp4
- Directory paths should not contain spaces or special characters
- Ensure sufficient disk space (at least 20GB recommended)
Setup Scene Directory
- Create a new folder, e.g., C:\path\to\scene\folder
- Place raw.mp4 in this folder

Run Installation and Reconstruction Scripts

Set environment variables PROJECT_DIR and DATA_PATH in install.bat and run.bat
For example:

set "PROJECT_DIR=C:\path\to\project"
set "DATA_PATH=C:\path\to\scene\folder"

Modify environment variables CUDA_HOME, BLENDER_DIR, VS_DIR in install.bat and run.bat

set "CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
set "BLENDER_DIR=C:\Program Files\Blender Foundation\Blender 3.6"
set "VS_DIR=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"

Run installation script

.\install.bat

Run reconstruction script

.\run.bat

License

This code is based on the 3D Gaussian Splatting project. See LICENSE_3DGS for terms and conditions. The rest of the code is distributed under CC BY-NC-SA 4.0.

Citation

If you find this code helpful for your research, please cite our paper:

@inproceedings{zakharov2024gh,
   title = {Human Hair Reconstruction with Strand-Aligned 3D Gaussians},
   author = {Zakharov, Egor and Sklyarova, Vanessa and Black, Michael J and Nam, Giljoo and Thies, Justus and Hilliges, Otmar},
   booktitle = {European Conference of Computer Vision (ECCV)},
   year = {2024}
}

3D Gaussian Splatting
Neural Haircut: FLAME fitting pipeline, strand prior and hairstyle diffusion prior
HAAR: Hair upsampling
Matte-Anything: Hair and body segmentation
PIXIE: FLAME fitting initialization
Face-Alignment, OpenPose: Keypoint detection for FLAME fitting

8.5 KiB Raw Blame History