CGNICO/GaussianHaircut
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update

Browse Source
This commit is contained in:
Jeffreytsai1004 2025-02-15 21:12:05 +08:00
parent 1d4f77fdec
commit cdc0d75a8a
6 changed files with 170 additions and 81 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

126
README.md
View File

@ -41,11 +41,32 @@ Expected output:
└── visualization/ # Rendering results and video └── visualization/ # Rendering results and video
``` ```
## 入门指南 Required resource structure:
```
resource/
├── NeuralHaircut/
│ ├── diffusion_prior/
│ │ └── model.pt # Diffusion prior model
│ └── PIXIE/
│ └── pixie_data/ # PIXIE model data
├── Matte-Anything/
│ ├── sam_vit_h_4b8939.pth # SAM model
│ ├── groundingdino_swint_ogc.pth # GroundingDINO model
│ └── model.pth # Matte-Anything model
├── openpose/
│ └── models/
│ ├── pose/ # OpenPose pose models
│ └── face/ # OpenPose face models
└── hyperIQA/
└── pretrained/
└── hyperIQA.pth # Image quality assessment model
```
### Linux 平台 ## Getting Started
1. **安装 CUDA 11.8** ### Linux Platform
1. **Install CUDA 11.8**
按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。 按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。
@ -55,11 +76,11 @@ Expected output:
该环境仅在此 CUDA 版本下进行了测试。 该环境仅在此 CUDA 版本下进行了测试。
2. **安装 Blender 3.6** 以创建股线可视化 2. **Install Blender 3.6** 以创建股线可视化
按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。 按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。
3. **克隆仓库并运行安装脚本** 3. **Clone repository and run installation script**
```bash ```bash
git clone git@github.com:eth-ait/GaussianHaircut.git git clone git@github.com:eth-ait/GaussianHaircut.git
@ -68,23 +89,23 @@ Expected output:
./install.sh ./install.sh
``` ```
### Windows 平台 ### Windows Platform
1. **安装 CUDA 11.8** 1. **Install CUDA 11.8**
- 从 https://developer.nvidia.com/cuda-11-8-0-download-archive 下载并安装 - 从 https://developer.nvidia.com/cuda-11-8-0-download-archive 下载并安装
- 默认安装路径：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 - 默认安装路径：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
- 确保CUDA版本与系统兼容 - 确保CUDA版本与系统兼容
2. **安装 Blender 3.6** 2. **Install Blender 3.6**
- 从 https://www.blender.org/download/lts/3-6 下载并安装 - 从 https://www.blender.org/download/lts/3-6 下载并安装
- 默认安装路径：C:\Program Files\Blender Foundation\Blender 3.6 - 默认安装路径：C:\Program Files\Blender Foundation\Blender 3.6
3. **安装 Visual Studio 2019 Build Tools** 3. **Install Visual Studio 2019 Build Tools**
- 从 https://visualstudio.microsoft.com/vs/older-downloads/ 下载并安装 - 从 https://visualstudio.microsoft.com/vs/older-downloads/ 下载并安装
- 选择"C++构建工具"工作负载 - 选择"C++构建工具"工作负载
- 默认安装路径：C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools - 默认安装路径：C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
4. **安装 COLMAP** 4. **Install COLMAP**
- 从 https://github.com/colmap/colmap/releases 下载并安装 - 从 https://github.com/colmap/colmap/releases 下载并安装
- 下载CUDA版本的COLMAP (例如：COLMAP-3.8-windows-cuda.zip) - 下载CUDA版本的COLMAP (例如：COLMAP-3.8-windows-cuda.zip)
- 解压到不含空格的路径 (例如：C:\COLMAP) - 解压到不含空格的路径 (例如：C:\COLMAP)
@ -96,7 +117,7 @@ Expected output:
5. 点击"确定"保存 5. 点击"确定"保存
- 重启终端使PATH生效 - 重启终端使PATH生效
5. **安装 7-Zip** 5. **Install 7-Zip**
- 从 https://7-zip.org/ 下载并安装 - 从 https://7-zip.org/ 下载并安装
- 将7-Zip安装目录添加到系统PATH: - 将7-Zip安装目录添加到系统PATH:
1. 打开"系统属性" > "环境变量" 1. 打开"系统属性" > "环境变量"
@ -106,7 +127,7 @@ Expected output:
5. 点击"确定"保存 5. 点击"确定"保存
- 重启终端使PATH生效 - 重启终端使PATH生效
6. **下载预训练模型和资源** 6. **Download pre-trained models and resources**
```cmd ```cmd
git clone https://github.com/Jeffreytsai1004/GaussianHairCut git clone https://github.com/Jeffreytsai1004/GaussianHairCut
cd GaussianHairCut cd GaussianHairCut
@ -119,7 +140,7 @@ Expected output:
- 如果下载失败，可以重新运行脚本 - 如果下载失败，可以重新运行脚本
- 确保有稳定的网络连接 - 确保有稳定的网络连接
6. **克隆仓库并运行安装脚本** 6. **Clone repository and run installation script**
```cmd ```cmd
git clone https://github.com/Jeffreytsai1004/GaussianHairCut git clone https://github.com/Jeffreytsai1004/GaussianHairCut
cd GaussianHairCut cd GaussianHairCut
@ -131,66 +152,47 @@ Expected output:
.\run.bat .\run.bat
``` ```
## 重建 ## Usage
1. **录制单目视频** 1. **Record Monocular Video**
- 参考项目页面上的示例视频 - Reference example videos on the project page
- 录制要求： - Recording requirements:
* 拍摄对象应缓慢转动头部，确保捕捉到所有角度 * Subject should rotate head slowly to capture all angles
* 保持头发和面部清晰可见 * Keep hair and face clearly visible
* 避免快速移动导致的运动模糊 * Avoid motion blur from fast movements
* 保持光照条件稳定 * Maintain stable lighting conditions
* 建议视频长度：10-20秒 * Recommended length: 10-20 seconds
* 建议分辨率：1920x1080或更高 * Recommended resolution: 1920x1080 or higher
2. **设置重建场景的目录** 2. **Setup Scene Directory**
创建场景目录结构：
```
[your_scene_folder]/
└── raw.mp4 # 您录制的视频文件
```
注意：
- DATA_PATH 应指向包含 raw.mp4 的目录
- 目录路径不应包含空格或特殊字符
- 脚本会在此目录下自动创建所需的子目录
- 确保有足够的磁盘空间(建议至少20GB)
3. **运行脚本**
Linux:
```bash
export PROJECT_DIR="[/path/to/]GaussianHaircut"
export BLENDER_DIR="[/path/to/blender/folder/]blender"
DATA_PATH="[path/to/scene/folder]" ./run.sh
```
Windows:
```cmd ```cmd
# 在CMD中运行: # In CMD:
set PROJECT_DIR=[path\to\]GaussianHaircut set PROJECT_DIR=[path\to\]GaussianHaircut
set DATA_PATH=[path\to\scene\folder] set DATA_PATH=[path\to\scene\folder]
run.bat run.bat
# 或在PowerShell中运行: # Or in PowerShell:
set PROJECT_DIR=[path\to\]GaussianHaircut $env:PROJECT_DIR="[path\to\]GaussianHaircut"
set DATA_PATH=[path\to\scene\folder] $env:DATA_PATH="[path\to\scene\folder]"
.\run.bat .\run.bat
``` ```
该脚本执行数据预处理、重建以及使用 Blender 进行最终可视化。使用 Tensorboard 查看中间可视化结果。 Note:
- DATA_PATH should point to directory containing raw.mp4
- Directory paths should not contain spaces or special characters
- Ensure sufficient disk space (at least 20GB recommended)
## 许可证 ## License
此代码基于 3D Gaussian Splatting 项目。有关条款和条件，请参阅 LICENSE_3DGS。其余代码根据 CC BY-NC-SA 4.0 分发。 This code is based on the 3D Gaussian Splatting project. See LICENSE_3DGS for terms and conditions. The rest of the code is distributed under CC BY-NC-SA 4.0.
如果此代码对您的项目有帮助，请引用以下论文。 ## Citation
## 引用 If you find this code helpful for your research, please cite our paper:
``` ```bibtex
@inproceedings{zakharov2024gh, @inproceedings{zakharov2024gh,
title = {Human Hair Reconstruction with Strand-Aligned 3D Gaussians}, title = {Human Hair Reconstruction with Strand-Aligned 3D Gaussians},
author = {Zakharov, Egor and Sklyarova, Vanessa and Black, Michael J and Nam, Giljoo and Thies, Justus and Hilliges, Otmar}, author = {Zakharov, Egor and Sklyarova, Vanessa and Black, Michael J and Nam, Giljoo and Thies, Justus and Hilliges, Otmar},
@ -199,11 +201,11 @@ Windows:
} }
``` ```
## 链接 ## Related Projects
- [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting) - [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)
- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME 拟合管线、股线先验和发型扩散先验 - [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME fitting pipeline, strand prior and hairstyle diffusion prior
- [HAAR](https://github.com/Vanessik/HAAR): 头发上采样 - [HAAR](https://github.com/Vanessik/HAAR): Hair upsampling
- [Matte-Anything](https://github.com/hustvl/Matte-Anything): 头发和身体分割 - [Matte-Anything](https://github.com/hustvl/Matte-Anything): Hair and body segmentation
- [PIXIE](https://github.com/yfeng95/PIXIE): FLAME 拟合的初始化 - [PIXIE](https://github.com/yfeng95/PIXIE): FLAME fitting initialization
- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): 用于 FLAME 拟合的关键点检测 - [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): Keypoint detection for FLAME fitting

48
README_CN.md
View File

@ -41,7 +41,53 @@
└── visualization/ # 渲染结果和视频 └── visualization/ # 渲染结果和视频
``` ```
## 安装指南 所需资源文件结构：
```
resource/
├── NeuralHaircut/
│ ├── diffusion_prior/
│ │ └── model.pt # 扩散先验模型
│ └── PIXIE/
│ └── pixie_data/ # PIXIE模型数据
├── Matte-Anything/
│ ├── sam_vit_h_4b8939.pth # SAM模型
│ ├── groundingdino_swint_ogc.pth # GroundingDINO模型
│ └── model.pth # Matte-Anything模型
├── openpose/
│ └── models/
│ ├── pose/ # OpenPose姿态模型
│ └── face/ # OpenPose人脸模型
└── hyperIQA/
└── pretrained/
└── hyperIQA.pth # 图像质量评估模型
```
## 环境配置
### Linux 平台
1. **安装 CUDA 11.8**
按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。
确保：
- PATH 包含 <CUDA_DIR>/bin
- LD_LIBRARY_PATH 包含 <CUDA_DIR>/lib64
该环境仅在此 CUDA 版本下进行了测试。
2. **安装 Blender 3.6** 以创建股线可视化
按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。
3. **克隆仓库并运行安装脚本**
```bash
git clone git@github.com:eth-ait/GaussianHaircut.git
cd GaussianHaircut
chmod +x ./install.sh
./install.sh
```
### Windows 平台 ### Windows 平台

49
download_resource.bat
View File

@ -25,14 +25,20 @@ IF %ERRORLEVEL% NEQ 0 (
REM 更新pip REM 更新pip
python -m pip install --upgrade pip python -m pip install --upgrade pip
REM 安装gdown REM 检查gdown是否已安装
echo 正在安装gdown... python -c "import gdown" >nul 2>nul
python -m pip install --user gdown --upgrade IF %ERRORLEVEL% EQU 0 (
python -m pip install --user requests --upgrade echo gdown已安装，跳过安装步骤...
IF %ERRORLEVEL% NEQ 0 ( ) ELSE (
echo 错误：gdown安装失败 REM 安装gdown
echo 请检查网络连接后重试 echo 正在安装gdown...
exit /b 1 python -m pip install --user gdown --upgrade
python -m pip install --user requests --upgrade
IF %ERRORLEVEL% NEQ 0 (
echo 错误：gdown安装失败
echo 请检查网络连接后重试
exit /b 1
)
) )
REM 添加Python Scripts目录到PATH REM 添加Python Scripts目录到PATH
@ -61,6 +67,11 @@ mkdir Matte-Anything 2>nul
mkdir openpose\models 2>nul mkdir openpose\models 2>nul
mkdir hyperIQA\pretrained 2>nul mkdir hyperIQA\pretrained 2>nul
REM 设置下载重试次数和超时时间
SET RETRY_COUNT=3
SET TIMEOUT_SECONDS=600
SET CURL_OPTS=--connect-timeout 30 --max-time %TIMEOUT_SECONDS% --retry %RETRY_COUNT% -L -C -
echo ************************************************************ echo ************************************************************
echo * 开始下载所需资源文件 * echo * 开始下载所需资源文件 *
echo * 下载时间取决于您的网络连接速度 * echo * 下载时间取决于您的网络连接速度 *
@ -73,8 +84,8 @@ echo 正在下载Neural Haircut文件...
IF EXIST "diffusion_prior\model.pt" ( IF EXIST "diffusion_prior\model.pt" (
echo Neural Haircut模型已存在，跳过下载... echo Neural Haircut模型已存在，跳过下载...
) ELSE ( ) ELSE (
echo 正在下载Neural Haircut文件...· echo 正在下载Neural Haircut文件...
python -m gdown --folder "https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8" python -m gdown --folder "https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8" --continue
) )
cd diffusion_prior cd diffusion_prior
@ -110,13 +121,17 @@ IF EXIST "sam_vit_h_4b8939.pth" (
echo SAM模型已存在，跳过下载... echo SAM模型已存在，跳过下载...
) ELSE ( ) ELSE (
echo 正在下载SAM模型... echo 正在下载SAM模型...
curl -L --retry 5 -o sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth curl %CURL_OPTS% -o sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
IF %ERRORLEVEL% NEQ 0 (
echo 错误：下载SAM模型失败
exit /b 1
)
) )
IF EXIST "groundingdino_swint_ogc.pth" ( IF EXIST "groundingdino_swint_ogc.pth" (
echo GroundingDINO模型已存在，跳过下载... echo GroundingDINO模型已存在，跳过下载...
) ELSE ( ) ELSE (
echo 正在下载GroundingDINO模型... echo 正在下载GroundingDINO模型...
curl -L --retry 5 -o groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth curl -L --retry 5 -C - -o groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
) )
IF EXIST "model.pth" ( IF EXIST "model.pth" (
echo Matte-Anything模型已存在，跳过下载... echo Matte-Anything模型已存在，跳过下载...
@ -159,6 +174,16 @@ echo 下载完成！
echo 资源文件已下载到resource目录。 echo 资源文件已下载到resource目录。
echo. echo.
REM 验证下载的文件完整性
echo 正在验证下载文件...
cd %PROJECT_DIR%\resource
IF NOT EXIST "NeuralHaircut\diffusion_prior\model.pt" echo 警告：缺少扩散先验模型文件
IF NOT EXIST "NeuralHaircut\PIXIE\pixie_data" echo 警告：缺少PIXIE模型数据
IF NOT EXIST "Matte-Anything\sam_vit_h_4b8939.pth" echo 警告：缺少SAM模型文件
IF NOT EXIST "Matte-Anything\groundingdino_swint_ogc.pth" echo 警告：缺少GroundingDINO模型文件
IF NOT EXIST "openpose\models\pose\coco" echo 警告：缺少OpenPose姿态模型
IF NOT EXIST "hyperIQA\pretrained\hyperIQA.pth" echo 警告：缺少hyperIQA模型文件
IF %ERRORLEVEL% NEQ 0 ( IF %ERRORLEVEL% NEQ 0 (
echo. echo.
echo 错误：下载失败 echo 错误：下载失败

13
install.bat
View File

@ -214,7 +214,20 @@ xcopy /Y "%PROJECT_DIR%\resource\hyperIQA\pretrained\*" pretrained\
REM 安装 PIXIE 环境 REM 安装 PIXIE 环境
CALL activate_pixie-env.bat CALL activate_pixie-env.bat
cd %PROJECT_DIR%\ext\PIXIE cd %PROJECT_DIR%\ext\PIXIE
echo 正在安装PIXIE依赖...
pip install pyyaml==5.4.1 pip install pyyaml==5.4.1
pip install git+https://github.com/1adrianb/face-alignment.git@54623537fd9618ca7c15688fd85aba706ad92b59 pip install git+https://github.com/1adrianb/face-alignment.git@54623537fd9618ca7c15688fd85aba706ad92b59
REM 检查资源文件是否已下载
IF NOT EXIST "resource" (
echo 错误：未找到resource目录，请先运行download_resource.bat
exit /b 1
)
REM 检查CUDA安装
IF NOT EXIST "%CUDA_HOME%" (
echo 错误：未找到CUDA安装目录，请确保CUDA 11.8已正确安装
exit /b 1
)
echo Installation completed! echo Installation completed!

10
requirements.txt
View File

@ -1,6 +1,6 @@
numpy>=1.21.0,<2.0.0 numpy>=1.21.0,<1.25.0
scipy>=1.7.0,<2.0.0 scipy>=1.7.0,<2.0.0
pillow>=8.3.1 pillow>=9.0.0
tqdm>=4.62.2 tqdm>=4.62.2
matplotlib>=3.4.2 matplotlib>=3.4.2
opencv-python>=4.5.3 opencv-python>=4.5.3
@ -9,14 +9,14 @@ imageio>=2.9.0
lpips>=0.1.4 lpips>=0.1.4
gdown>=4.7.1 gdown>=4.7.1
face-alignment>=1.3.5 face-alignment>=1.3.5
pytorch3d==0.7.5 pytorch3d>=0.7.0
ninja>=1.10.2 ninja>=1.10.2
trimesh>=3.9.35 trimesh>=3.9.35
pyhocon>=0.3.59 pyhocon>=0.3.59
wandb>=0.12.0 wandb>=0.12.0
tensorboard>=2.6.0 tensorboard>=2.6.0
open3d>=0.13.0 open3d>=0.13.0
kaolin>=0.15.0 kaolin>=0.13.0
pytorch-lightning>=1.6.0 pytorch-lightning>=1.6.0
plyfile>=0.8.1 plyfile>=0.8.1
icecream>=2.1.3 icecream>=2.1.3
@ -34,3 +34,5 @@ torchdiffeq>=0.2.3
torchsde>=0.2.5 torchsde>=0.2.5
resize-right>=0.0.2 resize-right>=0.0.2
colmap>=3.10 colmap>=3.10
torch>=1.12.0,<2.0.0
torchvision>=0.13.0,<1.0.0

1
run.bat
View File

@ -312,6 +312,7 @@ IF %ERRORLEVEL% NEQ 0 (
REM 制作视频 REM 制作视频
CALL "%MICROMAMBA_EXE%" activate -p %MAMBA_ROOT_PREFIX%\envs\gaussian_splatting_hair CALL "%MICROMAMBA_EXE%" activate -p %MAMBA_ROOT_PREFIX%\envs\gaussian_splatting_hair
cd %PROJECT_DIR%\src\postprocessing cd %PROJECT_DIR%\src\postprocessing
echo 正在生成最终视频...
set CUDA_VISIBLE_DEVICES=%CUDA_VISIBLE_DEVICES% set CUDA_VISIBLE_DEVICES=%CUDA_VISIBLE_DEVICES%
python concat_video.py --input_path "%DATA_PATH%" --exp_name_3 "%EXP_NAME_3%" python concat_video.py --input_path "%DATA_PATH%" --exp_name_3 "%EXP_NAME_3%"
IF %ERRORLEVEL% NEQ 0 ( IF %ERRORLEVEL% NEQ 0 (