Update
This commit is contained in:
parent
1d4f77fdec
commit
cdc0d75a8a
126
README.md
126
README.md
@ -41,11 +41,32 @@ Expected output:
|
||||
└── visualization/ # Rendering results and video
|
||||
```
|
||||
|
||||
## 入门指南
|
||||
Required resource structure:
|
||||
```
|
||||
resource/
|
||||
├── NeuralHaircut/
|
||||
│ ├── diffusion_prior/
|
||||
│ │ └── model.pt # Diffusion prior model
|
||||
│ └── PIXIE/
|
||||
│ └── pixie_data/ # PIXIE model data
|
||||
├── Matte-Anything/
|
||||
│ ├── sam_vit_h_4b8939.pth # SAM model
|
||||
│ ├── groundingdino_swint_ogc.pth # GroundingDINO model
|
||||
│ └── model.pth # Matte-Anything model
|
||||
├── openpose/
|
||||
│ └── models/
|
||||
│ ├── pose/ # OpenPose pose models
|
||||
│ └── face/ # OpenPose face models
|
||||
└── hyperIQA/
|
||||
└── pretrained/
|
||||
└── hyperIQA.pth # Image quality assessment model
|
||||
```
|
||||
|
||||
### Linux 平台
|
||||
## Getting Started
|
||||
|
||||
1. **安装 CUDA 11.8**
|
||||
### Linux Platform
|
||||
|
||||
1. **Install CUDA 11.8**
|
||||
|
||||
按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。
|
||||
|
||||
@ -55,11 +76,11 @@ Expected output:
|
||||
|
||||
该环境仅在此 CUDA 版本下进行了测试。
|
||||
|
||||
2. **安装 Blender 3.6** 以创建股线可视化
|
||||
2. **Install Blender 3.6** 以创建股线可视化
|
||||
|
||||
按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。
|
||||
|
||||
3. **克隆仓库并运行安装脚本**
|
||||
3. **Clone repository and run installation script**
|
||||
|
||||
```bash
|
||||
git clone git@github.com:eth-ait/GaussianHaircut.git
|
||||
@ -68,23 +89,23 @@ Expected output:
|
||||
./install.sh
|
||||
```
|
||||
|
||||
### Windows 平台
|
||||
### Windows Platform
|
||||
|
||||
1. **安装 CUDA 11.8**
|
||||
1. **Install CUDA 11.8**
|
||||
- 从 https://developer.nvidia.com/cuda-11-8-0-download-archive 下载并安装
|
||||
- 默认安装路径:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
|
||||
- 确保CUDA版本与系统兼容
|
||||
|
||||
2. **安装 Blender 3.6**
|
||||
2. **Install Blender 3.6**
|
||||
- 从 https://www.blender.org/download/lts/3-6 下载并安装
|
||||
- 默认安装路径:C:\Program Files\Blender Foundation\Blender 3.6
|
||||
|
||||
3. **安装 Visual Studio 2019 Build Tools**
|
||||
3. **Install Visual Studio 2019 Build Tools**
|
||||
- 从 https://visualstudio.microsoft.com/vs/older-downloads/ 下载并安装
|
||||
- 选择"C++构建工具"工作负载
|
||||
- 默认安装路径:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
|
||||
|
||||
4. **安装 COLMAP**
|
||||
4. **Install COLMAP**
|
||||
- 从 https://github.com/colmap/colmap/releases 下载并安装
|
||||
- 下载CUDA版本的COLMAP (例如:COLMAP-3.8-windows-cuda.zip)
|
||||
- 解压到不含空格的路径 (例如:C:\COLMAP)
|
||||
@ -96,7 +117,7 @@ Expected output:
|
||||
5. 点击"确定"保存
|
||||
- 重启终端使PATH生效
|
||||
|
||||
5. **安装 7-Zip**
|
||||
5. **Install 7-Zip**
|
||||
- 从 https://7-zip.org/ 下载并安装
|
||||
- 将7-Zip安装目录添加到系统PATH:
|
||||
1. 打开"系统属性" > "环境变量"
|
||||
@ -106,7 +127,7 @@ Expected output:
|
||||
5. 点击"确定"保存
|
||||
- 重启终端使PATH生效
|
||||
|
||||
6. **下载预训练模型和资源**
|
||||
6. **Download pre-trained models and resources**
|
||||
```cmd
|
||||
git clone https://github.com/Jeffreytsai1004/GaussianHairCut
|
||||
cd GaussianHairCut
|
||||
@ -119,7 +140,7 @@ Expected output:
|
||||
- 如果下载失败,可以重新运行脚本
|
||||
- 确保有稳定的网络连接
|
||||
|
||||
6. **克隆仓库并运行安装脚本**
|
||||
6. **Clone repository and run installation script**
|
||||
```cmd
|
||||
git clone https://github.com/Jeffreytsai1004/GaussianHairCut
|
||||
cd GaussianHairCut
|
||||
@ -131,66 +152,47 @@ Expected output:
|
||||
.\run.bat
|
||||
```
|
||||
|
||||
## 重建
|
||||
## Usage
|
||||
|
||||
1. **录制单目视频**
|
||||
1. **Record Monocular Video**
|
||||
|
||||
- 参考项目页面上的示例视频
|
||||
- 录制要求:
|
||||
* 拍摄对象应缓慢转动头部,确保捕捉到所有角度
|
||||
* 保持头发和面部清晰可见
|
||||
* 避免快速移动导致的运动模糊
|
||||
* 保持光照条件稳定
|
||||
* 建议视频长度:10-20秒
|
||||
* 建议分辨率:1920x1080或更高
|
||||
- Reference example videos on the project page
|
||||
- Recording requirements:
|
||||
* Subject should rotate head slowly to capture all angles
|
||||
* Keep hair and face clearly visible
|
||||
* Avoid motion blur from fast movements
|
||||
* Maintain stable lighting conditions
|
||||
* Recommended length: 10-20 seconds
|
||||
* Recommended resolution: 1920x1080 or higher
|
||||
|
||||
2. **设置重建场景的目录**
|
||||
2. **Setup Scene Directory**
|
||||
|
||||
创建场景目录结构:
|
||||
```
|
||||
[your_scene_folder]/
|
||||
└── raw.mp4 # 您录制的视频文件
|
||||
```
|
||||
|
||||
注意:
|
||||
- DATA_PATH 应指向包含 raw.mp4 的目录
|
||||
- 目录路径不应包含空格或特殊字符
|
||||
- 脚本会在此目录下自动创建所需的子目录
|
||||
- 确保有足够的磁盘空间(建议至少20GB)
|
||||
|
||||
3. **运行脚本**
|
||||
|
||||
Linux:
|
||||
```bash
|
||||
export PROJECT_DIR="[/path/to/]GaussianHaircut"
|
||||
export BLENDER_DIR="[/path/to/blender/folder/]blender"
|
||||
DATA_PATH="[path/to/scene/folder]" ./run.sh
|
||||
```
|
||||
|
||||
Windows:
|
||||
```cmd
|
||||
# 在CMD中运行:
|
||||
# In CMD:
|
||||
set PROJECT_DIR=[path\to\]GaussianHaircut
|
||||
set DATA_PATH=[path\to\scene\folder]
|
||||
run.bat
|
||||
|
||||
# 或在PowerShell中运行:
|
||||
set PROJECT_DIR=[path\to\]GaussianHaircut
|
||||
set DATA_PATH=[path\to\scene\folder]
|
||||
# Or in PowerShell:
|
||||
$env:PROJECT_DIR="[path\to\]GaussianHaircut"
|
||||
$env:DATA_PATH="[path\to\scene\folder]"
|
||||
.\run.bat
|
||||
```
|
||||
|
||||
该脚本执行数据预处理、重建以及使用 Blender 进行最终可视化。使用 Tensorboard 查看中间可视化结果。
|
||||
Note:
|
||||
- DATA_PATH should point to directory containing raw.mp4
|
||||
- Directory paths should not contain spaces or special characters
|
||||
- Ensure sufficient disk space (at least 20GB recommended)
|
||||
|
||||
## 许可证
|
||||
## License
|
||||
|
||||
此代码基于 3D Gaussian Splatting 项目。有关条款和条件,请参阅 LICENSE_3DGS。其余代码根据 CC BY-NC-SA 4.0 分发。
|
||||
This code is based on the 3D Gaussian Splatting project. See LICENSE_3DGS for terms and conditions. The rest of the code is distributed under CC BY-NC-SA 4.0.
|
||||
|
||||
如果此代码对您的项目有帮助,请引用以下论文。
|
||||
## Citation
|
||||
|
||||
## 引用
|
||||
If you find this code helpful for your research, please cite our paper:
|
||||
|
||||
```
|
||||
```bibtex
|
||||
@inproceedings{zakharov2024gh,
|
||||
title = {Human Hair Reconstruction with Strand-Aligned 3D Gaussians},
|
||||
author = {Zakharov, Egor and Sklyarova, Vanessa and Black, Michael J and Nam, Giljoo and Thies, Justus and Hilliges, Otmar},
|
||||
@ -199,11 +201,11 @@ Windows:
|
||||
}
|
||||
```
|
||||
|
||||
## 链接
|
||||
## Related Projects
|
||||
|
||||
- [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)
|
||||
- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME 拟合管线、股线先验和发型扩散先验
|
||||
- [HAAR](https://github.com/Vanessik/HAAR): 头发上采样
|
||||
- [Matte-Anything](https://github.com/hustvl/Matte-Anything): 头发和身体分割
|
||||
- [PIXIE](https://github.com/yfeng95/PIXIE): FLAME 拟合的初始化
|
||||
- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): 用于 FLAME 拟合的关键点检测
|
||||
- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME fitting pipeline, strand prior and hairstyle diffusion prior
|
||||
- [HAAR](https://github.com/Vanessik/HAAR): Hair upsampling
|
||||
- [Matte-Anything](https://github.com/hustvl/Matte-Anything): Hair and body segmentation
|
||||
- [PIXIE](https://github.com/yfeng95/PIXIE): FLAME fitting initialization
|
||||
- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): Keypoint detection for FLAME fitting
|
||||
|
48
README_CN.md
48
README_CN.md
@ -41,7 +41,53 @@
|
||||
└── visualization/ # 渲染结果和视频
|
||||
```
|
||||
|
||||
## 安装指南
|
||||
所需资源文件结构:
|
||||
```
|
||||
resource/
|
||||
├── NeuralHaircut/
|
||||
│ ├── diffusion_prior/
|
||||
│ │ └── model.pt # 扩散先验模型
|
||||
│ └── PIXIE/
|
||||
│ └── pixie_data/ # PIXIE模型数据
|
||||
├── Matte-Anything/
|
||||
│ ├── sam_vit_h_4b8939.pth # SAM模型
|
||||
│ ├── groundingdino_swint_ogc.pth # GroundingDINO模型
|
||||
│ └── model.pth # Matte-Anything模型
|
||||
├── openpose/
|
||||
│ └── models/
|
||||
│ ├── pose/ # OpenPose姿态模型
|
||||
│ └── face/ # OpenPose人脸模型
|
||||
└── hyperIQA/
|
||||
└── pretrained/
|
||||
└── hyperIQA.pth # 图像质量评估模型
|
||||
```
|
||||
|
||||
## 环境配置
|
||||
|
||||
### Linux 平台
|
||||
|
||||
1. **安装 CUDA 11.8**
|
||||
|
||||
按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。
|
||||
|
||||
确保:
|
||||
- PATH 包含 <CUDA_DIR>/bin
|
||||
- LD_LIBRARY_PATH 包含 <CUDA_DIR>/lib64
|
||||
|
||||
该环境仅在此 CUDA 版本下进行了测试。
|
||||
|
||||
2. **安装 Blender 3.6** 以创建股线可视化
|
||||
|
||||
按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。
|
||||
|
||||
3. **克隆仓库并运行安装脚本**
|
||||
|
||||
```bash
|
||||
git clone git@github.com:eth-ait/GaussianHaircut.git
|
||||
cd GaussianHaircut
|
||||
chmod +x ./install.sh
|
||||
./install.sh
|
||||
```
|
||||
|
||||
### Windows 平台
|
||||
|
||||
|
@ -25,14 +25,20 @@ IF %ERRORLEVEL% NEQ 0 (
|
||||
REM 更新pip
|
||||
python -m pip install --upgrade pip
|
||||
|
||||
REM 安装gdown
|
||||
echo 正在安装gdown...
|
||||
python -m pip install --user gdown --upgrade
|
||||
python -m pip install --user requests --upgrade
|
||||
IF %ERRORLEVEL% NEQ 0 (
|
||||
echo 错误:gdown安装失败
|
||||
echo 请检查网络连接后重试
|
||||
exit /b 1
|
||||
REM 检查gdown是否已安装
|
||||
python -c "import gdown" >nul 2>nul
|
||||
IF %ERRORLEVEL% EQU 0 (
|
||||
echo gdown已安装,跳过安装步骤...
|
||||
) ELSE (
|
||||
REM 安装gdown
|
||||
echo 正在安装gdown...
|
||||
python -m pip install --user gdown --upgrade
|
||||
python -m pip install --user requests --upgrade
|
||||
IF %ERRORLEVEL% NEQ 0 (
|
||||
echo 错误:gdown安装失败
|
||||
echo 请检查网络连接后重试
|
||||
exit /b 1
|
||||
)
|
||||
)
|
||||
|
||||
REM 添加Python Scripts目录到PATH
|
||||
@ -61,6 +67,11 @@ mkdir Matte-Anything 2>nul
|
||||
mkdir openpose\models 2>nul
|
||||
mkdir hyperIQA\pretrained 2>nul
|
||||
|
||||
REM 设置下载重试次数和超时时间
|
||||
SET RETRY_COUNT=3
|
||||
SET TIMEOUT_SECONDS=600
|
||||
SET CURL_OPTS=--connect-timeout 30 --max-time %TIMEOUT_SECONDS% --retry %RETRY_COUNT% -L -C -
|
||||
|
||||
echo ************************************************************
|
||||
echo * 开始下载所需资源文件 *
|
||||
echo * 下载时间取决于您的网络连接速度 *
|
||||
@ -73,8 +84,8 @@ echo 正在下载Neural Haircut文件...
|
||||
IF EXIST "diffusion_prior\model.pt" (
|
||||
echo Neural Haircut模型已存在,跳过下载...
|
||||
) ELSE (
|
||||
echo 正在下载Neural Haircut文件...·
|
||||
python -m gdown --folder "https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8"
|
||||
echo 正在下载Neural Haircut文件...
|
||||
python -m gdown --folder "https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8" --continue
|
||||
)
|
||||
|
||||
cd diffusion_prior
|
||||
@ -110,13 +121,17 @@ IF EXIST "sam_vit_h_4b8939.pth" (
|
||||
echo SAM模型已存在,跳过下载...
|
||||
) ELSE (
|
||||
echo 正在下载SAM模型...
|
||||
curl -L --retry 5 -o sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
|
||||
curl %CURL_OPTS% -o sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
|
||||
IF %ERRORLEVEL% NEQ 0 (
|
||||
echo 错误:下载SAM模型失败
|
||||
exit /b 1
|
||||
)
|
||||
)
|
||||
IF EXIST "groundingdino_swint_ogc.pth" (
|
||||
echo GroundingDINO模型已存在,跳过下载...
|
||||
) ELSE (
|
||||
echo 正在下载GroundingDINO模型...
|
||||
curl -L --retry 5 -o groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
|
||||
curl -L --retry 5 -C - -o groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
|
||||
)
|
||||
IF EXIST "model.pth" (
|
||||
echo Matte-Anything模型已存在,跳过下载...
|
||||
@ -159,6 +174,16 @@ echo 下载完成!
|
||||
echo 资源文件已下载到resource目录。
|
||||
echo.
|
||||
|
||||
REM 验证下载的文件完整性
|
||||
echo 正在验证下载文件...
|
||||
cd %PROJECT_DIR%\resource
|
||||
IF NOT EXIST "NeuralHaircut\diffusion_prior\model.pt" echo 警告:缺少扩散先验模型文件
|
||||
IF NOT EXIST "NeuralHaircut\PIXIE\pixie_data" echo 警告:缺少PIXIE模型数据
|
||||
IF NOT EXIST "Matte-Anything\sam_vit_h_4b8939.pth" echo 警告:缺少SAM模型文件
|
||||
IF NOT EXIST "Matte-Anything\groundingdino_swint_ogc.pth" echo 警告:缺少GroundingDINO模型文件
|
||||
IF NOT EXIST "openpose\models\pose\coco" echo 警告:缺少OpenPose姿态模型
|
||||
IF NOT EXIST "hyperIQA\pretrained\hyperIQA.pth" echo 警告:缺少hyperIQA模型文件
|
||||
|
||||
IF %ERRORLEVEL% NEQ 0 (
|
||||
echo.
|
||||
echo 错误:下载失败
|
||||
|
13
install.bat
13
install.bat
@ -214,7 +214,20 @@ xcopy /Y "%PROJECT_DIR%\resource\hyperIQA\pretrained\*" pretrained\
|
||||
REM 安装 PIXIE 环境
|
||||
CALL activate_pixie-env.bat
|
||||
cd %PROJECT_DIR%\ext\PIXIE
|
||||
echo 正在安装PIXIE依赖...
|
||||
pip install pyyaml==5.4.1
|
||||
pip install git+https://github.com/1adrianb/face-alignment.git@54623537fd9618ca7c15688fd85aba706ad92b59
|
||||
|
||||
REM 检查资源文件是否已下载
|
||||
IF NOT EXIST "resource" (
|
||||
echo 错误:未找到resource目录,请先运行download_resource.bat
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
REM 检查CUDA安装
|
||||
IF NOT EXIST "%CUDA_HOME%" (
|
||||
echo 错误:未找到CUDA安装目录,请确保CUDA 11.8已正确安装
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
echo Installation completed!
|
||||
|
@ -1,6 +1,6 @@
|
||||
numpy>=1.21.0,<2.0.0
|
||||
numpy>=1.21.0,<1.25.0
|
||||
scipy>=1.7.0,<2.0.0
|
||||
pillow>=8.3.1
|
||||
pillow>=9.0.0
|
||||
tqdm>=4.62.2
|
||||
matplotlib>=3.4.2
|
||||
opencv-python>=4.5.3
|
||||
@ -9,14 +9,14 @@ imageio>=2.9.0
|
||||
lpips>=0.1.4
|
||||
gdown>=4.7.1
|
||||
face-alignment>=1.3.5
|
||||
pytorch3d==0.7.5
|
||||
pytorch3d>=0.7.0
|
||||
ninja>=1.10.2
|
||||
trimesh>=3.9.35
|
||||
pyhocon>=0.3.59
|
||||
wandb>=0.12.0
|
||||
tensorboard>=2.6.0
|
||||
open3d>=0.13.0
|
||||
kaolin>=0.15.0
|
||||
kaolin>=0.13.0
|
||||
pytorch-lightning>=1.6.0
|
||||
plyfile>=0.8.1
|
||||
icecream>=2.1.3
|
||||
@ -34,3 +34,5 @@ torchdiffeq>=0.2.3
|
||||
torchsde>=0.2.5
|
||||
resize-right>=0.0.2
|
||||
colmap>=3.10
|
||||
torch>=1.12.0,<2.0.0
|
||||
torchvision>=0.13.0,<1.0.0
|
1
run.bat
1
run.bat
@ -312,6 +312,7 @@ IF %ERRORLEVEL% NEQ 0 (
|
||||
REM 制作视频
|
||||
CALL "%MICROMAMBA_EXE%" activate -p %MAMBA_ROOT_PREFIX%\envs\gaussian_splatting_hair
|
||||
cd %PROJECT_DIR%\src\postprocessing
|
||||
echo 正在生成最终视频...
|
||||
set CUDA_VISIBLE_DEVICES=%CUDA_VISIBLE_DEVICES%
|
||||
python concat_video.py --input_path "%DATA_PATH%" --exp_name_3 "%EXP_NAME_3%"
|
||||
IF %ERRORLEVEL% NEQ 0 (
|
||||
|
Loading…
Reference in New Issue
Block a user