CGNICO/GaussianHaircut
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update

Browse Source
This commit is contained in:
Jeffreytsai1004 2025-02-14 01:50:52 +08:00
parent b423614c1c
commit e9730a8612
14 changed files with 590 additions and 41 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.cursor/rules/project-rules.mdc
View File

@ -7,6 +7,7 @@ globs:
- 使用中文回复 - 使用中文回复
- 先查看Reference文件夹下的代码参考, 参考虚拟环境的创建和管理和使用,并按照 Readme.md 来查Readme和install.sh和run.sh以及Reference路径下参考代码来协助用户完成项目 - 先查看Reference文件夹下的代码参考, 参考虚拟环境的创建和管理和使用,并按照 Readme.md 来查Readme和install.sh和run.sh以及Reference路径下参考代码来协助用户完成项目
- 完善Readme.md
- Reference文件夹仅作为参考而不参与项目运行 - Reference文件夹仅作为参考而不参与项目运行
- 将该项目改成Windows平台应用, 代码必须考虑可在windows运行, 使用micromamba.exe来拿新建和管理虚拟环境 - 将该项目改成Windows平台应用, 代码必须考虑可在windows运行, 使用micromamba.exe来拿新建和管理虚拟环境
- 对于不同的环境分别创建activate的bat文件,虚拟环境分别有：gaussian_splatting_hair, matte_anything, openpose, pixie-env对应的文件有：activate_gaussian_splatting_hair.bat, activate_matte_anything.bat, activate_openpose.bat, activate_pixie-env.bat - 对于不同的环境分别创建activate的bat文件,虚拟环境分别有：gaussian_splatting_hair, matte_anything, openpose, pixie-env对应的文件有：activate_gaussian_splatting_hair.bat, activate_matte_anything.bat, activate_openpose.bat, activate_pixie-env.bat

105
README.md
View File

@ -1,26 +1,28 @@
# Gaussian Haircut: Human Hair Reconstruction with Strand-Aligned 3D Gaussians # Gaussian Haircut：使用股线对齐 3D 高斯模型进行人体头发重建
[**Paper**](https://arxiv.org/abs/2409.14778) | [**Project Page**](https://eth-ait.github.io/GaussianHaircut/) [**论文**](https://arxiv.org/abs/2409.14778) | [**项目页面**](https://eth-ait.github.io/GaussianHaircut/)
This repository contains an official implementation of Gaussian Haircut, a strand-based hair reconstruction approach for monocular videos. 本仓库包含了 Gaussian Haircut 的官方实现，这是一种基于股线的人体头发重建方法，用于单目视频。
## Getting started ## 入门指南
1. **Install CUDA 11.8** ### Linux 平台
Follow the instructions on https://developer.nvidia.com/cuda-11-8-0-download-archive. 1. **安装 CUDA 11.8**
Make sure that 按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。
- PATH includes <CUDA_DIR>/bin
- LD_LIBRARY_PATH includes <CUDA_DIR>/lib64
The environment was tested only with this CUDA version. 确保：
- PATH 包含 <CUDA_DIR>/bin
- LD_LIBRARY_PATH 包含 <CUDA_DIR>/lib64
2. **Install Blender 3.6** in order to create strand visualizations 该环境仅在此 CUDA 版本下进行了测试。
Follow instructions on https://www.blender.org/download/lts/3-6. 2. **安装 Blender 3.6** 以创建股线可视化
3. **Close the repo and run the install script** 按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。
3. **克隆仓库并运行安装脚本**
```bash ```bash
git clone git@github.com:eth-ait/GaussianHaircut.git git clone git@github.com:eth-ait/GaussianHaircut.git
@ -29,33 +31,71 @@ This repository contains an official implementation of Gaussian Haircut, a stran
./install.sh ./install.sh
``` ```
## Reconstruction ### Windows 平台
1. **Record a monocular video** 1. **安装 CUDA 11.8**
- 从 https://developer.nvidia.com/cuda-11-8-0-download-archive 下载并安装
- 默认安装路径：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
Use examples on the project page as references and introduce as little motion blur as possible. 2. **安装 Blender 3.6**
- 从 https://www.blender.org/download/lts/3-6 下载并安装
- 默认安装路径：C:\Program Files\Blender Foundation\Blender 3.6
2. **Setup a directory for the reconstructed scene** 3. **安装 Visual Studio 2019 Build Tools**
- 从 https://visualstudio.microsoft.com/vs/older-downloads/ 下载并安装
- 选择"C++构建工具"工作负载
- 默认安装路径：C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
Put the video file in it and rename it to raw.mp4 4. **安装 COLMAP**
- 从 https://github.com/colmap/colmap/releases 下载并安装
- 确保将COLMAP添加到系统PATH环境变量中
3. **Run the script** 5. **安装 micromamba**
- 从 https://mamba.readthedocs.io/en/latest/installation.html 下载并安装
6. **克隆仓库并运行安装脚本**
```cmd
git clone git@github.com:eth-ait/GaussianHaircut.git
cd GaussianHaircut
install.bat
```
## 重建
1. **录制单目视频**
参考项目页面上的示例，并尽可能减少运动模糊。
2. **设置重建场景的目录**
将视频文件放入其中，并将其重命名为 raw.mp4
3. **运行脚本**
Linux:
```bash ```bash
export PROJECT_DIR="[/path/to/]GaussianHaircut" export PROJECT_DIR="[/path/to/]GaussianHaircut"
export BLENDER_DIR="[/path/to/blender/folder/]blender" export BLENDER_DIR="[/path/to/blender/folder/]blender"
DATA_PATH="[path/to/scene/folder]" ./run.sh DATA_PATH="[path/to/scene/folder]" ./run.sh
``` ```
The script performs data pre-processing, reconstruction, and final visualization using Blender. Use Tensorboard to see intermediate visualizations. Windows:
```cmd
set PROJECT_DIR=[path\to\]GaussianHaircut
set DATA_PATH=[path\to\scene\folder]
run.bat
```
## License 该脚本执行数据预处理、重建以及使用 Blender 进行最终可视化。使用 Tensorboard 查看中间可视化结果。
This code is based on the 3D Gaussian Splatting project. For terms and conditions, please refer to LICENSE_3DGS. The rest of the code is distributed under CC BY-NC-SA 4.0. ## 许可证
If this code is helpful in your project, cite the papers below. 此代码基于 3D Gaussian Splatting 项目。有关条款和条件，请参阅 LICENSE_3DGS。其余代码根据 CC BY-NC-SA 4.0 分发。
## Citation 如果此代码对您的项目有帮助，请引用以下论文。
## 引用
``` ```
@inproceedings{zakharov2024gh, @inproceedings{zakharov2024gh,
@ -66,16 +106,11 @@ If this code is helpful in your project, cite the papers below.
} }
``` ```
## Links ## 链接
- [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting) - [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)
- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME 拟合管线、股线先验和发型扩散先验
- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME fitting pipeline, strand prior and hairstyle diffusion prior - [HAAR](https://github.com/Vanessik/HAAR): 头发上采样
- [Matte-Anything](https://github.com/hustvl/Matte-Anything): 头发和身体分割
- [HAAR](https://github.com/Vanessik/HAAR): hair upsampling - [PIXIE](https://github.com/yfeng95/PIXIE): FLAME 拟合的初始化
- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): 用于 FLAME 拟合的关键点检测
- [Matte-Anything](https://github.com/hustvl/Matte-Anything): hair and body segmentation
- [PIXIE](https://github.com/yfeng95/PIXIE): initialization for FLAME fitting
- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): keypoints detection for FLAME fitting

2
Reference/activate.bat Normal file
View File

@ -0,0 +1,2 @@
@CALL "%~dp0micromamba.exe" shell init --shell cmd.exe --prefix "%~dp0\"
start cmd /k "%~dp0condabin\micromamba.bat" activate StableDiffusion_001

32
Reference/install.bat Normal file
View File

@ -0,0 +1,32 @@
CALL "%~dp0micromamba.exe" create -n StableDiffusion_001 python==3.10.14 git==2.41.0 git-lfs==3.2.0 -c pytorch -c conda-forge -r "%~dp0\" -y
@CALL "%~dp0micromamba.exe" shell init --shell cmd.exe --prefix "%~dp0\"
@CALL condabin\micromamba.bat activate StableDiffusion_001
@CALL set GDOWN_CACHE=cache\gdown
@CALL set TORCH_HOME=cache\torch
@CALL set HF_HOME=cache\huggingface
@CALL set PYTHONDONTWRITEBYTECODE=1
@CALL set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
@CALL pip install --force-reinstall torch==2.2.0+cu121 torchvision==0.17.0+cu121 torchaudio==2.2.0+cu121 --index-url https://download.pytorch.org/whl/cu121 --no-cache-dir
@CALL git clone https://github.com/facebookresearch/xformers.git
@CALL cd xformers
@CALL git submodule update --init --recursive
@CALL pip install .
@CALL cd ..
@CALL git clone https://github.com/NVlabs/nvdiffrast .\nvdiffrast
@CALL cd nvdiffrast
@CALL pip install .
@CALL cd ..
@CALL git clone -b v1.9.4 https://github.com/AUTOMATIC1111/stable-diffusion-webui/ .\stable-diffusion-webui\
@CALL mkdir .\stable-diffusion-webui\cache\gdown\
@CALL mkdir .\stable-diffusion-webui\cache\torch\
@CALL mkdir .\stable-diffusion-webui\cache\huggingface\
@CALL mkdir .\stable-diffusion-webui\repositories\
@CALL git clone https://github.com/CompVis/stable-diffusion.git .\stable-diffusion-webui\repositories\stable-diffusion
@CALL git clone https://github.com/CompVis/taming-transformers.git .\stable-diffusion-webui\repositories\taming-transformers
@CALL git clone https://github.com/crowsonkb/k-diffusion.git .\stable-diffusion-webui\repositories\k-diffusion
@CALL git clone https://github.com/sczhou/CodeFormer.git .\stable-diffusion-webui\repositories\CodeFormer
@CALL git clone http://github.com/salesforce/BLIP.git .\stable-diffusion-webui\repositories\BLIP
@CALL cd .\stable-diffusion-webui
@CALL pip install -r .\stable-diffusion-webui\requirements.txt
@CALL python -B launch.py --xformers --opt-sdp-attention --autolaunch --theme dark --listen --port 7860 --enable-insecure-extension-access --skip-torch-cuda-test
@CALL PAUSE

3
Reference/micromamba.exe Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:77f9efb2c50a480086b93320829bb12cfd78ebaa9a5285a917b143e9b848378c
size 8381952

42
Reference/requirements.txt Normal file
View File

@ -0,0 +1,42 @@
pip==23.3.1
xformers==0.0.21
torch==2.0.1+cu118
torchvision
torchaudio
torchtext
PySoundFile
torchdiffeq==0.2.3
torchsde==0.2.5
GitPython==3.1.32
Pillow==9.5.0
accelerate==0.21.0
basicsr==1.4.2
blendmodes==2022
clean-fid==0.1.35
einops==0.4.1
fastapi==0.94.0
gfpgan==1.3.8
gradio==3.41.2
httpcore==0.15
inflection==0.5.1
jsonmerge==1.8.0
kornia==0.6.7
lark==1.1.2
numpy==1.23.5
omegaconf==2.2.3
open-clip-torch==2.20.0
piexif==1.1.3
psutil==5.9.5
pytorch_lightning==1.9.4
realesrgan==0.3.0
resize-right==0.0.2
safetensors==0.3.1
scikit-image==0.21.0
timm==0.9.2
tomesd==0.1.3
transformers==4.30.2
httpx==0.24.1
groundingdino
segment_anything
supervision
pyfunctional

11
Reference/run.bat Normal file
View File

@ -0,0 +1,11 @@
@CALL "%~dp0micromamba.exe" shell init --shell cmd.exe --prefix "%~dp0\"
@CALL condabin\micromamba.bat activate StableDiffusion_001
@CALL set GDOWN_CACHE=cache\gdown
@CALL set TORCH_HOME=cache\torch
@CALL set HF_HOME=cache\huggingface
@CALL set PYTHONDONTWRITEBYTECODE=1
@CALL set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
@CALL set COMMANDLINE_ARGS=--listen --port 7860 --autolaunch --theme dark --xformers --opt-sdp-attention --api --skip-install --skip-torch-cuda-test --skip-version-check --enable-insecure-extension-access
@CALL cd stable-diffusion-webui
@CALL python -B webui.py %COMMANDLINE_ARGS%
@CALL PAUSE

14
activate_gaussian_splatting_hair.bat
View File

@ -1 +1,13 @@
 @echo off
SET CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
SET PATH=%CUDA_HOME%\bin;%PATH%
SET PROJECT_DIR=%~dp0
SET PYTHONDONTWRITEBYTECODE=1
SET TORCH_HOME=%PROJECT_DIR%\cache\torch
SET HF_HOME=%PROJECT_DIR%\cache\huggingface
SET GDOWN_CACHE=%PROJECT_DIR%\cache\gdown
SET BLENDER_DIR="C:\Program Files\Blender Foundation\Blender 3.6"
SET VS2019_DIR="C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
SET VS2019_VCVARS="%VS2019_DIR%\VC\Auxiliary\Build\vcvars64.bat"
CALL micromamba activate gaussian_splatting_hair

14
activate_matte_anything.bat
View File

@ -1 +1,13 @@
 @echo off
SET CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
SET PATH=%CUDA_HOME%\bin;%PATH%
SET PROJECT_DIR=%~dp0
SET PYTHONDONTWRITEBYTECODE=1
SET TORCH_HOME=%PROJECT_DIR%\cache\torch
SET HF_HOME=%PROJECT_DIR%\cache\huggingface
SET GDOWN_CACHE=%PROJECT_DIR%\cache\gdown
SET BLENDER_DIR="C:\Program Files\Blender Foundation\Blender 3.6"
SET VS2019_DIR="C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
SET VS2019_VCVARS="%VS2019_DIR%\VC\Auxiliary\Build\vcvars64.bat"
CALL micromamba activate matte_anything

14
activate_openpose.bat
View File

@ -1 +1,13 @@
 @echo off
SET CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
SET PATH=%CUDA_HOME%\bin;%PATH%
SET PROJECT_DIR=%~dp0
SET PYTHONDONTWRITEBYTECODE=1
SET TORCH_HOME=%PROJECT_DIR%\cache\torch
SET HF_HOME=%PROJECT_DIR%\cache\huggingface
SET GDOWN_CACHE=%PROJECT_DIR%\cache\gdown
SET BLENDER_DIR="C:\Program Files\Blender Foundation\Blender 3.6"
SET VS2019_DIR="C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
SET VS2019_VCVARS="%VS2019_DIR%\VC\Auxiliary\Build\vcvars64.bat"
CALL micromamba activate openpose

14
activate_pixie-env.bat
View File

@ -1 +1,13 @@
 @echo off
SET CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
SET PATH=%CUDA_HOME%\bin;%PATH%
SET PROJECT_DIR=%~dp0
SET PYTHONDONTWRITEBYTECODE=1
SET TORCH_HOME=%PROJECT_DIR%\cache\torch
SET HF_HOME=%PROJECT_DIR%\cache\huggingface
SET GDOWN_CACHE=%PROJECT_DIR%\cache\gdown
SET BLENDER_DIR="C:\Program Files\Blender Foundation\Blender 3.6"
SET VS2019_DIR="C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
SET VS2019_VCVARS="%VS2019_DIR%\VC\Auxiliary\Build\vcvars64.bat"
CALL micromamba activate pixie-env

146
install.bat
View File

@ -1 +1,145 @@
 @echo off
setlocal enabledelayedexpansion
REM 设置环境变量
SET CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
SET PATH=%CUDA_HOME%\bin;%PATH%
SET PROJECT_DIR=%~dp0
SET PYTHONDONTWRITEBYTECODE=1
SET GDOWN_CACHE=cache\gdown
SET TORCH_HOME=cache\torch
SET HF_HOME=cache\huggingface
SET BLENDER_DIR=C:\Program Files\Blender Foundation\Blender 3.6
SET VS2019_DIR=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
SET VS2019_VCVARS=%VS2019_DIR%\VC\Auxiliary\Build\vcvars64.bat
REM 检查必要的环境和依赖
IF NOT EXIST "%CUDA_HOME%" (
echo ERROR: CUDA 11.8 not found at %CUDA_HOME%
exit /b 1
)
IF NOT EXIST "%BLENDER_DIR%" (
echo ERROR: Blender 3.6 not found at %BLENDER_DIR%
echo Please install Blender 3.6 from https://www.blender.org/download/lts/3-6/
exit /b 1
)
REM 检查Visual Studio
IF NOT EXIST "%VS2019_VCVARS%" (
echo ERROR: Visual Studio 2019 not found
echo Please install Visual Studio 2019 Build Tools with C++ development tools
exit /b 1
)
REM 检查COLMAP
where colmap >nul 2>nul
IF %ERRORLEVEL% NEQ 0 (
echo ERROR: COLMAP not found in PATH
echo Please install COLMAP and add it to PATH
exit /b 1
)
REM 创建缓存目录
mkdir cache\gdown
mkdir cache\torch
mkdir cache\huggingface
REM 创建ext目录
mkdir ext
cd ext
REM 克隆外部库
git clone https://github.com/CMU-Perceptual-Computing-Lab/openpose --depth 1
cd openpose
git submodule update --init --recursive --remote
cd ..
git clone https://github.com/hustvl/Matte-Anything
cd Matte-Anything
git clone https://github.com/IDEA-Research/GroundingDINO.git
cd ..
git clone https://github.com/egorzakharov/NeuralHaircut.git --recursive
git clone https://github.com/facebookresearch/pytorch3d
cd pytorch3d
git checkout 2f11ddc5ee7d6bd56f2fb6744a16776fab6536f7
cd ..
git clone https://github.com/camenduru/simple-knn
cd diff_gaussian_rasterization_hair\third_party
git clone https://github.com/g-truc/glm
cd glm
git checkout 5c46b9c07008ae65cb81ab79cd677ecc1934b903
cd ..\..\..
git clone --recursive https://github.com/NVIDIAGameWorks/kaolin
cd kaolin
git checkout v0.15.0
cd ..
git clone https://github.com/SSL92/hyperIQA
REM 创建环境
CALL micromamba create -n gaussian_splatting_hair python=3.8 pytorch=2.0.0 torchvision pytorch-cuda=11.8 cmake ninja -c pytorch -c nvidia -c conda-forge -y
CALL micromamba create -n matte_anything pytorch=2.0.0 pytorch-cuda=11.8 torchvision tensorboard timm=0.5.4 opencv=4.5.3 mkl=2024.0 setuptools=58.2.0 easydict wget scikit-image gradio=3.46.1 fairscale -c pytorch -c nvidia -c conda-forge -y
CALL micromamba create -n openpose python=3.8 cmake=3.20 -c conda-forge -y
CALL micromamba create -n pixie-env python=3.8 pytorch=2.0.0 torchvision pytorch-cuda=11.8 fvcore pytorch3d==0.7.5 kornia matplotlib -c pytorch -c nvidia -c fvcore -c conda-forge -c pytorch3d -y
REM 安装 gaussian_splatting_hair 环境
CALL activate_gaussian_splatting_hair.bat
pip install -r requirements.txt
cd %PROJECT_DIR%\ext\pytorch3d
pip install -e .
cd %PROJECT_DIR%\ext\NeuralHaircut\npbgpp
pip install -e .
cd %PROJECT_DIR%\ext\simple-knn
pip install -e .
cd %PROJECT_DIR%\ext\diff_gaussian_rasterization_hair
pip install -e .
cd %PROJECT_DIR%\ext\kaolin
pip install -e .
cd %PROJECT_DIR%
REM 下载 Neural Haircut 文件
cd %PROJECT_DIR%\ext\NeuralHaircut
gdown --folder https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8
cd pretrained_models\diffusion_prior
gdown 1_9EOUXHayKiGH5nkrayncln3d6m1uV7f
cd ..\..\PIXIE
gdown 1mPcGu62YPc4MdkT8FFiOCP629xsENHZf
tar -xf pixie_data.tar.gz
del pixie_data.tar.gz
REM 安装 Matte-Anything 环境
CALL activate_matte_anything.bat
cd %PROJECT_DIR%\ext\Matte-Anything
pip install git+https://github.com/facebookresearch/segment-anything.git
pip install 'git+https://github.com/facebookresearch/detectron2.git'
pip install -e GroundingDINO
pip install supervision==0.22.0
mkdir pretrained
cd pretrained
curl -LO https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
curl -LO https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
cd ..
gdown 1d97oKuITCeWgai2Tf3iNilt6rMSSYzkW
REM 安装 OpenPose
CALL activate_openpose.bat
cd %PROJECT_DIR%\ext\openpose
gdown 1Yn03cKKfVOq4qXmgBMQD20UMRRRkd_tV
tar -xf models.tar.gz
del models.tar.gz
mkdir build
cd build
CALL "%VS2019_VCVARS%"
cmake .. -DBUILD_PYTHON=true -DUSE_CUDNN=off -G "Visual Studio 16 2019" -A x64
cmake --build . --config Release
REM 安装 PIXIE 环境
CALL activate_pixie-env.bat
cd %PROJECT_DIR%\ext\PIXIE
pip install pyyaml==5.4.1
pip install git+https://github.com/1adrianb/face-alignment.git@54623537fd9618ca7c15688fd85aba706ad92b59
echo Installation completed!

35
requirements.txt Normal file
View File

@ -0,0 +1,35 @@
numpy>=1.21.0
scipy>=1.7.0
pillow>=8.3.1
tqdm>=4.62.2
matplotlib>=3.4.2
opencv-python>=4.5.3
scikit-image>=0.18.2
imageio>=2.9.0
lpips>=0.1.4
gdown>=4.7.1
face-alignment>=1.3.5
pytorch3d==0.7.5
ninja>=1.10.2
trimesh>=3.9.35
pyhocon>=0.3.59
wandb>=0.12.0
tensorboard>=2.6.0
open3d>=0.13.0
kaolin>=0.15.0
pytorch-lightning>=1.6.0
plyfile>=0.8.1
icecream>=2.1.3
einops>=0.6.0
accelerate>=0.18.0
jsonmerge>=1.9.0
easydict>=1.9
tensorboardx>=2.6
fvcore>=0.1.5
toml>=0.10.2
pysdf>=0.1.9
clean-fid>=0.1.35
clip>=0.2.0
torchdiffeq>=0.2.3
torchsde>=0.2.5
resize-right>=0.0.2

198
run.bat
View File

@ -1 +1,197 @@
 @echo off
setlocal enabledelayedexpansion
REM 设置环境变量
SET GPU=0
SET CAMERA=PINHOLE
SET EXP_NAME_1=stage1
SET EXP_NAME_2=stage2
SET EXP_NAME_3=stage3
SET BLENDER_DIR="C:\Program Files\Blender Foundation\Blender 3.6"
SET CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
REM 确保以下环境变量可用:
REM PROJECT_DIR 和 DATA_PATH
REM 检查必要的环境变量
IF "%PROJECT_DIR%"=="" (
echo ERROR: PROJECT_DIR environment variable is not set
exit /b 1
)
IF "%DATA_PATH%"=="" (
echo ERROR: DATA_PATH environment variable is not set
exit /b 1
)
IF NOT EXIST "%BLENDER_DIR%" (
echo ERROR: BLENDER_DIR path does not exist: %BLENDER_DIR%
exit /b 1
)
REM ##################
REM # 预处理阶段 #
REM ##################
REM 将原始图像整理成3D Gaussian Splatting格式
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python preprocess_raw_images.py --data_path %DATA_PATH%
REM 运行COLMAP重建并去畸变图像和相机
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src
set CUDA_VISIBLE_DEVICES=%GPU%
python convert.py -s %DATA_PATH% --camera %CAMERA% --max_size 1024
REM 运行Matte-Anything
CALL activate_matte_anything.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python calc_masks.py --data_path %DATA_PATH% --image_format png --max_size 2048
REM 使用IQA分数过滤图像
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python filter_extra_images.py --data_path %DATA_PATH% --max_imgs 128
REM 调整图像大小
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python resize_images.py --data_path %DATA_PATH%
REM 计算方向图
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python calc_orientation_maps.py --img_path %DATA_PATH%\images_2 --mask_path %DATA_PATH%\masks_2\hair --orient_dir %DATA_PATH%\orientations_2\angles --conf_dir %DATA_PATH%\orientations_2\vars --filtered_img_dir %DATA_PATH%\orientations_2\filtered_imgs --vis_img_dir %DATA_PATH%\orientations_2\vis_imgs
REM 运行OpenPose
CALL activate_openpose.bat
cd %PROJECT_DIR%\ext\openpose
mkdir %DATA_PATH%\openpose
set CUDA_VISIBLE_DEVICES=%GPU%
"%PROJECT_DIR%\ext\openpose\build\x64\Release\OpenPoseDemo.exe" --image_dir %DATA_PATH%\images_4 --scale_number 4 --scale_gap 0.25 --face --hand --display 0 --write_json %DATA_PATH%\openpose\json --write_images %DATA_PATH%\openpose\images --write_images_format jpg
REM 运行Face-Alignment
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python calc_face_alignment.py --data_path %DATA_PATH% --image_dir "images_4"
REM 运行PIXIE
CALL activate_pixie-env.bat
cd %PROJECT_DIR%\ext\PIXIE
set CUDA_VISIBLE_DEVICES=%GPU%
python demos\demo_fit_face.py -i %DATA_PATH%\images_4 -s %DATA_PATH%\pixie --saveParam True --lightTex False --useTex False --rasterizer_type pytorch3d
REM 合并所有PIXIE预测到单个文件
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python merge_smplx_predictions.py --data_path %DATA_PATH%
REM 将COLMAP相机转换为txt
CALL activate_gaussian_splatting_hair.bat
mkdir %DATA_PATH%\sparse_txt
set CUDA_VISIBLE_DEVICES=%GPU%
colmap model_converter --input_path %DATA_PATH%\sparse\0 --output_path %DATA_PATH%\sparse_txt --output_type TXT
REM 将COLMAP相机转换为H3DS格式
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python colmap_parsing.py --path_to_scene %DATA_PATH%
REM 删除原始文件以节省磁盘空间
rmdir /s /q %DATA_PATH%\input %DATA_PATH%\images %DATA_PATH%\masks %DATA_PATH%\iqa*
REM ##################
REM # 重建阶段 #
REM ##################
set EXP_PATH_1=%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%
REM 运行3D Gaussian Splatting重建
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src
set CUDA_VISIBLE_DEVICES=%GPU%
python train_gaussians.py -s %DATA_PATH% -m "%EXP_PATH_1%" -r 1 --port "888%GPU%" --trainable_cameras --trainable_intrinsics --use_barf --lambda_dorient 0.1
REM 运行FLAME网格拟合
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\ext\NeuralHaircut\src\multiview_optimization
set CUDA_VISIBLE_DEVICES=%GPU%
python fit.py --conf confs\train_person_1.conf --batch_size 1 --train_rotation True --fixed_images True --save_path %DATA_PATH%\flame_fitting\%EXP_NAME_1%\stage_1 --data_path %DATA_PATH% --fitted_camera_path %EXP_PATH_1%\cameras\30000_matrices.pkl
python fit.py --conf confs\train_person_1.conf --batch_size 4 --train_rotation True --fixed_images True --save_path %DATA_PATH%\flame_fitting\%EXP_NAME_1%\stage_2 --checkpoint_path %DATA_PATH%\flame_fitting\%EXP_NAME_1%\stage_1\opt_params_final --data_path %DATA_PATH% --fitted_camera_path %EXP_PATH_1%\cameras\30000_matrices.pkl
python fit.py --conf confs\train_person_1_.conf --batch_size 32 --train_rotation True --train_shape True --save_path %DATA_PATH%\flame_fitting\%EXP_NAME_1%\stage_3 --checkpoint_path %DATA_PATH%\flame_fitting\%EXP_NAME_1%\stage_2\opt_params_final --data_path %DATA_PATH% --fitted_camera_path %EXP_PATH_1%\cameras\30000_matrices.pkl
REM 裁剪重建的场景
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python scale_scene_into_sphere.py --path_to_data %DATA_PATH% -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%" --iter 30000
REM 移除与FLAME头部网格相交的头发高斯体
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python filter_flame_intersections.py --flame_mesh_dir %DATA_PATH%\flame_fitting\%EXP_NAME_1% -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%" --iter 30000 --project_dir %PROJECT_DIR%\ext\NeuralHaircut
REM 为训练视图运行渲染
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src
set CUDA_VISIBLE_DEVICES=%GPU%
python render_gaussians.py -s %DATA_PATH% -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%" --skip_test --scene_suffix "_cropped" --iteration 30000 --trainable_cameras --trainable_intrinsics --use_barf
REM 获取FLAME网格头皮图
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python extract_non_visible_head_scalp.py --project_dir %PROJECT_DIR%\ext\NeuralHaircut --data_dir %DATA_PATH% --flame_mesh_dir %DATA_PATH%\flame_fitting\%EXP_NAME_1% --cams_path %DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%\cameras\30000_matrices.pkl -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%"
REM 运行潜在头发股线重建
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src
set CUDA_VISIBLE_DEVICES=%GPU%
python train_latent_strands.py -s %DATA_PATH% -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%" -r 1 --model_path_hair "%DATA_PATH%\strands_reconstruction\%EXP_NAME_2%" --flame_mesh_dir "%DATA_PATH%\flame_fitting\%EXP_NAME_1%" --pointcloud_path_head "%EXP_PATH_1%\point_cloud_filtered\iteration_30000\raw_point_cloud.ply" --hair_conf_path "%PROJECT_DIR%\src\arguments\hair_strands_textured.yaml" --lambda_dmask 0.1 --lambda_dorient 0.1 --lambda_dsds 0.01 --load_synthetic_rgba --load_synthetic_geom --binarize_masks --iteration_data 30000 --trainable_cameras --trainable_intrinsics --use_barf --iterations 20000 --port "800%GPU%"
REM 运行头发股线重建
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src
set CUDA_VISIBLE_DEVICES=%GPU%
python train_strands.py -s %DATA_PATH% -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%" -r 1 --model_path_curves "%DATA_PATH%\curves_reconstruction\%EXP_NAME_3%" --flame_mesh_dir "%DATA_PATH%\flame_fitting\%EXP_NAME_1%" --pointcloud_path_head "%EXP_PATH_1%\point_cloud_filtered\iteration_30000\raw_point_cloud.ply" --start_checkpoint_hair "%DATA_PATH%\strands_reconstruction\%EXP_NAME_2%\checkpoints\20000.pth" --hair_conf_path "%PROJECT_DIR%\src\arguments\hair_strands_textured.yaml" --lambda_dmask 0.1 --lambda_dorient 0.1 --lambda_dsds 0.01 --load_synthetic_rgba --load_synthetic_geom --binarize_masks --iteration_data 30000 --position_lr_init 0.0000016 --position_lr_max_steps 10000 --trainable_cameras --trainable_intrinsics --use_barf --iterations 10000 --port "800%GPU%"
rmdir /s /q "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%\train_cropped"
REM ##################
REM # 可视化阶段 #
REM ##################
REM 导出结果的股线为pkl和ply格式
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\preprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python export_curves.py --data_dir %DATA_PATH% --model_name %EXP_NAME_3% --iter 10000 --flame_mesh_path "%DATA_PATH%\flame_fitting\%EXP_NAME_1%\stage_3\mesh_final.obj" --scalp_mesh_path "%DATA_PATH%\flame_fitting\%EXP_NAME_1%\scalp_data\scalp.obj" --hair_conf_path "%PROJECT_DIR%\src\arguments\hair_strands_textured.yaml"
REM 渲染可视化
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\postprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python render_video.py --blender_path "%BLENDER_DIR%" --input_path "%DATA_PATH%" --exp_name_1 "%EXP_NAME_1%" --exp_name_3 "%EXP_NAME_3%"
REM 渲染股线
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src
set CUDA_VISIBLE_DEVICES=%GPU%
python render_strands.py -s %DATA_PATH% --data_dir "%DATA_PATH%" --data_device 'cpu' --skip_test -m "%DATA_PATH%\3d_gaussian_splatting\%EXP_NAME_1%" --iteration 30000 --flame_mesh_dir "%DATA_PATH%\flame_fitting\%EXP_NAME_1%" --model_hair_path "%DATA_PATH%\curves_reconstruction\%EXP_NAME_3%" --hair_conf_path "%PROJECT_DIR%\src\arguments\hair_strands_textured.yaml" --checkpoint_hair "%DATA_PATH%\strands_reconstruction\%EXP_NAME_2%\checkpoints\20000.pth" --checkpoint_curves "%DATA_PATH%\curves_reconstruction\%EXP_NAME_3%\checkpoints\10000.pth" --pointcloud_path_head "%EXP_PATH_1%\point_cloud\iteration_30000\raw_point_cloud.ply" --interpolate_cameras
REM 制作视频
CALL activate_gaussian_splatting_hair.bat
cd %PROJECT_DIR%\src\postprocessing
set CUDA_VISIBLE_DEVICES=%GPU%
python concat_video.py --input_path "%DATA_PATH%" --exp_name_3 "%EXP_NAME_3%"