From cdc0d75a8a33ce2ce3aca1ff10f2db32e6b751ee Mon Sep 17 00:00:00 2001 From: Jeffreytsai1004 Date: Sat, 15 Feb 2025 21:12:05 +0800 Subject: [PATCH] Update --- README.md | 128 +++++++++++++++++++++--------------------- README_CN.md | 48 +++++++++++++++- download_resource.bat | 49 ++++++++++++---- install.bat | 13 +++++ requirements.txt | 12 ++-- run.bat | 1 + 6 files changed, 170 insertions(+), 81 deletions(-) diff --git a/README.md b/README.md index fa44814..37d39e3 100644 --- a/README.md +++ b/README.md @@ -41,11 +41,32 @@ Expected output: └── visualization/ # Rendering results and video ``` -## 入门指南 +Required resource structure: +``` +resource/ +├── NeuralHaircut/ +│ ├── diffusion_prior/ +│ │ └── model.pt # Diffusion prior model +│ └── PIXIE/ +│ └── pixie_data/ # PIXIE model data +├── Matte-Anything/ +│ ├── sam_vit_h_4b8939.pth # SAM model +│ ├── groundingdino_swint_ogc.pth # GroundingDINO model +│ └── model.pth # Matte-Anything model +├── openpose/ +│ └── models/ +│ ├── pose/ # OpenPose pose models +│ └── face/ # OpenPose face models +└── hyperIQA/ + └── pretrained/ + └── hyperIQA.pth # Image quality assessment model +``` -### Linux 平台 +## Getting Started -1. **安装 CUDA 11.8** +### Linux Platform + +1. **Install CUDA 11.8** 按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。 @@ -55,11 +76,11 @@ Expected output: 该环境仅在此 CUDA 版本下进行了测试。 -2. **安装 Blender 3.6** 以创建股线可视化 +2. **Install Blender 3.6** 以创建股线可视化 按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。 -3. **克隆仓库并运行安装脚本** +3. **Clone repository and run installation script** ```bash git clone git@github.com:eth-ait/GaussianHaircut.git @@ -68,23 +89,23 @@ Expected output: ./install.sh ``` -### Windows 平台 +### Windows Platform -1. **安装 CUDA 11.8** +1. **Install CUDA 11.8** - 从 https://developer.nvidia.com/cuda-11-8-0-download-archive 下载并安装 - 默认安装路径：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 - 确保CUDA版本与系统兼容 -2. **安装 Blender 3.6** +2. **Install Blender 3.6** - 从 https://www.blender.org/download/lts/3-6 下载并安装 - 默认安装路径：C:\Program Files\Blender Foundation\Blender 3.6 -3. **安装 Visual Studio 2019 Build Tools** +3. **Install Visual Studio 2019 Build Tools** - 从 https://visualstudio.microsoft.com/vs/older-downloads/ 下载并安装 - 选择"C++构建工具"工作负载 - 默认安装路径：C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools -4. **安装 COLMAP** +4. **Install COLMAP** - 从 https://github.com/colmap/colmap/releases 下载并安装 - 下载CUDA版本的COLMAP (例如：COLMAP-3.8-windows-cuda.zip) - 解压到不含空格的路径 (例如：C:\COLMAP) @@ -96,7 +117,7 @@ Expected output: 5. 点击"确定"保存 - 重启终端使PATH生效 -5. **安装 7-Zip** +5. **Install 7-Zip** - 从 https://7-zip.org/ 下载并安装 - 将7-Zip安装目录添加到系统PATH: 1. 打开"系统属性" > "环境变量" @@ -106,7 +127,7 @@ Expected output: 5. 点击"确定"保存 - 重启终端使PATH生效 -6. **下载预训练模型和资源** +6. **Download pre-trained models and resources** ```cmd git clone https://github.com/Jeffreytsai1004/GaussianHairCut cd GaussianHairCut @@ -119,7 +140,7 @@ Expected output: - 如果下载失败，可以重新运行脚本 - 确保有稳定的网络连接 -6. **克隆仓库并运行安装脚本** +6. **Clone repository and run installation script** ```cmd git clone https://github.com/Jeffreytsai1004/GaussianHairCut cd GaussianHairCut @@ -131,66 +152,47 @@ Expected output: .\run.bat ``` -## 重建 +## Usage -1. **录制单目视频** +1. **Record Monocular Video** - - 参考项目页面上的示例视频 - - 录制要求： - * 拍摄对象应缓慢转动头部，确保捕捉到所有角度 - * 保持头发和面部清晰可见 - * 避免快速移动导致的运动模糊 - * 保持光照条件稳定 - * 建议视频长度：10-20秒 - * 建议分辨率：1920x1080或更高 + - Reference example videos on the project page + - Recording requirements: + * Subject should rotate head slowly to capture all angles + * Keep hair and face clearly visible + * Avoid motion blur from fast movements + * Maintain stable lighting conditions + * Recommended length: 10-20 seconds + * Recommended resolution: 1920x1080 or higher -2. **设置重建场景的目录** +2. **Setup Scene Directory** - 创建场景目录结构： - ``` - [your_scene_folder]/ - └── raw.mp4 # 您录制的视频文件 - ``` - - 注意： - - DATA_PATH 应指向包含 raw.mp4 的目录 - - 目录路径不应包含空格或特殊字符 - - 脚本会在此目录下自动创建所需的子目录 - - 确保有足够的磁盘空间(建议至少20GB) - -3. **运行脚本** - -Linux: - ```bash - export PROJECT_DIR="[/path/to/]GaussianHaircut" - export BLENDER_DIR="[/path/to/blender/folder/]blender" - DATA_PATH="[path/to/scene/folder]" ./run.sh - ``` - -Windows: ```cmd - # 在CMD中运行: + # In CMD: set PROJECT_DIR=[path\to\]GaussianHaircut set DATA_PATH=[path\to\scene\folder] run.bat - # 或在PowerShell中运行: - set PROJECT_DIR=[path\to\]GaussianHaircut - set DATA_PATH=[path\to\scene\folder] + # Or in PowerShell: + $env:PROJECT_DIR="[path\to\]GaussianHaircut" + $env:DATA_PATH="[path\to\scene\folder]" .\run.bat ``` + + Note: + - DATA_PATH should point to directory containing raw.mp4 + - Directory paths should not contain spaces or special characters + - Ensure sufficient disk space (at least 20GB recommended) - 该脚本执行数据预处理、重建以及使用 Blender 进行最终可视化。使用 Tensorboard 查看中间可视化结果。 +## License -## 许可证 +This code is based on the 3D Gaussian Splatting project. See LICENSE_3DGS for terms and conditions. The rest of the code is distributed under CC BY-NC-SA 4.0. -此代码基于 3D Gaussian Splatting 项目。有关条款和条件，请参阅 LICENSE_3DGS。其余代码根据 CC BY-NC-SA 4.0 分发。 +## Citation -如果此代码对您的项目有帮助，请引用以下论文。 +If you find this code helpful for your research, please cite our paper: -## 引用 - -``` +```bibtex @inproceedings{zakharov2024gh, title = {Human Hair Reconstruction with Strand-Aligned 3D Gaussians}, author = {Zakharov, Egor and Sklyarova, Vanessa and Black, Michael J and Nam, Giljoo and Thies, Justus and Hilliges, Otmar}, @@ -199,11 +201,11 @@ Windows: } ``` -## 链接 +## Related Projects - [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting) -- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME 拟合管线、股线先验和发型扩散先验 -- [HAAR](https://github.com/Vanessik/HAAR): 头发上采样 -- [Matte-Anything](https://github.com/hustvl/Matte-Anything): 头发和身体分割 -- [PIXIE](https://github.com/yfeng95/PIXIE): FLAME 拟合的初始化 -- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): 用于 FLAME 拟合的关键点检测 +- [Neural Haircut](https://github.com/SamsungLabs/NeuralHaircut): FLAME fitting pipeline, strand prior and hairstyle diffusion prior +- [HAAR](https://github.com/Vanessik/HAAR): Hair upsampling +- [Matte-Anything](https://github.com/hustvl/Matte-Anything): Hair and body segmentation +- [PIXIE](https://github.com/yfeng95/PIXIE): FLAME fitting initialization +- [Face-Alignment](https://github.com/1adrianb/face-alignment), [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): Keypoint detection for FLAME fitting diff --git a/README_CN.md b/README_CN.md index 6bc5e75..0a47ae5 100644 --- a/README_CN.md +++ b/README_CN.md @@ -41,7 +41,53 @@ └── visualization/ # 渲染结果和视频 ``` -## 安装指南 +所需资源文件结构： +``` +resource/ +├── NeuralHaircut/ +│ ├── diffusion_prior/ +│ │ └── model.pt # 扩散先验模型 +│ └── PIXIE/ +│ └── pixie_data/ # PIXIE模型数据 +├── Matte-Anything/ +│ ├── sam_vit_h_4b8939.pth # SAM模型 +│ ├── groundingdino_swint_ogc.pth # GroundingDINO模型 +│ └── model.pth # Matte-Anything模型 +├── openpose/ +│ └── models/ +│ ├── pose/ # OpenPose姿态模型 +│ └── face/ # OpenPose人脸模型 +└── hyperIQA/ + └── pretrained/ + └── hyperIQA.pth # 图像质量评估模型 +``` + +## 环境配置 + +### Linux 平台 + +1. **安装 CUDA 11.8** + + 按照 https://developer.nvidia.com/cuda-11-8-0-download-archive 上的说明进行操作。 + + 确保： + - PATH 包含 /bin + - LD_LIBRARY_PATH 包含 /lib64 + + 该环境仅在此 CUDA 版本下进行了测试。 + +2. **安装 Blender 3.6** 以创建股线可视化 + + 按照 https://www.blender.org/download/lts/3-6 上的说明进行操作。 + +3. **克隆仓库并运行安装脚本** + + ```bash + git clone git@github.com:eth-ait/GaussianHaircut.git + cd GaussianHaircut + chmod +x ./install.sh + ./install.sh + ``` ### Windows 平台 diff --git a/download_resource.bat b/download_resource.bat index f34ea88..5fb9a20 100644 --- a/download_resource.bat +++ b/download_resource.bat @@ -25,14 +25,20 @@ IF %ERRORLEVEL% NEQ 0 ( REM 更新pip python -m pip install --upgrade pip -REM 安装gdown -echo 正在安装gdown... -python -m pip install --user gdown --upgrade -python -m pip install --user requests --upgrade -IF %ERRORLEVEL% NEQ 0 ( - echo 错误：gdown安装失败 - echo 请检查网络连接后重试 - exit /b 1 +REM 检查gdown是否已安装 +python -c "import gdown" >nul 2>nul +IF %ERRORLEVEL% EQU 0 ( + echo gdown已安装，跳过安装步骤... +) ELSE ( + REM 安装gdown + echo 正在安装gdown... + python -m pip install --user gdown --upgrade + python -m pip install --user requests --upgrade + IF %ERRORLEVEL% NEQ 0 ( + echo 错误：gdown安装失败 + echo 请检查网络连接后重试 + exit /b 1 + ) ) REM 添加Python Scripts目录到PATH @@ -61,6 +67,11 @@ mkdir Matte-Anything 2>nul mkdir openpose\models 2>nul mkdir hyperIQA\pretrained 2>nul +REM 设置下载重试次数和超时时间 +SET RETRY_COUNT=3 +SET TIMEOUT_SECONDS=600 +SET CURL_OPTS=--connect-timeout 30 --max-time %TIMEOUT_SECONDS% --retry %RETRY_COUNT% -L -C - + echo ************************************************************ echo * 开始下载所需资源文件 * echo * 下载时间取决于您的网络连接速度 * @@ -73,8 +84,8 @@ echo 正在下载Neural Haircut文件... IF EXIST "diffusion_prior\model.pt" ( echo Neural Haircut模型已存在，跳过下载... ) ELSE ( - echo 正在下载Neural Haircut文件...· - python -m gdown --folder "https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8" + echo 正在下载Neural Haircut文件... + python -m gdown --folder "https://drive.google.com/drive/folders/1TCdJ0CKR3Q6LviovndOkJaKm8S1T9F_8" --continue ) cd diffusion_prior @@ -110,13 +121,17 @@ IF EXIST "sam_vit_h_4b8939.pth" ( echo SAM模型已存在，跳过下载... ) ELSE ( echo 正在下载SAM模型... - curl -L --retry 5 -o sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth + curl %CURL_OPTS% -o sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth + IF %ERRORLEVEL% NEQ 0 ( + echo 错误：下载SAM模型失败 + exit /b 1 + ) ) IF EXIST "groundingdino_swint_ogc.pth" ( echo GroundingDINO模型已存在，跳过下载... ) ELSE ( echo 正在下载GroundingDINO模型... - curl -L --retry 5 -o groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth + curl -L --retry 5 -C - -o groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth ) IF EXIST "model.pth" ( echo Matte-Anything模型已存在，跳过下载... @@ -159,6 +174,16 @@ echo 下载完成！ echo 资源文件已下载到resource目录。 echo. +REM 验证下载的文件完整性 +echo 正在验证下载文件... +cd %PROJECT_DIR%\resource +IF NOT EXIST "NeuralHaircut\diffusion_prior\model.pt" echo 警告：缺少扩散先验模型文件 +IF NOT EXIST "NeuralHaircut\PIXIE\pixie_data" echo 警告：缺少PIXIE模型数据 +IF NOT EXIST "Matte-Anything\sam_vit_h_4b8939.pth" echo 警告：缺少SAM模型文件 +IF NOT EXIST "Matte-Anything\groundingdino_swint_ogc.pth" echo 警告：缺少GroundingDINO模型文件 +IF NOT EXIST "openpose\models\pose\coco" echo 警告：缺少OpenPose姿态模型 +IF NOT EXIST "hyperIQA\pretrained\hyperIQA.pth" echo 警告：缺少hyperIQA模型文件 + IF %ERRORLEVEL% NEQ 0 ( echo. echo 错误：下载失败 diff --git a/install.bat b/install.bat index cb68a43..4ac45fa 100644 --- a/install.bat +++ b/install.bat @@ -214,7 +214,20 @@ xcopy /Y "%PROJECT_DIR%\resource\hyperIQA\pretrained\*" pretrained\ REM 安装 PIXIE 环境 CALL activate_pixie-env.bat cd %PROJECT_DIR%\ext\PIXIE +echo 正在安装PIXIE依赖... pip install pyyaml==5.4.1 pip install git+https://github.com/1adrianb/face-alignment.git@54623537fd9618ca7c15688fd85aba706ad92b59 +REM 检查资源文件是否已下载 +IF NOT EXIST "resource" ( + echo 错误：未找到resource目录，请先运行download_resource.bat + exit /b 1 +) + +REM 检查CUDA安装 +IF NOT EXIST "%CUDA_HOME%" ( + echo 错误：未找到CUDA安装目录，请确保CUDA 11.8已正确安装 + exit /b 1 +) + echo Installation completed! diff --git a/requirements.txt b/requirements.txt index b686e71..0d90041 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,6 @@ -numpy>=1.21.0,<2.0.0 +numpy>=1.21.0,<1.25.0 scipy>=1.7.0,<2.0.0 -pillow>=8.3.1 +pillow>=9.0.0 tqdm>=4.62.2 matplotlib>=3.4.2 opencv-python>=4.5.3 @@ -9,14 +9,14 @@ imageio>=2.9.0 lpips>=0.1.4 gdown>=4.7.1 face-alignment>=1.3.5 -pytorch3d==0.7.5 +pytorch3d>=0.7.0 ninja>=1.10.2 trimesh>=3.9.35 pyhocon>=0.3.59 wandb>=0.12.0 tensorboard>=2.6.0 open3d>=0.13.0 -kaolin>=0.15.0 +kaolin>=0.13.0 pytorch-lightning>=1.6.0 plyfile>=0.8.1 icecream>=2.1.3 @@ -33,4 +33,6 @@ clip>=0.2.0 torchdiffeq>=0.2.3 torchsde>=0.2.5 resize-right>=0.0.2 -colmap>=3.10 \ No newline at end of file +colmap>=3.10 +torch>=1.12.0,<2.0.0 +torchvision>=0.13.0,<1.0.0 \ No newline at end of file diff --git a/run.bat b/run.bat index 2f1d122..e1bdc0d 100644 --- a/run.bat +++ b/run.bat @@ -312,6 +312,7 @@ IF %ERRORLEVEL% NEQ 0 ( REM 制作视频 CALL "%MICROMAMBA_EXE%" activate -p %MAMBA_ROOT_PREFIX%\envs\gaussian_splatting_hair cd %PROJECT_DIR%\src\postprocessing +echo 正在生成最终视频... set CUDA_VISIBLE_DEVICES=%CUDA_VISIBLE_DEVICES% python concat_video.py --input_path "%DATA_PATH%" --exp_name_3 "%EXP_NAME_3%" IF %ERRORLEVEL% NEQ 0 (