开源软件名称：

ai_choreographer

开源软件地址：

https://gitee.com/mirrors/ai_choreographer

开源软件介绍：

AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ [ICCV-2021].

Overview

This package contains the model implementation and training infrastructure ofour AI Choreographer.

Get started

Pull the code

git clone https://github.com/liruilong940607/mint --recursive

Note here --recursive is important as it will automatically clone the submodule (orbit) as well.

Install dependencies

conda create -n mint python=3.7conda activate mintconda install protobuf numpypip install tensorflow absl-py tensorflow-datasets librosasudo apt-get install libopenexr-devpip install --upgrade OpenEXRpip install tensorflow-graphics tensorflow-graphics-gpugit clone https://github.com/arogozhnikov/einops /tmp/einopscd /tmp/einops/ && pip install . -Ugit clone https://github.com/google/aistplusplus_api /tmp/aistplusplus_apicd /tmp/aistplusplus_api && pip install -r requirements.txt && pip install . -U

Note if you meet environment conflicts about numpy, you can try with pip install numpy==1.20.

Get the data

See the website

Get the checkpoint

Download from google drive here, and put them to the folder ./checkpoints/

Run the code

complie protocols

protoc ./mint/protos/*.proto

preprocess dataset into tfrecord

python tools/preprocessing.py \    --anno_dir="/mnt/data/aist_plusplus_final/" \    --audio_dir="/mnt/data/AIST/music/" \    --split=trainpython tools/preprocessing.py \    --anno_dir="/mnt/data/aist_plusplus_final/" \    --audio_dir="/mnt/data/AIST/music/" \    --split=testval

run training

python trainer.py --config_path ./configs/fact_v5_deeper_t10_cm12.config --model_dir ./checkpoints

Note you might want to change the batch_size in the config file if you meet OUT-OF-MEMORY issue.

run testing and evaluation

# caching the generated motions (seed included) to `./outputs`python evaluator.py --config_path ./configs/fact_v5_deeper_t10_cm12.config --model_dir ./checkpoints# calculate FIDspython tools/calculate_scores.py

Citation

@inproceedings{li2021dance,  title={AI Choreographer: Music Conditioned 3D Dance Generation with AIST++},  author={Ruilong Li and Shan Yang and David A. Ross and Angjoo Kanazawa},  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},  year = {2021}}