STGCN

Abstract

Dynamics of human body skeletons convey significant information for human action recognition. Conventional approaches for modeling skeletons usually rely on hand-crafted parts or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we propose a novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data. This formulation not only leads to greater expressive power but also stronger generalization capability. On two large datasets, Kinetics and NTU-RGBD, it achieves substantial improvements over mainstream methods.

Citation

@inproceedings{yan2018spatial,
  title={Spatial temporal graph convolutional networks for skeleton-based action recognition},
  author={Yan, Sijie and Xiong, Yuanjun and Lin, Dahua},
  booktitle={Thirty-second AAAI conference on artificial intelligence},
  year={2018}
}

Model Zoo

NTU60_XSub

config	keypoint	gpus	backbone	Top-1	ckpt	log	json
stgcn_80e_ntu60_xsub_keypoint	2d	2	STGCN	86.91	ckpt	log	json
stgcn_80e_ntu60_xsub_keypoint_3d	3d	1	STGCN	84.61	ckpt	log	json

BABEL

config	gpus	backbone	Top-1	Mean Top-1	Top-1 Official (AGCN)	Mean Top-1 Official (AGCN)	ckpt	log
stgcn_80e_babel60	8	ST-GCN	42.39	28.28	41.14	24.46	ckpt	log
stgcn_80e_babel60_wfl	8	ST-GCN	40.31	29.79	33.41	30.42	ckpt	log
stgcn_80e_babel120	8	ST-GCN	38.95	20.58	38.41	17.56	ckpt	log
stgcn_80e_babel120_wfl	8	ST-GCN	33.00	24.33	27.91	26.17*	ckpt	log

* The number is copied from the paper, the performance of the released checkpoints for BABEL-120 is inferior.

Train

You can use the following command to train a model.

python tools/train.py ${CONFIG_FILE} [optional arguments]

Example: train STGCN model on NTU60 dataset in a deterministic option with periodic validation.

python tools/train.py configs/skeleton/stgcn/stgcn_80e_ntu60_xsub_keypoint.py \
    --work-dir work_dirs/stgcn_80e_ntu60_xsub_keypoint \
    --validate --seed 0 --deterministic

For more details, you can refer to Training setting part in getting_started.

Test

You can use the following command to test a model.

python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]

Example: test STGCN model on NTU60 dataset and dump the result to a pickle file.

python tools/test.py configs/skeleton/stgcn/stgcn_80e_ntu60_xsub_keypoint.py \
    checkpoints/SOME_CHECKPOINT.pth --eval top_k_accuracy mean_class_accuracy \
    --out result.pkl

For more details, you can refer to Test a dataset part in getting_started.