HumanRecognition

The system is develped to perform person recognition task on PIPA dataset, the detailed description of the approach of this system can be found here.

Performance (Last Updated: 5/19/2017)

Config	Accuracy
Face	62.18%
Head	63.19%
Upper-body	67.44%
Full-body	58.96%
All modality fused	82.31%
All modality fused + MRF refining	86.18%

Usage

Preparation

get dataset
```
	./scripts/get_PIPA.sh
```
get models
```
	./scripts/get_models.sh
```
get features (if you don't wish to extract them)
```
	./scripts/get_features.sh
```

Face Feature

We use FaceNet for face feature extraction. FaceNet is a CNN trained to directly optimize the embedding itself.

test face feature extractor

	python pyHumanRecog/face_feature_extractor_test.py

Head Feature

train head feature extractor (feel free to experiment with different batch size)
```
	python pyHumanRecog/head_feature_extractor_train.py --batch_size 32
```

test head feature extractor

	python pyHumanRecog/head_feature_extractor_test.py --batch_size 32

Body Feature

train body feature extractor (feel free to experiment with different batch size)
```
	python pyHumanRecog/body_feature_extractor_train.py --batch_size 32
```

test body feature extractor

	python pyHumanRecog/body_feature_extractor_test.py --batch_size 32

Upper-body Feature

train upper-body feature extractor (feel free to experiment with different batch size)
```
	python pyHumanRecog/upper_body_feature_extractor_train.py --batch_size 32
```

test upper-body feature extractor

	python pyHumanRecog/upper_body_feature_extractor_test.py --batch_size 32

Pose estimation

We use CPM for pose estimation. The estimated CPM pose will mainly be used for image warping.

CPM pose estimation
```
	python pyHumanRecog/extract_pose.py <img_dump_folder> <pose_dump_folder>
```
<image_dump_folder>: folder to dump CPM pose visualization images

<pose_dump_folder>: folder to dump CPM pose positions

Evaluation

For performance evaluation, Please first modify performance_test_config.py (within pyHumanRecog folder) to specify the features you wish to use and their corresponding weights. Then execute the following command.

python pyHumanRecog/performance_test.py

To perform MRF optimization (which incorporates the photo-level cooccurrence and mutual exclusive pattern into the final prediction), Set refine_with_photo_level_context = True in HumanRecog/performance_test_config.py.