Foot Calib Pos Image Processor

This project uses the TvCalib library to calculate the homography matrix for a football pitch image. This matrix allows mapping image points onto a standard 2D representation of the pitch (minimap).

The project also includes a pose estimation step (ViTPose) to detect players and calculate the average color of their torso.

The main result is a minimap where each player is represented by their colored skeleton, drawn at a dynamically reduced scale around their projected position on the pitch. The position is determined by projecting a reference point (feet/bottom of bbox) using the homography. The skeleton is then drawn using its relative coordinates (original image), scaled, and translated. The scale used depends on the player's vertical position (Y) on the minimap (higher on the minimap = smaller) and a base factor adjustable via the --target_avg_scale option.

It is also possible to visualize a minimap with the projection of the original image for comparison.

Features

Homography calculation from a single image via TvCalib.
Person detection (RT-DETR) and pose estimation (ViTPose).
Calculation of the filtered average torso color for each player.
Projection of each player's reference point (feet/bbox) onto the minimap.
Generation of a minimap with the original skeletons (colored, dynamically scaled based on projected Y position, and offset) drawn around the projected point.
(Optional) Generation of a minimap with the projected original image.
Possibility to save the calculated homography matrix.

Project Structure

.
├── .git/               # Git metadata
├── .venv/              # Python virtual environment (recommended)
├── common/             # Common Python modules (potentially)
├── data/               # Data (input images, etc.)
├── models/
│   └── segmentation/
│       └── train_59.pt # Pre-trained segmentation model (TO DOWNLOAD)
├── tvcalib/            # Source code of the TvCalib library (or a fork/adaptation)
│   └── infer/
│       └── module.py   # Main module for TvCalib inference
├── .gitignore          # Files ignored by Git
├── main.py             # Main script entry point
├── requirements.txt    # Python dependencies file
├── visualizer.py       # Module for generating visualization minimaps
├── pose_estimator.py   # Module for pose estimation and player data extraction
└── README.md           # This file

Installation

Clone the repository:

git clone <repository-url>
cd Foot_calib_pos_image_processor

Create a virtual environment (recommended):

python -m venv venv
.\venv\Scripts\Activate.ps1

Install dependencies:
```
pip install -r requirements.txt
```
(Make sure to install PyTorch with appropriate CUDA support if needed.)
Download the segmentation model: Place train_59.pt in models/segmentation/.
(Automatic) Download detection/pose models: The RT-DETR and ViTPose models will be downloaded automatically.

Usage

Run the main.py script providing the path to the image:

python main.py path/to/your/image.jpg [OPTIONS]

Options:

image_path: Path to the input image (required).
--output_homography PATH.npy: Saves the calculated homography matrix.
--optim_steps NUMBER: Number of optimization steps for calibration (default: 500, was 1000 in original README example).
--target_avg_scale FLOAT: Target average scale factor for drawing skeletons (default: 0.35). The script attempts to adjust the internal base scale so that the resulting average scale (after inverse dynamic modulation) is close to this value.

Example:

# Simple usage (target average size 0.35)
python main.py data/img3.png

# Aim for larger skeletons on average (target 0.5)
python main.py data/img2.png --target_avg_scale 0.5

The script will display:

Time taken and homography matrix.
Estimated internal base scale.
Requested TARGET average scale.
ACTUALLY applied FINAL average scale.
Window: Minimap with Original Projection.
Window: Minimap with Offset Skeletons (dynamically scaled inversely, targeting the average scale).
Press any key to close.

Key Dependencies

PyTorch, OpenCV, NumPy, PyTorch Lightning
SoccerNet, Kornia, Hugging Face Transformers, Pillow