i have about 600 images 1280x720 training set and ...

Below is an accuracy-focused starting point for your 600-image microscopy dataset. It uses conservative augmentation because Ultralytics specifically recommends reducing Mosaic, disabling MixUp/copy-paste, and lowering the learning rate for datasets under 1,000 images. Custom Albumentations transforms are supported through the Python API and run alongside YOLO’s built-in geometric augmentation. (GitHub)

No training script can guarantee the highest accuracy—the label quality, specimen-level train/validation split, object size, and class balance will have a larger effect—but this configuration is appropriate for your dataset size and domain.

1. Dataset configuration

Create sperm.yaml:

yaml
# Use an absolute path
path: /path/to/sperm_dataset

train: images/train
val: images/val

names:
  0: sperm
  1: c1
  2: c2
  3: c3
  4: c4
  5: c5

Expected structure:

text
sperm_dataset/
├── images/
│   ├── train/
│   └── val/
└── labels/
    ├── train/
    └── val/

Each label must use normalized YOLO detection format:

text
class_id x_center y_center width height

Make certain the class-number order in the labels matches sperm.yaml.

2. Install dependencies

The script below uses the current Albumentations 2.x argument names.

bash
pip install -U ultralytics "albumentations>=2.0.0"

Install the correct CUDA-enabled PyTorch build separately for your GPU.

3. train.py

python
#!/usr/bin/env python3
"""
Accuracy-focused YOLO26s training for small microscopy detection datasets.

Example:
    python train.py --data sperm.yaml --device 0

For moderately imbalanced classes:
    python train.py --data sperm.yaml --device 0 --cls-pw 0.25

Resume an interrupted run:
    python train.py --resume runs/sperm_yolo26/yolo26s_1280/weights/last.pt
"""

from __future__ import annotations

import argparse
import os
from pathlib import Path
from typing import Union

import albumentations as A
import torch
from ultralytics import YOLO


BatchType = Union[int, float]


def get_default_device() -> str:
    """Choose CUDA, Apple MPS, or CPU."""
    if torch.cuda.is_available():
        return "0"

    mps_backend = getattr(torch.backends, "mps", None)
    if mps_backend is not None and mps_backend.is_available():
        return "mps"

    return "cpu"


def parse_batch(value: str) -> BatchType:
    """
    Parse a fixed batch size or automatic GPU-memory fraction.

    Examples:
        --batch 8       -> fixed batch size 8
        --batch 0.70    -> automatically use about 70% GPU memory
    """
    try:
        number = float(value)
    except ValueError as exc:
        raise argparse.ArgumentTypeError(
            "Batch must be an integer such as 8 or a fraction such as 0.70."
        ) from exc

    if 0.0 < number < 1.0:
        return number

    if number >= 1.0 and number.is_integer():
        return int(number)

    raise argparse.ArgumentTypeError(
        "Batch must be a positive integer or a fraction between 0 and 1."
    )


def microscopy_augmentations() -> list:
    """
    Pixel-level augmentations designed for microscope images.

    These transformations intentionally avoid strong hue changes, heavy blur,
    elastic distortion, and aggressive noise that could destroy morphology.
    """

    return [
        # Simulate moderate microscope illumination and contrast differences.
        A.OneOf(
            [
                A.RandomBrightnessContrast(
                    brightness_range=(-0.12, 0.12),
                    contrast_range=(-0.15, 0.15),
                    p=1.0,
                ),
                A.CLAHE(
                    clip_range=(1.0, 2.0),
                    tile_grid_size=(8, 8),
                    p=1.0,
                ),
            ],
            p=0.30,
        ),

        # Mild sensor noise. Strong noise can erase thin sperm structures.
        A.GaussNoise(
            std_range=(0.005, 0.025),
            mean_range=(0.0, 0.0),
            per_channel=False,
            p=0.12,
        ),

        # Occasional mild focus or movement degradation.
        A.OneOf(
            [
                A.GaussianBlur(
                    blur_range=(3, 3),
                    sigma_range=(0.1, 0.8),
                    p=1.0,
                ),
                A.MotionBlur(
                    blur_range=(3, 3),
                    angle_range=(0.0, 360.0),
                    direction_range=(-0.3, 0.3),
                    p=1.0,
                ),
            ],
            p=0.08,
        ),
    ]


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Train YOLO26s for six-class microscopy detection."
    )

    parser.add_argument(
        "--data",
        type=str,
        default="sperm.yaml",
        help="Path to the YOLO dataset YAML file.",
    )
    parser.add_argument(
        "--model",
        type=str,
        default="yolo26s.pt",
        help="Pretrained YOLO model.",
    )
    parser.add_argument(
        "--imgsz",
        type=int,
        default=1280,
        help="Training image size. Use 1280 to preserve small microscope objects.",
    )
    parser.add_argument(
        "--epochs",
        type=int,
        default=200,
        help="Maximum epochs. Early stopping will normally finish sooner.",
    )
    parser.add_argument(
        "--batch",
        type=parse_batch,
        default=0.70,
        help="Fixed batch size or automatic GPU-memory fraction.",
    )
    parser.add_argument(
        "--device",
        type=str,
        default=get_default_device(),
        help="Examples: 0, 1, cpu, mps.",
    )
    parser.add_argument(
        "--workers",
        type=int,
        default=min(8, max(1, (os.cpu_count() or 2) - 1)),
        help="DataLoader workers.",
    )
    parser.add_argument(
        "--cls-pw",
        type=float,
        default=0.0,
        help=(
            "Class weighting power. Keep 0 for balanced data; "
            "try 0.25 for moderate imbalance."
        ),
    )
    parser.add_argument(
        "--mosaic",
        type=float,
        default=0.35,
        help="Conservative Mosaic probability for the small dataset.",
    )
    parser.add_argument(
        "--max-det",
        type=int,
        default=1000,
        help="Maximum detections per image during validation.",
    )
    parser.add_argument(
        "--seed",
        type=int,
        default=0,
        help="Random seed.",
    )
    parser.add_argument(
        "--project",
        type=str,
        default="runs/sperm_yolo26",
        help="Output project directory.",
    )
    parser.add_argument(
        "--name",
        type=str,
        default=None,
        help="Optional run name.",
    )
    parser.add_argument(
        "--resume",
        type=str,
        default=None,
        help="Path to last.pt from an interrupted run.",
    )

    return parser.parse_args()


def main() -> None:
    args = parse_args()

    # Resume restores the optimizer, scheduler, epoch, and original settings.
    if args.resume:
        checkpoint = Path(args.resume).expanduser()

        if not checkpoint.is_file():
            raise FileNotFoundError(f"Resume checkpoint not found: {checkpoint}")

        print(f"Resuming training from: {checkpoint}")
        YOLO(str(checkpoint)).train(resume=True)
        return

    data_path = Path(args.data).expanduser()

    if not data_path.is_file():
        raise FileNotFoundError(
            f"Dataset YAML not found: {data_path}\n"
            "Pass it with: python train.py --data /path/to/sperm.yaml"
        )

    if not 0.0 <= args.cls_pw <= 1.0:
        raise ValueError("--cls-pw must be between 0 and 1.")

    if not 0.0 <= args.mosaic <= 1.0:
        raise ValueError("--mosaic must be between 0 and 1.")

    # Fractional AutoBatch is intended for GPU training.
    batch: BatchType = args.batch
    if args.device in {"cpu", "mps"} and isinstance(batch, float):
        batch = 4

    run_name = args.name or f"yolo26s_{args.imgsz}_seed{args.seed}"

    # Early stopping parameters scale with the requested run length.
    patience = max(20, min(50, args.epochs // 5))
    close_mosaic = max(10, min(25, args.epochs // 10))

    print("\nTraining configuration")
    print(f"  Model:       {args.model}")
    print(f"  Dataset:     {data_path.resolve()}")
    print(f"  Image size:  {args.imgsz}")
    print(f"  Batch:       {batch}")
    print(f"  Device:      {args.device}")
    print(f"  Epoch limit: {args.epochs}")
    print(f"  Patience:    {patience}")
    print(f"  Class power: {args.cls_pw}\n")

    model = YOLO(args.model)

    model.train(
        # Dataset and runtime
        data=str(data_path.resolve()),
        imgsz=args.imgsz,
        epochs=args.epochs,
        patience=patience,
        batch=batch,
        device=args.device,
        workers=args.workers,
        cache="disk",
        pretrained=True,

        # Optimization for small-set fine-tuning
        optimizer="AdamW",
        lr0=0.001,
        lrf=0.01,
        momentum=0.90,
        weight_decay=0.0005,
        warmup_epochs=5.0,
        cos_lr=True,
        amp=True,

        # Reproducibility
        seed=args.seed,
        deterministic=True,

        # Keep square training so Mosaic remains available.
        rect=False,
        multi_scale=0.0,

        # Conservative sample-combination augmentation
        mosaic=args.mosaic,
        close_mosaic=close_mosaic,
        mixup=0.0,
        cutmix=0.0,

        # Microscope objects generally have no meaningful up/down direction.
        degrees=20.0,
        translate=0.05,
        scale=0.15,
        shear=0.0,
        perspective=0.0,
        fliplr=0.50,
        flipud=0.50,

        # Preserve biological/stain colors. Illumination is handled above.
        hsv_h=0.0,
        hsv_s=0.0,
        hsv_v=0.0,

        # Custom pixel-level microscopy transformations
        augmentations=microscopy_augmentations(),

        # Optional inverse-frequency class weighting
        cls_pw=args.cls_pw,

        # Validation and outputs
        max_det=args.max_det,
        val=True,
        plots=True,
        save=True,
        save_period=25,
        project=args.project,
        name=run_name,
        exist_ok=False,
        verbose=True,
    )

    best_path = Path(model.trainer.best)

    if not best_path.is_file():
        raise RuntimeError(f"Training completed, but best.pt was not found: {best_path}")

    print(f"\nBest checkpoint: {best_path}")

    # YOLO26's one-to-many head uses NMS and is the accuracy-oriented mode.
    best_model = YOLO(str(best_path))
    metrics = best_model.val(
        data=str(data_path.resolve()),
        imgsz=args.imgsz,
        device=args.device,
        max_det=args.max_det,
        end2end=False,
        plots=True,
        project=args.project,
        name=f"{run_name}_accuracy_validation",
    )

    print("\nFinal validation — one-to-many accuracy mode")
    print(f"  mAP50-95: {metrics.box.map:.4f}")
    print(f"  mAP50:    {metrics.box.map50:.4f}")
    print(f"  mAP75:    {metrics.box.map75:.4f}")

    names = best_model.names
    print("\nPer-class mAP50-95:")

    for class_id, class_map in enumerate(metrics.box.maps):
        if isinstance(names, dict):
            class_name = names.get(class_id, str(class_id))
        else:
            class_name = names[class_id]

        print(f"  {class_id}: {class_name:<12} {float(class_map):.4f}")


if __name__ == "__main__":
    main()

4. Start training

bash
python train.py --data sperm.yaml --device 0

Ultralytics supports a fractional batch value such as 0.70, which automatically selects a batch size based on GPU memory. The script trains at 1280 because reducing your 1280×720 images to 640 would halve the dimensions of already-small targets. (Ultralytics Docs)

For moderately imbalanced classes:

bash
python train.py \
    --data sperm.yaml \
    --device 0 \
    --cls-pw 0.25

Ultralytics documents cls_pw=0.25 as a reasonable starting point for moderate imbalance; leave it at 0.0 when the six classes have similar annotation counts. (Ultralytics Docs)

Important accuracy considerations

Do not randomly split adjacent video frames. Keep all frames from the same patient, slide, specimen, recording, or microscope session entirely in either training or validation. Otherwise, the validation score can look excellent while performance on new specimens is poor.

Inspect object size. If many sperm boxes are only a few pixels wide or tall at 1280 resolution, use overlapping image tiles rather than stronger augmentation. Augmentation cannot recover details that disappear during resizing.

Check class counts by bounding boxes, not images. One image may contain hundreds of sperm but only a few c4 or c5 objects.

Use end2end=False for final accuracy testing and inference. YOLO26 documents this one-to-many head as the accuracy-oriented option, while the default end-to-end head prioritizes NMS-free inference. (GitHub)