Synthetic Data Enables Human-Grade Microtubule Segmentation with Foundation Models

Microtubules are fundamental building blocks of cellular life. They orchestrate intracellular transport, enable cell division, and play a central role in drug discovery. Yet, quantifying microtubules still often means manually tracing thin, overlapping filaments—image by image, hour by hour.

In our new work, Synthetic data enables human-grade microtubule analysis with foundation models for segmentation, we show that this bottleneck can be removed—and that doing so fundamentally changes what is possible.

From a concrete challenge to a general solution

The project started with a simple but meaningful challenge:

At first glance, these images appear visually simple. To human experts, tracing microtubules is tedious but straightforward. For automated methods, however, the problem is deceptively hard: microtubules are thin, elongated, overlapping, and break many assumptions built into common segmentation pipelines.

Even worse, there is no reliable way to evaluate progress, because manually annotating such data at scale is prohibitively expensive.

Can we fully automate microtubule segmentation in real in vitro microscopy images?

Figure 1 gives an overview of the problem and our benchmark setup.

(a) shows a synthetic image that mimics interference reflection microscopy (IRM) of in vitro reconstituted microtubules. Importantly, the synthetic data reproduces key biological properties such as filament length, curvature, and intersections—properties that are critical for downstream analysis.

(b) Because the data are synthetic, we obtain perfect instance-level ground truth masks for every microtubule. This enables quantitative evaluation that would be nearly impossible with real data alone.

(c) illustrates the output of a classical method (FIESTA). While such approaches have been widely used, the failure modes are clear: fragmented filaments, missing segments, and artifacts at crossings.

(d) shows the same image processed by SAM3, guided only by a simple text prompt: “thin line”. The result is a precise, human-grade tracing of intersecting microtubules—reflected in a high skeleton-based IoU score (SKIoU = 0.99).
In our new work, Synthetic data enables human-grade microtubule analysis with foundation models for segmentation, we show that this bottleneck can be removed—and that doing so fundamentally changes what is possible.

How do we get realistic synthetic data without annotations?

Generating synthetic images is not new—but making them realistic enough to matter is.
This is where our data generation pipeline comes in.

Aligning synthetic and real microscopy data—without labels

Figure 2 illustrates the core idea behind SynthMT.

We start with unlabeled real microscopy images (left). These are embedded using pretrained visual representations from DINOv2, which capture rich semantic and structural information without task-specific supervision.

Our parametric generator (center) produces synthetic images by sampling from distributions that control:

• Geometric properties (number of filaments, length, curvature)
• Imaging characteristics (PSF, noise, contrast, distortions, artifacts)

All of these factors are governed by a parameter vector θ.
An optimization loop then iteratively updates θ by maximizing the cosine similarity between embeddings of real and synthetic images. In other words, we align image distributions directly in feature space, without requiring any human annotations.
The result is synthetic data that matches the statistical and visual characteristics of real IRM microscopy, while still providing perfect ground truth.

SynthMT: a benchmark for evaluating microtubule segmentation

Using our generation pipeline, we construct and release SynthMT, a synthetic benchmark dataset for microtubule instance segmentation.
SynthMT provides:

• Realistic IRM-like microtubule images
• Perfect instance-level ground-truth masks
• A controlled setting for systematic, reproducible evaluation

[…]

The work is currently under submission at PLOS Computational Biology.