• Common quality characteristics (Accuracy, Completeness, Consistency, Credibility, Currentness, etc.) — fundamental quality independent of data type. Measures whether files open, whether duplicates exist, and whether metadata is complete.
• AI/ML additional quality characteristics (Balance, Diversity, Effectiveness, Similarity, Representativeness) — distributional quality specific to image ML. Measures class balance, representativeness within feature space, and sample independence.
• Task-specific extensions — measures label and annotation quality matched to the task type, such as IoU for detection datasets and CLIP similarity for VLP datasets.

Type A Pure Images (no annotations)

Generative AI training images and unsupervised learning datasets belong here. Because there are no labels, label-related QMs do not apply.

Key concerns: pixel quality, deduplication, distributional balance, representativeness
Applicable QMs: Com-ML-1, Con-ML-1, Cre-ML-1, Bal-ML-1/2, Sim-ML-1/2/3, Rep-ML-1

Type B Classification / Detection / Segmentation Annotations

Supervised learning datasets such as ImageNet (classification), COCO (detection), and Cityscapes (segmentation). All Type A QMs apply, plus label quality QMs.

Key concerns: label accuracy, class balance, bounding box quality
Additional QMs: Acc-ML-6 (IoU), Acc-ML-7 (label accuracy), Bal-ML-3/4/5/6/7/8, Div-ML-1/2/3

Type C Image-Text Pairs (VLP / CLIP / Captioning)

Datasets where images are paired with text, such as LAION-400M, COCO Captions, and Conceptual Captions. Semantic alignment between image and text is the central concern.

Key concerns: image-text semantic alignment, caption completeness
Additional QMs: Acc-ML-2 (CLIP similarity), Com-ML-2 (object presence check), Con-ML-2 (label consistency)

Com-ML-1 Value Completeness — File Integrity ✅ Auto (L1)

Definition: The proportion of image files that can be opened and read successfully.

Measurement: Successful file header parses / total file count.

Image application: Attempt to read files with PIL/OpenCV. Files that raise IOError are treated as null values.

Com-ML-2 Value Occurrence Completeness — Object Presence Check ❌ External Tool

Definition: The proportion of images where the object specified in the annotation is actually present.

Measurement: Images confirmed by an object detection model / total annotated images.

Image application: Verify annotated class objects using YOLO/Faster-RCNN. Applies to Type B and C only.

Com-ML-3 Feature Completeness — Annotation Completeness ❌ External Tool

Definition: The proportion of images where specific features (masks, bboxes, keypoints) are annotated without omissions.

Annotation schema validation is required.

Com-ML-4 Record Completeness — Metadata Completeness ⚠️ Partial

Definition: The proportion of records with all metadata fields present (capture timestamp, resolution, camera info, etc.).

DataClinic offers partial support for reading file metadata.

Com-ML-5 Label Completeness — Label Coverage ✅ Auto (L1)

Definition: The proportion of images that have a label assigned (supervised learning only).

Measurement: Labeled images / total images.

Con-ML-1 Data Record Consistency — Duplicate Images ✅ Auto (L1)

Exact duplicates: Byte-identical files detected via SHA-256 hash.

Near-duplicates: pHash (Perceptual Hash) and dHash. A pHash distance < 10 is treated as a duplicate.

Con-ML-2 Data Label Consistency — Label-Image Consistency ⚠️ L2/L3

Detects cases where the same visual pattern receives different labels. Checks label mismatches among pHash near-duplicate image pairs.

DataClinic visualizes label inconsistencies among similar samples at L2/L3.

Con-ML-3 Data Format Consistency — Format Consistency ✅ Auto (L1)

Measures the mix rate of RGB vs. grayscale images and channel count uniformity (1ch vs. 3ch vs. 4ch).

Con-ML-4 Semantic Consistency — Semantic Anomaly Detection ❌ External Tool

Detects logical inconsistencies within images — for example, a scene labeled as summer that contains snow in the background.

Requires multimodal LLM-based verification using tools such as Qwen-VL or LLaVA.

Cre-ML-1 Values Credibility — Pixel Quality ⚠️ Partial (L1)

BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator): lower is better (range 0–100).

Laplacian Variance: used for blur detection. Low values indicate blur.

Watermark detection: performed via template matching or CLIP. DataClinic supports brightness/saturation distributions, but BRISQUE requires an external tool.

Cre-ML-2 Source Credibility — Data Provenance 〰️ Manual

Requires verification of C2PA (Content Credentials) digital signatures and confirmed supplier metadata.

Cre-ML-3 Data Dictionary Credibility — Schema Alignment 〰️ Manual

Verifies that header metadata maps correctly to annotation file properties.

Cre-ML-4 Data Model Credibility — Standard Schema Compliance ❌ External Tool

Validates conformance to COCO JSON, Pascal VOC XML, and YOLO TXT formats.

Acc-ML-3 Data Accuracy Assurance — Quality Assurance 〰️ Manual

Measures the proportion of data collected from verified sources and the proportion that has undergone expert dual review.

Acc-ML-4 Risk of Dataset Inaccuracy — Inaccuracy Risk (RPN) ⚠️ L2/L3

Calculates risk priority using an FMEA-based approach.

\[ \text{RPN} = S \times O \times D \]

Where \(S\) is Severity, \(O\) is Occurrence, and \(D\) is Detectability.

Image risk types: label errors, missing classes, blur/noise, metadata errors, class bias. DataClinic can estimate Occurrence (\(O\)) through outlier detection.

Acc-ML-5 Data Model Accuracy — Ontology Alignment ❌ External Tool

Checks whether the class hierarchy aligns with domain ontologies such as WordNet or the ImageNet hierarchy.

Bal-ML-1 Brightness Balance — Coefficient of variation of the brightness (mean pixel value) distribution ✅ L1

\[ CV_{\text{brightness}} = \frac{\sigma_{\text{brightness}}}{\mu_{\text{brightness}}} \]

Bal-ML-2 Resolution Balance — Uniformity of the resolution distribution ✅ L1

Sim-ML-1 Sample Similarity — Mean cosine similarity between samples in feature space ✅ L2/L3

Sim-ML-2 Samples Tightness — Cluster cohesion (concentration in high-density regions) ✅ L2/L3

Sim-ML-3 Samples Independency — Independence measured via nearest-neighbor distance distribution ✅ L2/L3

Rep-ML-1 Representativeness — Measures whether high-density clusters represent the full dataset ✅ L2/L3

Detects bias within feature space. Examples include the Antoine Blanchard effect (over-representation of a specific visual pattern) and the peacock effect (excessive concentration of visually striking samples).

Eft-ML-1 Feature Effectiveness — Feature validity (separability between classes) ✅ L2/L3

Eft-ML-2 Class Size Effectiveness — Whether class sizes are effective for training ✅ L2/L3

Eft-ML-3 Label Effectiveness — Whether labels are effectively separated in feature space ✅ L2/L3

Acc-ML-7 Label Accuracy ✅ L2/L3

Computes the distance from each sample to its class centroid in ViT/ResNet embedding space. Samples closer to a different class centroid are label error candidates.

ImageNet case study: Northcutt et al. 2021 — a 6% error rate, with approximately 85,870 mislabeled images. DataClinic automatically identifies these error candidates at L2/L3 through low-density sample detection.

Bal-ML-3 Inter-Class Balance ✅ L1

Imbalance ratio is calculated as max class count / min class count.

Real examples: SpectralWaste at 19.6:1, WikiArt at 133:1. Higher imbalance ratios make learning minority classes increasingly difficult.

Diversity — Div-ML-1/2/3 ✅ L1

Div-ML-1 Label Richness — Total number of unique classes in the dataset

Div-ML-2 Relative Label Abundance — Mean number of samples per class

Div-ML-3 Category Size Diversity — Diversity of the sample count distribution across classes

Label distribution — Bal-ML-7/8 ❌ External Tool

Bal-ML-7 Label Proportion Balance — Evenness of each class's share of total labels

Bal-ML-8 Label Distribution Balance — Evenness of per-image label count distribution (for multi-label classification)

Acc-ML-6 Bounding Box Accuracy (IoU) ❌ External Tool

\[ \text{IoU} = \frac{|\text{Pred} \cap \text{GT}|}{|\text{Pred} \cup \text{GT}|} \geq \text{threshold} \]

The threshold is typically 0.5 (AP50) or 0.75 (AP75). Validated through dual annotation plus expert review.

Bounding Box Balance — Bal-ML-4/5/6 ❌ External Tool

Bal-ML-4 Bbox H/W Ratio Balance — Evenness of bounding box height/width ratio distribution. Detects imbalance between portrait and landscape objects.

Bal-ML-5 Bbox Area Balance by Category — Evenness of mean bbox area across classes. Measures imbalance between small-object and large-object classes.

Bal-ML-6 Bbox Area Balance by Sample — Evenness of total bbox area distribution per sample.

Acc-ML-1 Syntactic Accuracy — Caption Grammar Accuracy ❌ External Tool

Detects grammatical errors and special-character contamination in caption text. Requires text processing tools.

Acc-ML-2 Semantic Accuracy — CLIP Semantic Alignment ✅ L3

\[ \text{CLIP cosine similarity}(\text{image}, \text{text}) \geq \text{threshold} \]

The threshold is typically 0.25–0.30. DataClinic L3 supports BLIP image-text matching.

Con-ML-2 Label-Image Consistency ⚠️ Partial

Detects cases where the same caption is used with different images. DataClinic provides partial support.