2025.09 · Pebblous Data Communication Team
Reading time: ~10 min · 한국어
Introduction: Understanding the AI Data Quality Standard
It is no exaggeration to say that the success of artificial intelligence (AI) and machine learning (ML) projects depends entirely on data quality. Data is the essential raw material for analytics and machine learning, and data quality issues can directly lead to degraded model performance, biased results, and serious malfunctions. To systematically manage and address these issues, the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) established the ISO/IEC 5259 series, an international standard framework for evaluating and improving AI data quality.
The ISO/IEC 5259 series builds upon ISO/IEC 25012 and ISO/IEC 25024, which provide frameworks for specifying and evaluating data quality requirements. Here, ISO/IEC 25012 is a foundational standard that defines the traditional quality model for data within computer systems, while the ISO/IEC 5259 series extends this foundation by additionally defining data quality characteristics essential for modern AI and ML contexts (e.g., diversity, representativeness, similarity).
This cheat sheet aims to provide a quick and easy reference (Quick Access) to the Data Quality Measures (QMs) specified in ISO/IEC 5259-2, one of the key documents in this standard. It clearly categorizes dozens of complex quality measures into four major characteristic groups, helping practitioners define data quality requirements for AI projects, diagnose the current state of datasets, and set improvement directions.
Furthermore, you can confirm that most of the quality measurement methods used by Pebblous Data Clinic, an AI training data quality assessment solution, correspond to these ISO/IEC 5259-2 quality measures. Pebblous Data Clinic leverages DataLens and Data Imaging technology to transform AI training data into feature vectors in embedding space, analyzing data in observable and measurable forms. In particular, the Level II/III diagnostics performed through this technology are powerfully aligned with the Fidelity-related characteristics (similarity, diversity, representativeness, balance) that the standard additionally requires for ML datasets, enabling quantitative and visual diagnosis.
1
Inherent Data Quality Characteristics
1.1. Strategic Importance
'Inherent data quality characteristics' are fundamental properties of the data itself, independent of any system or program. They are not mere checklist items but the starting point of data integrity. If this foundation is flawed -- if data is inaccurate (Accuracy), incomplete (Completeness), or contradictory (Consistency) -- the problem propagates throughout the entire project. This ultimately leads to enormous rework costs and unreliable AI models. Therefore, neglecting inherent quality management is equivalent to deliberately accumulating 'data debt' that must eventually be repaid. This is the single greatest risk factor blocking the success of any AI project.
1.2. Quality Measures List
| Sub-category | QM ID | QM Item | QM Description |
|---|---|---|---|
| Accuracy | Acc-ML-1 | Syntactic data accuracy | Measures how close data values are to a syntactically correct set of data values. |
| Acc-ML-2 | Semantic data accuracy | Measures how close data values are to a semantically correct set of data values. | |
| Acc-ML-3 | Data accuracy assurance | Measures the degree to which data is assured to be accurate. | |
| Acc-ML-4 | Risk of dataset inaccuracy | Measures the potential risk arising from inaccuracies in the dataset. | |
| Acc-ML-5 | Data model accuracy | Measures how accurately the data model represents the actual characteristics of the data. | |
| Acc-ML-6 | Data accuracy range | Measures the allowable range of accuracy for data values. | |
| Acc-ML-7 | Data label accuracy | Measures whether labels are accurately assigned to each element in the dataset. | |
| Completeness | Com-ML-1 | Value completeness | Measures the ratio of data items without null values to the total number of data items. |
| Com-ML-2 | Value occurrence completeness | Measures the ratio between the actual occurrence count of a given data value and the expected occurrence count specified in quality requirements. | |
| Com-ML-3 | Feature completeness | Measures the ratio of data items without null values among those related to a specific feature. | |
| Com-ML-4 | Record completeness | Measures the ratio of non-empty data records to the total number of data records. | |
| Com-ML-5 | Label completeness | Measures the ratio of samples with missing or incompletely assigned labels in the dataset. | |
| Consistency | Con-ML-1 | Data record consistency | Measures the ratio of duplicate data records in the dataset. |
| Con-ML-2 | Data label consistency | Measures the degree to which the same label is assigned to similar data items. | |
| Con-ML-3 | Data format consistency | Measures the degree to which data items meet data format consistency requirements. | |
| Con-ML-4 | Semantic consistency | Measures the degree to which data items meet semantic consistency requirements. | |
| Credibility | Cre-ML-1 | Values credibility | Measures the credibility of data values. |
| Cre-ML-2 | Source credibility | Measures the credibility of data sources. | |
| Cre-ML-3 | Data dictionary credibility | Measures the credibility of the data dictionary. | |
| Cre-ML-4 | Data model credibility | Measures the credibility of the data model. | |
| Currentness | Cur-ML-1 | Feature currentness | Measures the ratio of data items within the acceptable date range for a feature. |
| Cur-ML-2 | Record currentness | Measures the ratio of records in which all data items fall within the required age range. |
2
Inherent and System-dependent Data Quality Characteristics
2.1. Strategic Importance
'Inherent and system-dependent characteristics' are quality attributes created by combining the intrinsic properties of the data itself with the systems that store, process, and utilize it. No matter how accurate the data is (inherent quality), its real value depends on how well it can be leveraged in practice. For example, if data accessibility is poor (Acs-ML-3), the data science team's development slows down, delaying AI product launches. If data processing efficiency is low (Eff-ML), or storage waste is excessive (Eff-ML-3), cloud costs spike immediately and ROI deteriorates. These characteristics are therefore key management metrics for converting data's potential into actual business outcomes. Ignoring them is tantamount to voluntarily surrendering competitive advantage.
2.2. Quality Measures List
| Sub-category | QM ID | QM Item | QM Description |
|---|---|---|---|
| Accessibility | Acs-ML-1 | User accessibility | Measures the ratio of authorized users who can access the required data. |
| Acs-ML-2 | Data format accessibility | Measures the degree to which data is accessible using standard or open data formats. | |
| Acs-ML-3 | Data accessibility | Measures the ratio of accessible records in the dataset. | |
| Compliance | Cmp-ML-1 | Data item compliance | Measures the degree to which data items meet compliance requirements such as laws, standards, and rules. |
| Efficiency | Eff-ML-1 | Data format efficiency | Measures the storage and transmission efficiency of the format used to represent data. |
| Eff-ML-2 | Data processing efficiency | Measures the efficiency of time and resources (CPU, memory) consumed in processing data. | |
| Eff-ML-3 | Risk of wasted space | Measures the degree of risk of unnecessarily wasted storage space during data storage. | |
| Precision | Pre-ML-1 | Precision of data values | Measures how finely data values represent actual values (significant digits). |
| Traceability | Tra-ML-1 | Traceability of data values | Measures the degree to which the origin, change history, and creation process of data values can be traced. |
| Tra-ML-2 | User access traceability | Measures the degree to which user data access, modification, and deletion records can be audit-traced. | |
| Tra-ML-3 | Data values traceability | Measures the degree to which the entire history from creation to the present can be traced for data values. | |
| Understandability | Und-ML-1 | Symbols understandability | Measures the degree to which data symbols, abbreviations, and codes can be easily understood by users. |
| Und-ML-2 | Semantic understandability | Measures the degree to which users can clearly understand the meaning and context of data. | |
| Und-ML-3 | Data values understandability | Measures the degree to which users can interpret and understand data values themselves. | |
| Und-ML-4 | Data representation understandability | Measures the degree to which users can understand how data is represented (charts, graphs, tables, etc.). |
3
System-dependent Data Quality Characteristics
3.1. Strategic Importance
'System-dependent data quality characteristics' are quality metrics entirely governed by the performance and architecture of the IT infrastructure that stores, transmits, and processes data. For data-driven services to be trustworthy, the underlying system must be reliable. Whether data is available without failure when needed (Availability), whether it can be easily migrated to different environments (Portability), and whether it can be quickly recovered after failures (Recoverability) are at the core of Business Continuity Planning (BCP). These characteristics demonstrate how technically robust data assets are and serve as essential elements for stable operations. In particular, to evaluate the additional data quality characteristics that affect AI/ML model performance, this system foundation must be established first.
3.2. Quality Measures List
| Sub-category | QM ID | QM Item | QM Description |
|---|---|---|---|
| Availability | Ava-ML-1 | Data availability ratio | Measures the ratio of time (uptime) during which data is available when needed. |
| Portability | Por-ML-1 | Data portability ratio | Measures the degree to which data can be successfully ported to other systems or environments. |
| Por-ML-2 | Prospective data portability | Measures expected compatibility and ease of porting when migrating to other systems in the future. | |
| Recoverability | Rec-ML-1 | Data recoverability ratio | Measures the ratio of data that can be recovered in the event of a failure or disaster. |
| Rec-ML-2 | Feature recoverability ratio | Measures the degree to which dataset features transmitted incrementally can be recovered. |
4
Additional Data Quality Characteristics for Analytics and ML
4.1. Strategic Importance
'Additional data quality characteristics' are not optional. They are essential elements for building fair, stable, and production-ready AI. These metrics measure how well a dataset captures the real world (Balance, Bal-ML), whether it sufficiently reflects the target population (Representativeness, Rep-ML-1), and whether it covers diverse scenarios (Diversity). This is not merely a model performance issue but a matter directly tied to social responsibility. Failure in this area can not only degrade performance but also result in biased models that reflect only certain groups, causing reputational damage and legal liability. These characteristics are therefore the most critical criteria determining the success or failure of AI projects.
4.2. Quality Measures List
| Sub-category | QM ID | QM Item | QM Description |
|---|---|---|---|
| Auditability | Aud-ML-1 | Audited records | Measures the ratio of records in the dataset that have undergone an audit. |
| Aud-ML-2 | Auditable records | Measures the ratio of records in the dataset that are available for auditing. | |
| Balance | Bal-ML-1 | Brightness balance | Measures the inverse of the maximum brightness difference relative to the average brightness of image samples. |
| Bal-ML-2 | Resolution balance | Measures the inverse of the maximum resolution difference relative to the average resolution of image samples. | |
| Bal-ML-3 | Balance of images between categories | Measures the inverse of the maximum category size difference relative to the average category size (number of samples) of the dataset. | |
| Bal-ML-4 | Bounding box height to width ratio balance | Measures the inverse of the maximum difference relative to the average bounding box aspect ratio in the dataset. | |
| Bal-ML-5 | Category bounding box area balance | Measures the inverse of the maximum difference between category-level average bounding box area and the overall average bounding box area across all samples. | |
| Bal-ML-6 | Sample bounding box area balance | Measures the inverse of the maximum difference between sample-level bounding box area and the overall average bounding box area across all samples. | |
| Bal-ML-7 | Label proportion balance | Measures the difference in data item ratios between two categories with a specific label value. | |
| Bal-ML-8 | Label distribution balance | Measures the divergence between the label distribution and a uniform label distribution. | |
| Diversity | Div-ML-1 | Label richness | Measures the ratio of distinct labels in the dataset. |
| Div-ML-2 | Relative label abundance | Measures the ratio of individual data (items, records, frames) with a specific label in the dataset. | |
| Div-ML-3 | Category size diversity | Measures the ratio of categories with fewer categorized data items than the threshold defined in quality requirements. | |
| Effectiveness | Eft-ML-1 | Feature effectiveness | Measures the ratio of samples with acceptable features in the dataset. |
| Eft-ML-2 | Category size effectiveness | Measures the ratio of categories with fewer categorized samples than the threshold. | |
| Eft-ML-3 | Label effectiveness | Measures the ratio of samples with acceptable labels in the dataset. | |
| Identifiability | Idn-ML-1 | Identifiability ratio | Measures the ratio of data records in the dataset that could be used for identifiability (PII). |
| Relevance | Rel-ML-1 | Feature relevance | Measures the ratio of features in the dataset that are relevant to the given context. |
| Rel-ML-2 | Record relevance | Measures the ratio of records in the dataset that are relevant to the given context. | |
| Representativeness | Rep-ML-1 | Representativeness ratio | Measures the ratio of relevant attributes found in the dataset relative to the relevant attributes of the Target Population. |
| Similarity | Sim-ML-1 | Sample similarity | Measures the ratio of similar samples in the dataset using the number of clusters derived from clustering algorithms. |
| Sim-ML-2 | Samples tightness | Measures the difference between the maximum and minimum eigenvalues of the normalized dataset (geometrically measuring dataset tightness). | |
| Sim-ML-3 | Samples independency | Measures the dimensionality reduction potential (sample independence) of the dataset using PCA (Principal Component Analysis). | |
| Timeliness | Tml-ML-1 | Timeliness of data items | Measures the ratio of data items that meet timeliness requirements. |
4.3. Deeper Understanding of Additional Quality Characteristics
The AI/ML additional quality characteristics in ISO/IEC 5259-2 have a fundamentally different measurement philosophy from traditional data quality standards. They are designed from the consumer (user) perspective rather than the data producer perspective, and accurate measurement is only possible when the specific AI/ML project context and purpose are explicitly stated.
4.3.1. Data Producer Perspective vs. Data Consumer Perspective
| Criterion | Traditional Standards (ISO/IEC 25012, etc.) | ISO/IEC 5259 AI/ML Characteristics |
|---|---|---|
| Primary perspective | Data production and management perspective | Data consumer (AI/ML user) perspective |
| Measurement focus | Inherent properties of the data itself (accuracy, consistency, etc.) | Whether specific AI project requirements are met |
| Context dependency | Context-independent measurement possible | Usage context specification required |
| Evaluation criteria | Compliance with data production standards | Contribution to AI model performance and analysis result quality |
Key Difference: Traditional standards were developed from the production perspective because data producers were the primary consumers, but in AI/ML environments, data users do not create data -- they search for, collect, and process data they deem necessary for their projects. Therefore, ISO/IEC 5259-2 is designed to meet the requirements of AI model developers.
4.3.2. The Need for User-Perspective (Context)-Dependent Measurement
The additional AI/ML-specific quality characteristics do not merely measure the inherent properties of data itself; accurate measurement is only possible when the specific context and purpose for which the data will be used are explicitly stated.
Definition of Data Quality: Data Quality refers to the characteristics by which data meets an organization's data requirements 'in a specific context'.
Context Dependency: Data that satisfies requirements in one application domain or context does not necessarily satisfy requirements in another application domain or context.
4.3.3. Context-Dependent Examples of Additional Quality Characteristics
| Quality Characteristic | QM Example | Context Dependency Explanation |
|---|---|---|
| Representativeness | Rep-ML-1 (Representativeness ratio) |
Measures how well the dataset reflects relevant attributes of the 'Target Population'. Since the target population is the subject the AI project aims to make inferences about, measurement is impossible without the user's clear objectives. |
| Relevance | Rel-ML-1/2 (Feature/Record relevance) |
Measures the ratio of features or records relevant to a 'given context'. Example: Judging the relevance of height and weight features in a credit scoring model depends on the model's purpose. |
| Effectiveness | Eft-ML-1/2 (Accuracy/Recall-based) |
Measures whether the dataset meets requirements for 'use in a specific ML task'. The same data may have different effectiveness depending on the task type (classification/regression/clustering). |
| Balance | Bal-ML-3/8 (Class balance) |
Measures the degree to which sample distributions across categories are even, but the tolerable level of imbalance depends on the model's training strategy (oversampling, weight adjustment, etc.). |
Structural Fidelity:
The additional quality characteristics of ISO/IEC 5259-2 evaluate the structural fidelity of a dataset for achieving performance in a specific AI/ML task. This is not simply about creating "good data" but rather an evaluation framework for creating "data useful for a specific AI project". Therefore, all measurements must be quantified according to the purpose and context defined by the user (consumer).
5
[Application] Practical Measurement Methodology for ISO 5259-2 QMs: Pebblous Data Clinic Case
5.1. Strategic Importance
The ISO/IEC 5259-2 standard provides a clear framework for 'what' to measure for data quality. However, in practice -- especially for evaluating complex characteristics of unstructured data such as images and text -- a concrete methodology for 'how' to measure is essential.
Consider the example of Data Clinic, an AI training data quality management tool launched in 2024 by Pebblous. 'DataLens' technology provides a concrete solution to these challenges. It transforms data into high-dimensional embedding space to enable quantitative and visual diagnosis of abstract quality standards in real data environments. This is an important case demonstrating how the conceptual requirements of the standard and cutting-edge data analysis technology can combine to enhance the effectiveness of data quality management.
5.2. Analysis of QM Correspondence by Diagnostic Level
1. Level I (Basic Quality) Diagnostics and Inherent Characteristics
Pebblous Level I diagnostics evaluate the fundamental statistical and physical characteristics of datasets, including data consistency, missing values, and class balance. This directly connects to the traditional Inherent Data Quality Characteristics group in the ISO standard.
- Missing value measurement: Checking for missing data values, records, and labels corresponds directly to the ISO standard's Completeness characteristics, specifically Com-ML-1 (Value completeness), Com-ML-4 (Record completeness), and Com-ML-5 (Label completeness).
- Data consistency measurement: Checking data formats, sizes, and labeling errors is equivalent to evaluating Consistency characteristics such as Con-ML-2 (Label consistency) and Con-ML-3 (Format consistency).
- Class balance measurement: Checking data counts per class is a key activity for identifying imbalance issues in Balance characteristics such as Bal-ML-3 (Balance of images between categories) and Bal-ML-8 (Label distribution balance).
- Statistical measurement: Understanding basic data distributions and properties relates to Accuracy, evaluating how well data represents actual values.
2. Level II/III (Advanced Quality) Diagnostics and Additional Characteristics
Level II/III diagnostics leverage Pebblous's core DataLens technology to transform AI training data into feature vectors in embedding space, then analyze the geometric and distributional properties of the data. This methodology is optimized for evaluating the 'Additional Data Quality Characteristics' group, specifically the Fidelity of datasets that the ISO standard specially defines for AI/ML.
- Density measurement: Quantifying the density of data points in embedding space through DataLens enables identification of similar/duplicate data. Areas with abnormally high density likely contain duplicates, directly diagnosing ISO standard issues of Sim-ML-1 (Sample similarity) and Con-ML-1 (Data record consistency). The practical prescription is strategic removal of unnecessary data -- 'Data Diet'.
- Manifold shape analysis: The manifold shape visualized through DataLens is key evidence for judging dataset Diversity (Div-ML) and Representativeness (Rep-ML-1). In particular, low-density regions (gaps) within the manifold indicate a lack of edge cases, which can be diagnosed as Div-ML-3 (Category size diversity) or Bal-ML-8 (Label distribution balance) issues. The prescription for such data gaps is targeted generation of missing data -- 'Data Bulk-up'.
- Intrinsic dimension computation: Computing the minimum dimension reflecting the data's inherent characteristics using custom lenses in Level III diagnostics is a process of evaluating the dataset's compressibility and information complexity. This connects conceptually to Sim-ML-3 (Samples independency), which measures dimensionality reduction potential using PCA (Principal Component Analysis).
5.3. Concluding Insights
The ISO/IEC 5259-2 standard serves as an essential 'architectural blueprint' for evaluating and managing AI data quality. This standard provides a clear vision of what to measure and what goals to pursue. On the other hand, advanced diagnostic technologies like Pebblous's DataLens can be likened to specialized ultrasound equipment that sees through walls and measures structural stress -- precisely measuring the blueprint's requirements in actual data and uncovering hidden defects.
In conclusion, when the systematic framework provided by international standards combines with innovative technology that uncovers invisible structural defects in real data, AI data quality management can advance beyond abstract concepts to a substantive and sophisticated level. The synergy between these two elements will be the most important driving force for building trustworthy, fair, and efficient AI systems.
References
- ISO/IEC 5259-1:2024. Artificial intelligence -- Data quality for analytics and machine learning (ML) -- Part 1: Overview, terminology and examples
- ISO/IEC 5259-2:2024. Artificial intelligence -- Data quality for analytics and machine learning (ML) -- Part 2: Data quality measures
- ISO/IEC 25012:2008. Software engineering -- Software product Quality Requirements and Evaluation (SQuaRE) -- Data quality model
- ISO/IEC 25024:2015. Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- Measurement of data quality
- Pebblous official website. AI-Ready Data Solutions
- Pebblous Data Clinic. All-in-one solution for AI training data quality diagnosis and improvement
Related Materials
For more details about the data quality measures in the ISO/IEC 5259-2 standard, please refer to the PDF documents below.