Research Data and AI: FAIR Principles and Reusable Datasets

FAIR research data is data that is Findable, Accessible, Interoperable and Reusable, by people and by machines. The FAIR Guiding Principles, published in Scientific Data in 2016, set out 15 short principles for achieving this, and most of them are about metadata: the identifiers, descriptions, vocabularies and provenance that let someone else find a dataset, understand it and use it correctly.

For research data used in AI and large-scale analysis, the bottleneck is rarely the data files. It is the metadata describing them: what each sample is, where it came from and how it was processed. In many public repositories that information is written as free text, and every submitter writes it differently.

This guide explains the FAIR principles in plain language, the metadata standards and repositories researchers rely on (with NCBI GEO as the main example), why free-text metadata is so hard to reuse, and how AI extraction with confidence scores and human review can turn it into structured, reusable records.

Key takeaways

• The FAIR principles put specific emphasis on machines being able to find and use data, which makes structured, controlled metadata essential.
• FAIR is not the same as open: the principles allow authentication and authorisation where necessary, and require metadata to stay available even when the data does not.
• Public repositories such as NCBI GEO hold enormous amounts of valuable data, but sample descriptions are often inconsistent free text that search and analysis tools cannot rely on.
• Large language models can extract and normalise metadata against ontologies far better than simple string matching, but published research shows they still need human curators to check the results.
• Confidence scores are useful for deciding what people review first, but they must be calibrated against a checked sample before anyone trusts them.

The FAIR principles in plain language

The FAIR Guiding Principles for scientific data management and stewardship was published in March 2016 by Mark Wilkinson and 52 co-authors representing academia, industry, funding agencies and scholarly publishers. What set it apart, in the authors' words, was that the principles "put specific emphasis on enhancing the ability of machines to automatically find and use the data, in addition to supporting its reuse by individuals."

The GO FAIR initiative publishes the principles with explanations. In summary:

Two points are often misunderstood. First, FAIR does not mean open. Principle A1.2 explicitly allows an authentication and authorisation procedure, so controlled-access clinical or genomic data can still be FAIR if its metadata is findable and the route to access is clear. Second, FAIR is a direction of travel rather than a pass or fail. The paper describes the principles as steps along a path towards machine-actionability, where a machine can make a useful decision about data it has not encountered before.

Metadata standards for research data

Principle R1.3 asks for "domain-relevant community standards". In practice, research metadata is usually built in three layers.

1. General descriptive metadata. Title, creators, identifiers, dates, related publications and access conditions, which apply to any dataset.
2. Domain minimum-information standards. These define what must be reported for a type of experiment. MIAME, the Minimum Information About a Microarray Experiment (Brazma et al., 2001), lists six critical elements: raw data for each hybridisation, final processed data, essential sample annotation including experimental factors and their values, experimental design including sample-data relationships, sufficient annotation of the array, and essential laboratory and data-processing protocols.
3. Controlled values from ontologies. Principle I2 asks for vocabularies that themselves follow FAIR. Rather than typing "HeLa cells" or "liver tissue" freely, curators link values to ontology terms, such as the Experimental Factor Ontology (EFO) or the Cell Line Ontology (CLO), each with a stable identifier.

The third layer is where most reuse problems sit. A record can satisfy a minimum-information checklist (it has a "sample annotation" field) while the value in that field is ambiguous, misspelled or inconsistent with every other record. Data Quality for AI: Measuring and Improving a Content Dataset covers how to measure that kind of problem.

What repositories such as NCBI GEO hold

The Gene Expression Omnibus (GEO), run by the US National Center for Biotechnology Information, is a public functional genomics data repository that supports MIAME-compliant submissions and accepts both array-based and sequence-based data. A 2026 study describes it as holding hundreds of thousands of high-throughput functional genomic datasets, which researchers reuse for secondary analyses, meta-analyses, method development and derivative databases.

GEO organises submissions into linked records, each with a stable accession number whose prefix shows the record type:

GEO's 2005 database paper explains that depositors retain editorial control and are responsible for the content and quality of their records, while GEO validates submissions syntactically and applies basic curation. The same 2026 study notes that GEO's submission standards are intentionally flexible and not strictly enforced, to reduce the burden on submitters. The result can be missing, ambiguous or mistyped experiment information, and discrepancies between a record and its associated publication.

Why free-text metadata is hard to reuse

A 2019 study by Rafael Gonçalves and Mark Musen tested 11.4 million sample metadata records from 2017 snapshots of two major repositories, NCBI BioSample and EBI BioSamples, checking whether a set of documented, machine-checkable fields held values that met each repository's stated requirements. It found that most metadata field names and values were not standardised or controlled, and that even simple yes-or-no and numeric fields were often filled with values of the wrong type. The authors found 32 different ways of representing the collection time of a sample, and in NCBI BioSample only 27% of the Boolean-type attribute values they checked were well specified. For a field asking whether a donor smoked, values included Non-smoker, nonsmoker, non smoker, ex-smoker, Former, current smoker, Y, N and 0.

The 2026 study of GEO metadata found the same kind of variation in mouse-strain names: a submitter writing "C57/Bl6" for C57BL/6, a record giving FVB where the linked paper specified FVB/N, and one record listing C57BL/6J in its overall design but C57BL/6 in its sample metadata.

Each variant has a cost:

• Search misses. A researcher looking for samples collected in the last year has to anticipate every way the date might be written.
• Analyses mix unlike samples. Combining datasets for a meta-analysis or a training set can silently merge different strains, tissues or conditions.
• Models learn noise. An AI model trained on mislabelled samples learns the labelling errors along with the biology.
• Curation does not scale. Manual curation into resources such as the Gemma database is, in the 2026 study's words, resource-intensive and does not eliminate the risk of errors.

The goal is to turn variants into one controlled value per field. As a generic illustration:

Extracting structured metadata with AI

Large language models are well suited to reading messy descriptions and mapping them to controlled terms, provided the task is constrained. The 2026 bioRxiv study by Rogic and colleagues at the University of British Columbia is a useful worked example. It used GPT-4o to annotate mouse strains and cell lines in 9,390 GEO experiments, compared the output against manually curated annotations in the Gemma database, and had curators review every disagreement.

The method illustrates good practice:

1. Gather all the text. GEO titles, summaries, overall design, sample characteristics and protocols, plus the linked paper's title, abstract and methods where openly available.
2. Constrain the choices. A short list of 156 strain terms fitted in the prompt. The 46,032 cell-line terms did not, so the system retrieved the 50 most similar ontology terms for each mention and asked the model to choose among them.
3. Require evidence. The model returned a verbatim quote supporting each annotation.
4. Use structured output. Responses followed a JSON schema so they could be validated automatically.
5. Fix settings for reproducibility. Temperature was set to 0 and a fixed seed was used.

The results show both the promise and the limits. For mouse strains, 77% of 6,013 experiments were annotated correctly, with average precision and recall of 0.82. A simple string-matching baseline found the relevant strains in 74% of cases but produced false positives in 80% of experiments, leaving only 6% correct overall. Cell lines were harder: 59% of experiments were annotated perfectly, with average precision and recall of 0.72. The supporting quotes were typically accurate and let curators verify annotations quickly, and the model's output led curators to correct the reference database in 230 mouse-strain and 17 cell-line experiments. The authors concluded that LLMs are not ready to fully replace manual curators but can already support them effectively.

A practical output format records the value, its evidence and its review state together, one line per extracted value (the identifiers here are placeholders):

{"sample": "GSM0000000", "field": "cell_line", "value": "HeLa", "ontology_id": "EXAMPLE:0001", "evidence": "HeLa cells were transfected with...", "source": "GEO sample protocol", "confidence": 0.94, "model": "model-name-and-version", "review_status": "unreviewed"}

Delivering a Dataset: JSONL, Parquet and Other Formats AI Teams Use covers how to package records like these.

Confidence scores and human checks

A confidence score attached to each extracted value tells reviewers where to look first. It is not proof that the value is right. Research on confidence elicitation by Xiong and colleagues, presented at ICLR 2024, found that LLMs tend to be overconfident when asked to state their confidence; techniques such as checking consistency across multiple responses reduced the problem, but every method struggled on tasks requiring professional knowledge.

So treat scores as something to calibrate:

1. Build a reviewed sample. Have curators check a random set of extracted values for each field.
2. Compare stated confidence with measured accuracy. If values scored above 0.9 are right only 80% of the time, adjust the thresholds or the scoring method.
3. Set thresholds per field. A wrong disease label usually matters more than a wrong instrument model.
4. Route by band as in the table below, and keep auditing a sample of high-confidence values.
5. Feed corrections back into the prompts, the candidate lists and the test set.

Empty is better than wrong. A field marked "not stated" is honest and searchable; a guessed value contaminates every analysis that uses it.

Before release, check the dataset against the principles: persistent identifiers for each study and sample, ontology identifiers for controlled fields, provenance for every extracted value (source text, model version, date, reviewer), a clear statement of access conditions, and documentation of how the metadata was produced and validated. Documenting a Dataset: Datasheets, Data Cards and Nutrition Labels describes how to write that documentation.

How Druid Learning helps

Druid Learning is an AI archival processing platform, based in Dublin and New York, that turns content archives into AI-ready datasets. For the Netherlands Cancer Institute, Druid worked with 1,000+ genomics studies from NCBI GEO, linked to 660 published papers, enriching more than 20,000 research samples with standardised metadata and producing more than 100,000 structured data points, each with a confidence score. Archive Optimization turns archives in any format into an indexed, labelled, AI-ready dataset. Read the Netherlands Cancer Institute case study.

Frequently asked questions

What are the FAIR principles?

The FAIR principles are 15 guidelines, published in Scientific Data in 2016, for making research data Findable, Accessible, Interoperable and Reusable. They cover persistent identifiers, rich metadata, standard access protocols, shared vocabularies, provenance, usage licences and community standards. Unlike peer initiatives focused on human researchers, they put specific emphasis on machines being able to find and use data automatically.

Is FAIR data the same as open data?

No. FAIR data can be openly available, but it does not have to be. Principle A1.2 allows an authentication and authorisation procedure where necessary, so sensitive clinical or genomic data can be FAIR if its metadata is findable and the route to request access is clear. Principle A2 also asks for metadata to remain accessible even when the data itself is no longer available.

What is NCBI GEO?

The Gene Expression Omnibus (GEO) is a public functional genomics repository run by the US National Center for Biotechnology Information. It accepts array-based and sequence-based data and organises it into Platform, Sample, Series and curated DataSet records, each with its own accession number. Its submission standards are deliberately flexible, so sample descriptions are often free text that varies widely between studies.

Can AI curate research metadata without human review?

Not yet, according to published evaluations. A 2026 study using GPT-4o to annotate mouse strains and cell lines in GEO experiments found it far more accurate than string matching, with average precision and recall of 0.82 for strains, but concluded that LLMs are not ready to fully replace manual curators. The recommended approach is AI first-pass extraction with evidence quotes, followed by curator review.

What does a confidence score on extracted metadata mean?

A confidence score estimates how likely an extracted value is to be correct, and is mainly used to decide which values people should review first. Scores from language models are often overconfident, so they should be calibrated against a sample that curators have checked. Once calibrated, thresholds can route high-confidence values to spot checks and low-confidence values to full review.

Sources

• The FAIR Guiding Principles for scientific data management and stewardship — Scientific Data
• FAIR Principles — GO FAIR Foundation
• The variable quality of metadata about biological samples used in biomedical experiments — Scientific Data
• Gene Expression Omnibus (GEO) — Data.gov, National Institutes of Health
• NCBI GEO: mining millions of expression profiles, database and tools — Nucleic Acids Research
• MIAME: Minimum Information About a Microarray Experiment — FGED
• Application of large language models to the annotation of cell lines and mouse strains in genomics data — bioRxiv
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs — arXiv