Key takeaways

• Five traits define good R&D machine learning data: coverage, resolution, provenance, consistency, and structure.
• An 800-row dataset can lose to 30 well-designed experiments if the data clusters in one region.
• Most Bayesian optimization campaigns need 5 to 15 seed experiments to start and 20 to 50 to converge.
• Categorical-only parameterization, encoding chemistries as labels rather than descriptors, is the most common and most damaging failure.
• A half-day, 10-point data audit catches these problems before a campaign starts, when fixing them is cheapest.

‍

Good data quality for R&D machine learning depends on five traits: coverage, resolution, provenance, consistency, and structure. Row count is not one of them. A pharma client once handed Sunthetics 800 historical reactions collected over two years and asked for a model. The model built on that data was close to useless, because the rows shared one solvent, three temperature values, and catalyst labels with no link to the underlying chemistry. This post gives a working definition of good experimental data, the failure patterns that quietly kill modeling projects, and a checklist to run before your next campaign.

Is row count a good measure of data quality?

The 800-row dataset failed for specific reasons: 60% of the rows used a single solvent, the temperature column carried only three distinct values across all 800 entries, and the catalyst was recorded as a label with no descriptors and no link to the underlying chemistry. The data was abundant and inert. A campaign built on 30 well-designed experiments would have outperformed a model trained on those 800 every time.

The dominant assumption in popular ML discourse is that data volume drives performance. For deep learning on images and text, that holds. For R&D, where each experiment costs hundreds to thousands of dollars, it doesn't. The relevant axis is information per experiment: a 30-point dataset that systematically explores the design space will outperform a 1,000-point dataset living in one corner of it. The real question is whether the data you have is informative enough to be worth modeling, not whether you have enough of it.

Where does your data need to live in the design space?

Good data spans the design space. If 90% of your historical experiments cluster in one corner, the resulting model is a local model, regardless of row count. The practical test: plot the data along each input dimension. Empty regions are blind spots the model will not see.

The comparison is direct. Eight hundred clustered experiments concentrated in a low-temperature, low-polarity corner give a model no basis for predicting anywhere else, despite the row count. Thirty experiments distributed across the full design space let a model reason across the entire region, not just the corner it has already seen.

Five traits that separate modelable data from noise

• Coverage. Good data varies every input dimension systematically, with no empty corners. A dataset with 90% of its experiments in one solvent at three temperatures fails this test regardless of size.
• Resolution. Good data has effect sizes large relative to measurement noise. An assay with 8% standard deviation can't support a model built to detect 3% effects.
• Provenance. Good data carries its history: who ran it, on what equipment, when. Three technicians and two reagent batches collapsed into one spreadsheet is data without provenance.
• Consistency. Good data follows a stable protocol. A workup change at experiment 200, or a yield definition that shifts from HPLC to isolated mass partway through, creates two datasets that can't be honestly combined.
• Structure. Good data is encoded as descriptors a model can generalize from: polarity, donor number, steric and electronic features, rather than as labels like "Solvent: DMSO" or "Catalyst: Cat-A through Cat-E." Labels let a model memorize. Descriptors let it reason.

‍

Categorical-only parameterization is the most common failure among the five, since it blocks generalization to anything new, the entire reason to use machine learning in R&D. Chemistry-aware parameterization (Hansen parameters, Kamlet-Taft scales, molecular fingerprints, computed steric and electronic features) converts labels into features a surrogate model can reason about.

Which data patterns kill R&D ML campaigns?

Most teams carry at least one of these. The most dangerous ones don't fail loudly; they produce a model that looks like it works, then performs poorly on anything new.

• The categorical-only dataset. Every input is a label with no link to chemistry. The model memorizes; it never learns.
• The one-chemist, one-notebook problem. One person, one instrument, one window of time, so the model predicts the technician instead of the chemistry.
• The unmeasured confound. Yield correlates with reaction time, but reaction time also correlates with which technician ran the campaign. The model extrapolates in the wrong direction.
• The historical-data graveyard. Years of legacy spreadsheets, multiple protocols, and instruments. Cleaning it often costs more than running 20 to 30 well-designed new experiments.
• Over-replicated, under-explored data. Twenty replicates of one condition and one shot at everything else optimizes for assay validation, not design-space coverage.
• The leaked-objective problem. The modeling target is derived from a calculation that already uses the input variables, so the model looks accurate in validation and fails on real campaigns.

How much data do you actually need?

For a small-to-moderate dimensional Bayesian optimization problem (3 to 8 factors), 5 to 15 well-chosen seed experiments, parameterized with chemistry-aware descriptors, is enough to start. Twenty to 50 total experiments is usually enough to converge. Past roughly 100, the marginal value depends on whether new data explores new regions or simply replicates what the model already knows. That's far smaller than the volume most teams expect coming from a deep-learning frame of reference.

That leaves the harder question: fix the data you have, or start over? Good coverage and a consistent protocol is worth fixing and using, since descriptors can be added retroactively. Reliable measurements with uncertain provenance are worth using as a prior, not a primary modeling basis. One-chemist-one-notebook collection, categorical-only inputs, and protocol drift are usually worth replacing outright, since cleaning it typically costs more than running 20 to 30 new, well-designed experiments. Sunk-cost bias is what keeps bad data alive in modeling pipelines for years after that math stops working.

The 10-point data quality audit

Before committing data to a modeling campaign, run through these ten questions. A team that can answer all ten cleanly has a dataset worth modeling. A team that can't has a dataset worth fixing, or worth replacing with a small, well-designed campaign.

1. Coverage. Have all input dimensions been varied? Plot it.
2. Signal-to-noise. What's the assay's standard deviation, relative to the effects you care about?
3. Provenance. Can you identify who, when, and on what equipment for every row?
4. Consistency. Has the protocol been stable? Document any breaks.
5. Parameterization. Are inputs encoded as descriptors a model can generalize from?
6. Confounds. Are there variables that correlate with both inputs and outputs that aren't recorded?
7. Leakage. Is the objective computed from features in a way that makes modeling trivial?
8. Replicate balance. Is the dataset over-weighted on a few repeated conditions?
9. Historical hygiene. If using legacy data, is each row's source traceable?
10. Objective definition. Is the target metric defined consistently across every row?

‍

Half a day spent on this checklist is worth more than any model selection decision that follows. Machine learning fails in R&D more often because of the data than because of the model.

To see how chemistry-aware parameterization plays out on a real campaign, take a look at Sunthetics' case studies, or bring your own dataset to a working session with our team.

Book a 30-minute conversation with our data experts. Bring your actual data and your campaign, and we'll walk through whether SuntheticsML is the right fit.

‍

FAQs

How much data do I need for Bayesian optimization?

For a small-to-moderate dimensional problem (3 to 8 factors), 5 to 15 well-chosen seed experiments is enough to start, and 20 to 50 is usually enough to converge. The quality of parameterization matters more than row count.

Is more data always better?

No. More data with the same blind spots, the same protocol issues, and the same poor parameterization just repeats the problem at higher cost. More data only helps if it fills gaps the existing data doesn't cover.

What's the single biggest data quality mistake in R&D ML?

Categorical-only parameterization. Encoding chemistries as labels rather than descriptors prevents a model from generalizing to anything new, which is the entire reason to use machine learning in R&D in the first place.

Can I use legacy historical data with a modern campaign?

Sometimes. Legacy data can inform priors and identify regions of interest, but be cautious about training models on it directly. Protocol drift, missing covariates, and untracked confounds make legacy data unreliable as a primary modeling source.

How do I know if my assay is good enough?

Run replicates of a representative condition and compute the standard deviation. Compare it to the effect sizes you care about. If the noise is comparable to the signal, the assay is the bottleneck, not the model.

‍

Sources

1. Wilkinson, M. D., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.
2. Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), 100804.
3. Sculley, D., et al. (2015). Hidden Technical Debt in Machine Learning Systems. NeurIPS.
4. Shields, B. J., et al. (2021). Bayesian reaction optimization as a tool for chemical synthesis. Nature, 590, 89–96.
5. Hansen, C. M. (2007). Hansen Solubility Parameters: A User's Handbook. CRC Press.