The OpenADMET initiative

“Quantity has a quality all of its own”

Original source unknown

The quote with which I’ve opened the post on the OpenADMET initiative is often attributed to Joseph Stalin and some suggest that he might have been referring to the T-34. While the safety and comfort of crews were not priorities for Soviet tank designers the T-34 was a better tank than the quote might be taken to imply. In particular, the distinctive sloped armour presented challenges for Wehrmacht anti-tank gunners (at least until the introduction of the the formidable 88 mm PAK 43) while the wider tracks of the T-34 enabled it to operate effectively in conditions that the more refined (and heavier) Tiger could not. Here are some photos of T-34s taken at the Brest Fortress and the War Memorial of Korea:

 

The principal objective of the OpenADMET initiative appears to be generation of measured data to enable machine learning (ML) models to be built for prediction of toxicity and absorption, distribution, metabolism, and excretion (ADME). Just as the success of the T-34 on the Eastern Front was not just down to being available in huge numbers there is a bit more to data generation than simply generating massive data sets. A significant challenge for initiatives such as OpenADMET is covering chemical space at a sufficiently fine resolution and with sufficient spread in the measured data to enable building of ML models that can predict accurately across a diverse range of chemotypes. As I’ve noted previously drug discovery project teams have delivered (and continue to deliver) clinical development candidates without ever having sufficient data for building ML models that can accurately predict all the quantities of interest to the project teams. While I'll be criticising aspects of the OpenADMET initiative in this post it must be stressed that I do see great value to drug discovery in making relevant data freely available in the public domain (and Open Science in general).   

Previously I suggested that drug design can be thought of in terms of three objectives and the OpenADMET initiative addresses the second (minimize off-target bioactivity) and third (maximize controllability of exposure) of these. As I argued in that previous post Absorption, Distribution, Metabolism, Excretion (ADME) and Toxicity (T) should be seen as separate issues in the design context (my view is that using the term ADMET projects a lack of familiarity with the practical realities of drug design). Put another way toxicity is something that drugs do to the human body while ADME determines what the human body does to drugs (pharmacy students typically encounter the distinction between pharmacodynamics and pharmacokinetics early in their training). This is a good point at which to mention the Avoid-ome and here’s a post from a couple of years ago. While I agree that toxicity fits naturally into an Avoid-ome framework, I'm unconvinced that the introduction of the term is the Great Leap Forward that some believe it to be. However, ADME issues generally cannot be accommodated within an Avoid-ome framework because ADME-based design is ultimately about control of exposure (concentration of drug in contact with the target or anti-target) and not about avoidance.

This is a good point at which to take a look at the recent F2026 article (Mapping the avoid-ome: a systematic open-science approach to predictive ADMET). As is customary for posts at this blog quoted text is indented with my comments enclosed within square brackets in red italics, and I’ve used the F2026 reference numbers. 

I’ll start with the abstract:

Drug discovery often fails due to unpredictable ADMET issues, which account for 30% of clinical setbacks. [While fully agreeing that toxicity and poor ADME are important issues that that do need to be addressed more effectively I do not consider references (3) (4) (5) to support what the authors have stated here or their claim that “more than 90% of molecules created during discovery fail to meet basic ADME standards”. For example, reference (3) states that “the major causes of attrition in the clinic in 2000 were lack of efficacy (accounting for approximately 30% of failures) and safety (toxicology and clinical safety accounting for a further approximately 30%)”. All that said, decisions to take compounds into clinical development are based on measurements made in a range of assays and failure in clinical development reflects an inability of these assays to predict clinical outcomes. To more effectively address attrition we actually need new assays that are more predictive of outcomes in clinical development as opposed to new ML models that are more predictive of quantities that will need to be measured anyway.] Conventional methods lack the atomistic detail needed to navigate the “Avoid-ome”—a finite set of proteins acting as “anti-targets”. OpenADMET is an open-science initiative addressing this by creating pre-competitive, mechanistic datasets. [With respect to to "atomistic detail" it's important to bear in mind that structures for transition states (relevant when the quantity of interest is rate of turnover by metabolic enzymes) cannot actually be observed in experimental protein structural studies.] Using high-throughput structural biology, active learning, and community challenges, it builds generalizable models grounded in structural “ground truth”. [I would question the wisdom of invoking “ground truth” in scientific studies because it brings to mind stirring sermons on themes like "Pastor needs an additional private jet" and invocation of “ground truth” will endow your arguments with a distinctly pastoral odour. Building truly generalizable ML models for the quantities of interest to drug designers would certainly be of great value in drug discovery but it will take much more than anointing models with "ground truth" to achieve this.] By directly studying the Avoid-ome, OpenADMET facilitates an era of rational, multi-parameter drug design. [My view is that drug design should be described as ‘multi-objective’ rather than ‘multi-parameter’ because design against a single objective such as affinity maximisation can still be multi-parameter in nature, and I consider it tautological to describe drug design as ‘rational’.]

Here's the conclusion to F2026:

Understanding and navigating the Avoid-ome is the central universal challenge of modern drug discovery. [My view, which might be shared by a few others, is that the principal challenge for drug discovery is (and has always been) the uncertainty that results from the complexity of human biology and here’s a relevant blog post by "That Dude That Says That AI Drug Discovery Isn't So Amazing".] By creating open, structural, and mechanistic datasets and benchmarking predictive models through blind challenges, OpenADMET provides a practical foundation for a new era of rational drug design. [I am a big fan of of Open Science and see a significant value in making data relevant to drug discovery freely available. At the risk of repetition the term “rational drug design” is tautological and the promise of (yet another) “new era” will trigger the eye-rolling reflex for many experienced drug hunters. Given the aspirational nature of the initiative at this stage I think it would be more accurate to state that "OpenADMET aims to provide" rather than "OpenADMET provides".] The best way to increase the effectiveness of drug discovery in the coming decade is to stop avoiding the Avoid-ome and instead study it directly. [While I certainly see benefits from a deeper understanding of the proteins that cause toxicity and influence ADME, I don’t see doing so as quite the panacea that the authors of F2026 would have us to believe it to be. I argued over two years ago that the Avoid-ome is generally not a useful concept for consideration of ADME issues and my view is that shackling OpenADMET to the Avoid-ome has actually reduced the scientific credibility of the initiative. My advice to those leading the OpenADMET initiative is to unshackle it from the Avoid-ome before it's too late.] 

To be fair the authors of F2026 do concede ensuring that that there is a lot more to ADME optimization than ensuring that anti-targets are not engaged although I would still challenge their assertion that "understanding and navigating the Avoid-ome is the central universal challenge of modern drug discovery".   

While our framework heavily emphasizes the specific protein anti-targets of the Avoid-ome, we recognize that fundamental physicochemical and integrative properties—such as aqueous solubility, membrane permeability (logD), and metabolic stability are major drivers of ADMET outcomes, particularly for absorption and excretion [absorption and excretion would  be more accurately described as 'ADME outcomes' than 'ADMET outcomes']. Although these factors are not mediated by a single anti-target, [these factors are not mediated by anti-targets] they are critical bulk molecular properties [I consider the term "bulk molecular properties" to be an oxymoron] that often dictate whether a compound succeeds or fails.

Having got the general stuff out of the way I’ll examine the OpenADMET initiative from the perspectives of both ADME and toxicity, starting with the latter. My view is that any off-target bioactivity is undesirable given the complexity of human biology although bioactivity against known anti-targets such as hERG is clearly unacceptable. It’s also important to take account of the concentration at which off-target effects are observed and a weakness shared by many (most?) studies of pharmacological promiscuity is that bioactivity thresholds are set far too permissively to have any physiological relevance whatsoever (LS2007 classifies compounds that exhibit >30% inhibition at 10 µM as 'active' and, more recently, FOM2025 states that "Mestres et al. (p4) anticipated that the average number of proteins with which a drug interacts with potentially relevant bioactivity (<10 µm) was close to six").

It’s perhaps appropriate to take a general look at QSAR/QSPR approaches given that the main focus of OpenADMET appears to be generation of data for training what could be referred to as 'QSAR-like' or 'QSPR-like' ML models. In my view, the impact of QSAR/QSPR modelling on real world drug discovery was limited and claims to the contrary are generally not verifiable. A difficulty faced by those advocating the use of QSAR/QSPR approaches was that projects had either delivered or been put out of their misery by the time there was sufficient data for building predictively useful models. My view is that modern ML models, like the QSAR and QSPR models that preceded them, generally can't  extrapolate out of the chemical spaces in which they were trained. While I certainly wouldn’t claim ground truth for this view, I’m not aware of any studies in which a QSAR or QSPR model built using only data from one structural series was convincingly shown to be usefully predictive of for compounds in a different structural series.  Medicinal chemists typically perform their optimizations within specific structural series and this means that structure-activity/property relationships (SARs/SPRs) tend to be local in nature. For users of ML models of bioactivity and other properties of compounds it is important to know whether chemical structures for which predictions are being made lie within the applicability domains of the models. Put another way, medicinal chemists who use ML models are generally much more interested in how well the models will predict for the structural series that they're working on and much less interested in how well the models have fit the training data (anybody who has received financial advice will be familiar with the "past performance is not indicative of future results" disclaimer). The selection criteria for assays and compounds by the OpenADMET initiative are not currently clear.

I see a degree of overlap between the OpenADMET and OpenBind initiatives in that safety assessment will often require prediction of binding affinity of anti-targets for compounds being considered for synthesis. Indeed, there is no reason that structures for complexes of anti-targets with ligands should be excluded from data sets when the objective is to build universal ML models for prediction of binding affinity. Nevertheless, categorical models for prediction of off-target bioactivity still have value in hit-to-lead work and lead optimization whereas categorical models for predicting on-target bioactivity are generally only useful during hit identification. 

While it will often prove feasible to build ML models for binding affinity of anti-targets for ligands, many users will want to know whether the chemical structures of interest to them lie within the applicability domains of the models. As discussed in my post on the OpenBind initiative, using geometric features in protein-ligand complexes as descriptors to train ML models can potentially enable accurate affinity predictions to be made for chemotypes that are not represented in the training data. My view is that those leading the OpenADMET initiative will need to be more explicit about how (or even whether) they propose to use protein structural information for affinity prediction. Not all off-target effects of drugs can be specified in terms of reversible binding (as is the case for PXR activation which forms the basis for the current OpenADMET challenge) and this is generally more likely to be an issue when attempting to build models for off-target bioactivity. 

Let's now take a look ADME from the perspective of ML modelling. As noted earlier this post, ADME and toxicity are completely different issues in drug design and using the term ‘ADMET’ conveys (at least to me) an impression that some of the practical realities of drug discovery have not been properly understood. Drug action is driven by the concentration of the drug at its site(s) of action (the term exposure is commonly used) and one of the practical realities of drug discovery is that drug concentration at sites of action generally cannot be measured in vivo unless binding sites are directly in contact with plasma (see post on the objectives of drug design). I also suggest that readers take a look at SR2019 (Smith and Rowland, Intracellular and Intraorgan Concentrations of Small Molecule Drugs: Theory, Uncertainties in Infectious Diseases and Oncology, and Promise | DMD 2019 47:665-672) that I recommend to everybody working in drug discovery and chemical biology. While maximization of affinity is a legitimate design objective, exposure is something that needs to be carefully controlled rather than simply maximized (I'm guessing that Paracelsus might have cautioned against maximization of exposure and he’s been dead for almost half a millennium).

Pharmacokinetic/pharmacodynamic (PK/PD) modelling (DM1999 | D2008 | R2008 | NCF2011 | W2015 | SS2017 | BL2023 | HR2024 | C2024 ) is used in the later stages of drug discovery to predict the (dose-dependent) effects of drugs on humans. The inputs for PK/PD modelling are predicted human pharmacokinetic profiles (typically generated from the results from PK profiles observed in animal studies) and bioactivity measurements. In PK/PD modelling it is usual to invoke the free drug hypothesis (SYF2022 | W2025)  by assuming that the concentration of a drug at its site of action is equal to the unbound concentration of the drug in plasma (the terms ‘free drug principle’ and ‘free drug theory’ are also used although I prefer ‘free drug hypothesis’ because it’s an assumption that is being made). There are two scenarios under which this assumption is known to break down. First, there is active transport at one or more points in the path between the drug’s site of administration and its site of action. Second, the drug is ionizable and its site of action is within a compartment where the pH differs from plasma pH (basic centres not required for binding to the target are generally not recommended when targeting lysosomal enzymes if you’re concerned about selectivity).

The ADME acronym refers to in vivo phenomena and physicochemical properties (lipophilicity, aqueous solubility, passive membrane permeability) and in vitro biochemical quantities (turnover by metabolic enzymes, active efflux) should be described as ‘ADME predictors’ and not ‘ADME properties’. My view (which I’ll be happy to change in the light of compelling evidence) is that it isn’t currently possible to predict in vivo plasma concentration to the level of accuracy required for PK/PD modelling if you’re only using in vitro ADME predictors. That said, I’m certainly not suggesting that genuinely predictive models for aqueous solubility, membrane permeability and turnover by metabolic enzymes are without value in drug discovery projects.

In a previous post I argued that drug designers should aim to maximize controllability of exposure and to do so requires a focus on pharmacokinetic profile rather than individual ADME predictors. In many cases the ideal pharmacokinetic profile would be that resulting from intravenous infusion whereby the plasma concentration of a drug is maintained at the minimum level required for therapeutically useful effects (the variation in plasma concentration over the dosing interval for an orally-dosed drug is actually undesirable from the perspective of simultaneously achieving efficacy and safety). While ML models for quantities such as aqueous solubility, membrane permeability and turnover by metabolic enzymes certainly have a place in drug design, decisions as to whether a compound should be evaluated in vivo will generally be based on in measured values rather than predictions from ML models. 

Let’s now take a look at ADME in the context of the Avoid-ome. To be fair, the authors of F2026 do concede that ADME doesn’t quite fit into the Avoid-ome framework although this does rather beg the question as to why they assert that “understanding and navigating the Avoid-ome is the central universal challenge of modern drug discovery”.  One fundamental problem with the F2026 article is that its authors have got themselves in a bit of a tangle with respect to how they’ve defined anti-targets. In drug discovery, the term ‘anti-target’ generally refers to a protein which is associated with the risk of toxicity when engaged in vivo. The authors of F2026 appear to be broadening the 'anti-target’ definition to include proteins that they believe have detrimental effects on the ADME behaviour of compounds and, in my view,  it is neither valid nor useful to do so.

Let’s take a look at Fig. 1 (The set of protein anti-targets that comprise the Avoid-ome) in F2026 and you’ll see that a number of transporters are included as anti-targets in the graphic. While I certainly agree that active efflux is generally undesirable from the perspective of achieving adequate exposure, transporters would not automatically be regarded as anti-targets from the safety perspective. That said, inhibition of bile salt export pump (BESP) is considered a risk factor for drug-induced liver injury (DILI) and here's a link to a relevant article. 

The pitfalls associated the broadening the definition of ‘anti-target’ are brought sharply into focus when metabolic enzymes enter the picture. Inhibition of CYPs is widely accepted as a safety issue because of the potential for drug-drug interactions (LL1998 | H2020 | L2024) but CYPs have been labelled in Fig. 1 of F2026 as metabolism (M) anti-targets.  While I certainly agree that high clearance makes it difficult to maintain exposure at levels required for therapeutic benefits, a recommendation that clearance be entirely avoided would generally be regarded by drug safety expected as very bad advice indeed (remember why CYP inhibition is considered to be a safety issue). As a reviewer of F2026 I would have pressed its authors to explain why they had classified the aromatic hydrocarbon receptor (AhR) in Fig. 1 as a metabolism (M) anti-target given that the toxicity of dioxin results from engagement of this receptor (M2005 | S2014).

The authors of F2026 include serum albumin (HSA) in the Avoid-ome and it’s fair to say that plasma protein binding (PPB) is a source of confusion even for medicinal chemists (there would probably be much less confusion if unbound plasma concentration was measured directly in PK studies and I’ll direct readers to the SDK2010 article). The recent B2025 study (What Do Oral Drugs Really Look Like? Dose Regimen, Pharmacokinetics, and Safety of Recently Approved Small-Molecule Oral Drugs) reports that "most drugs are >95% plasma protein bound (58%), with a large fraction >99% bound (29%)" which appears to contradict the view expressed in F2026 that HSA should be considered as an anti-target. I would argue that PPB is potentially beneficial in that it protects the drug to some extent during first pass metabolism (distribution from plasma also protects of drug from metabolism but only after the drug has first passed through the liver). In the design context a prediction of unbound fraction is unlikely to be of much interest before a compound has been synthesized while measured values will be used in conjunction with pharmacokinetic studies. 

This is a good point at which to wrap up. While OpenADMET is likely to generate useful data and models, I don’t see it as a game-changer and consider the claim that the initiative “provides a practical foundation for a new era of rational drug design” to be fanciful. Something that comes across from my reading of F2026 is a general lack of expertise in the areas of drug safety, pharmacokinetics and PK/PD modelling, and I urge those leading OpenADMET to get some experts on board (or to listen more carefully to the experts who are already on board). I have argued that the Avoid-ome is neither valid nor even useful as a framework for ADME-based drug design and my advice to those leading OpenADMET is to quietly drop the Avoid-ome.     

The OpenBind initiative

I’ll open the post on the OpenBind initiative with photos from my visit last year to Korea which was timed to coincide with the cherry blossoms (this meant that the customary April Fools post was from Seoul). Things did not start well on the day that I took these photos (having lined up the first shot for the day it became abundantly clear that the camera’s battery was still being charged at the hotel) and I wondered whether Great Leader’s grandson might have labelled me as a dotard. Fortunately, Seoul’s Metro is excellent and I was still able to get some photos at Huiujeong-ro Cherry Blossom Road and Yangjaecheon Stream.

In this post I’ll be taking a look at the OpenBind initiative and here's a summary of the concept. I certainly see great value in having large quantities of this type of data (affinity measurements with X-ray crystal structures for the corresponding protein-ligand complexes) to the drug discovery and chemical biology communities. The grating-coupled interferometry (GCI) protocol used for affinity measurement enables association and dissociation to be observed in real time and presumably it is also possible to characterize stoichiometry using this technique. I would expect he GCI protocol to enable weaker binding affinities to be reliably quantified (likely to increase the dynamic range of the assay) as well as allowing measurement of binding affinity of glycoproteins for ligands. Given the focus on enabling affinity prediction, there is no reason for excluding anti-targets or non-human proteins.  

Generation of data for training machine learning (ML) models, which are renowned for their voracious appetite for data, appears to be the principal aim of the initiative. However, the availability of large quantities of such data will also enable more extensive evaluation of physics-based methods for calculating binding affinity and can potentially inform hypothesis-driven design by identifying bioisosteric relationships between elements of substructure. One point worth making is that having affinity measurements linked to protein-ligand structures for structurally-related compounds of varying molecular complexity (see HLH2001) enables frustration of molecular interactions to be studied (this is particularly relevant to fragment-based design) and I discussed in HBD3 how frustration of hydration might be exploited in design. Given the importance of aqueous solvation in biomolecular recognition it may be beneficial to measure some alkane/water partition coefficient values and I'll point you to a post on this topic in case it's of interest. As discussed in KMP2013 and B2017 polarity parameters can be derived from alkane/water partition coefficient measurements for functional groups. 

I've suggested that there are three objectives to drug design and the OpenBind initiative addresses the first of these which is to maximize on-target bioactivity. It's worth noting that proteins are not the only drug target class of interest (see CD2022) while bioactivity for ‘new modalities’ such as targeted protein degradation (see CC2026) and irreversible covalent bond formation between targets and ligands cannot be quantified in terms of affinity alone. My view is that OpenBind would be more accurately described as an initiative for ligand discovery than for drug discovery given its focus on enabling methods for affinity prediction.  Modern ML models for affinity prediction are effectively quantitative structure-activity relationship (QSAR) models and I would question whether the use of the AI label is justified in either case. All that said, I would expect OpenBind to catalyse significant progress in the affinity prediction field which hopefully will translate to tangible benefits for drug discovery.

It’s perhaps appropriate to take a general look at QSAR approaches given that the main focus of OpenBind appears to be generation of data for training what could be referred to as 'QSAR-like' ML models. In my view, QSAR modelling never made much of a splash in real world drug discovery and claims that particular models have made significant impact on drug discovery projects are generally not verifiable. A difficulty faced by QSAR practitioners was that projects had delivered or been put out of their misery by the time there was sufficient data for building predictively useful models. Medicinal chemists typically perform their optimizations within specific structural series and this means that structure-activity relationships (SARs) tend to be local in nature (I’m not aware of any studies in which a QSAR model built using only data from one structural series was convincingly shown to be usefully predictive of bioactivity for compounds in a different structural series). For users of ML bioactivity models it is important to know whether chemical structures for which predictions are being made lie within the applicability domains of the models. Put another way, medicinal chemists who use ML models are generally more interested in how well the models predict for the structural series that they're working on and less interested in how well the models have fit the training data (anybody who has received financial advice will be familiar with the "past performance is not indicative of future results" disclaimer). The selection criteria for inclusion of targets and ligands by the OpenBind initiative are not currently clear and I'm guessing that large scale structural determination might prove challenging for membrane proteins.  

The availability of affinity measurements that are linked to X-ray crystal structures for the corresponding protein-ligand complexes enables affinity to be modelled in terms of the molecular interactions between proteins and their ligands. This is the approach used to create the scoring functions used in virtual screening and it provides a means to address the local nature of SARs. While this might seem to be an obvious way to model affinity data it's important to be aware that the contribution to affinity of an individual contact, such as a hydrogen bond, between the protein and ligand is not an experimental observable (see NoLE). Put another way, there is no unique way of decomposing a value of ΔG° (standard Gibbs free energy of binding) into a sum of terms based on individual noncovalent contacts between the protein and ligand. One reason reason for this is that association of proteins with their ligands occurs in aqueous media and this point has been clearly articulated in the S2012 study:

Molecular binding in an aqueous solvent can be usefully viewed not as an association reaction, in which only new intermolecular interactions are introduced between receptor and ligand, but rather as an exchange reaction in which some receptor–solvent and ligand–solvent interactions present in the unbound state are lost to accommodate the gain of receptor–ligand interactions in the bound complex.

However, there’s another reason why there’s no unique way to decompose binding free energy into a sum of terms based on individual noncovalent contacts and here’s a well-known equation written a bit differently to how you normally see it written:

This shows that the value of ΔG° varies with the concentration, C°, that defines the standard state. By convention C° is set to 1 M although this is arbitrary and has no physical basis (see G1997) and this means that the binding free energy values encountered by drug discovery scientists are always negative (consider the feasibility of measuring a K_d value of greater than 1 M). Writing ΔG° as a sum of terms based on individual non-covalent contacts is challenging because each term needs to depend on C° while the sum of terms needs to reproduce the dependence of ΔG° on C°. This is discussed in NoLE and the problems can be seen more easily if you think about how you might write K_d as a product of terms based on individual non-covalent contacts. The dependence of ΔG° on C° has implications for interpretability of ML models for binding affinity.

My understanding is that scoring functions (see GPD2018 | WBS2017 | A2015 | C2012 | S2012 | F2004 | SR2001 | GHK2000 | MM1999 | E1997 | MSK1992)  used in virtual screening are generally not predictive of affinity to the extent that they can be routinely used in lead optimization. Perhaps it will be different for Boltz-2 (described in the P2025 preprint) although questions have been raised in BSR2026 as to whether Boltz-2 "truly relies on the physics of intermolecular interactions" and the term “absolute FEP” does ring some alarm bells for me. Various explanations have been offered for the typically underwhelming performance of scoring functions for affinity prediction including the usual suspects (protein flexibility, solvation and entropy). However, a much simpler explanation might be that scoring functions are trained to predict the difference in free energy between two states by only using the structure corresponding to one of the states.

I remain sceptical that it will prove feasible to build genuinely universal models for prediction of binding affinity from structures of protein-ligand complexes although I'll be very happy if my scepticism is shown to be unfounded. Describing energetics of target-ligand interactions in a general manner to enable ML modelling of affinity will be challenging because of the necessity to encode factors such as interaction potential, geometric dependence and solvent exposure (bear in mind that physics-based methods for prediction of affinity are already available and I'll direct readers to the Open Free Energy and open forcefield initiatives). While modelling affinity in terms of molecular interactions circumvents the need for training data to sample every conceivable combination of structural series with target, the need to meaningfully define applicability domains does not disappear.  My view is that when affinity datasets for different targets are combined for ML modelling, data should be split at the target level for cross-validation. This would entail splitting data so that each test set consists of only (and all) the data for a single target. I have argued in a previous post that the complexity (for example, number of parameters used to fit the training data) of models should be properly accounted for when comparing performance for ML models. 

Datasets generated by OpenBind are likely to also prove valuable for testing and development of physics-based approaches to affinity prediction such as use of simulation to calculate ‘absolute’ (ΔG°) and ‘relative’ (ΔΔG) free energy of binding.  Physics-based free energy calculations are typically more computationally demanding for ΔG° than for ΔΔG (a view expressed in B2009 is that it's generally easier to predict differences in property values for pairs of structurally-related compounds than it is to predict property values from chemical structures of compounds). Methods for calculating ΔΔG (here’s a helpful review) are especially relevant to drug design because medicinal chemists typically work within structural series, defining SARs in terms of ratios of affinity (or potency) for pairs of structurally-related compounds. Put another way, ΔΔG calculations enable project team scientists to exploit existing project data to predict affinity for potential synthetic candidates and I would argue that ML modellers really do need to be thinking more about prediction of differences in affinity (and other pharmaceutically relevant properties) between structurally related compounds. As an aside, free energy perturbation (FEP) was a major source of inspiration when I started to use the Leatherface (don't ask  😁😁😁) chemical structure editing software to do matched molecular pair analysis (MMPA) in the late 1990s, even though physics-based ΔΔG calculations were still largely seen as academic curiosities at that time.

While I’m certainly enthusiastic about physics-based methods such as FEP for calculating ΔΔG it’s not clear how generally these can handle significant modifications to the core of a structure (this is the scaffold-hopping scenario) and I would anticipate difficulties when the main effect of the structural perturbation is to alter conformational preference (as is the case for N-methylation of the secondary amide that is conserved in a number of SARS-CoV-2 main protease inhibitors). That said, the data generation capability of the OpenBind initiative should enable perceived weaknesses in FEP methodology to be addressed. I'll highlight a couple of general ways to use the data sets that OpenBind will generate might be used to validate methods for predicting relative affinity. First, you can use the relative affinity values that correspond to specific structural transformations such as chloro substitution (a good way to study activity cliffs and focusing on specific structural transformations counters criticism that predictive models are just capturing lipophilicity or molecular size), chloro to bromo (a good way to see if you're modelling halogen bonding effectively), and aromatic nitrogen to CH (in design it is useful to determine where polarity can be introduced with minimal loss of affinity). Second, you can use relative affinity measurements to assess how well models predict non-additivity in SARs (non-additivity can be also be considered in th activity cliff framework). I should point out that neither of these suggestions is novel (see L2012 and C2016) and activity to ML modellers are already looking at activity cliffs (see vT2022).     

This is a good point at which to wrap up and I'll be taking a look at the OpenADMET initiative in the next post.

Two new open data initiatives

I'll open the post with photos that I took in Seoul last year at the Dongdaemun Design Plaza (DDP) which was designed by Zaha Hadid (1950-2016) and I'm ashamed to admit to only having become aware of her ten years ago while wandering around the American University of Beirut (she studied mathematics at AUB and much later designed the building there that houses the Issam Fares Institute for Public Policy and International Affairs).

In the two posts that will follow the current post I’ll be taking a look at the OpenBind and OpenADMET initiatives. A key objective of each initiative is to generate large bodies of high-quality data that will be relevant to drug discovery and make these freely available in the public domain. I certainly see massive value in open data and consider it important that vital resources such as ChEMBL and BindingDB be funded generously. As we rightly celebrate the 2024 Nobel Chemistry Prize we also need to recognize the remarkable foresight of those who launched the Protein Data Bank in 1971 with just seven X-ray crystal structures. All that said, achieving a coverage of chemical space that enables usefully predictive models for diverse pharmaceutically relevant phenomena is likely to prove challenging and those leading the OpenBind and OpenADMET initiatives will need to make it clear as to how compounds are to be selected for assaying and synthesis. 

Artificial Intelligence (AI) is currently touted as a panacea for the various difficulties faced by drug discovery scientists and sometimes it seems that drugs will condense out of the ether if only the experimentalists would generate enough data. It’s worth pointing out that many (most?) discovery projects that deliver candidates for clinical development do so without ever having sufficient data for building machine learning (ML) models for everything that needs to be measured. In my view this counters arguments that ML models are essential for drug discovery although I’m certainly not denying an important role for usefully predictive ML models.  Many (most?) of the ML models built for drug design are essentially what we used to call quantitative structure-activity/property (QSAR/QSPR) models and I would not label them as AI even when they are used to assess chemical structures generated by AI. One challenge for ML modellers is that they need to demonstrate that their models are usefully predictive outside the chemical spaces in which they have been trained. I argued in this blog post that ML models cannot be compared simply on the basis of how well they fit the data on which they have been trained and that it is necessary to account for model complexity (typically quantified by the number of adjustable parameters used to fit the data) when asserting that one ML model is superior to another.   

Recently I posted on the objectives of drug design in a way that I hoped would be useful for drug discovery scientists using AI and ML in design.  Drug action is driven by concentration (it’s more accurate to describe affinity as sensitivity to a driving force than as a driving force in its own right) and another way of stating this is that the effects of a drug on the human body are determined by the concentration of the drug that is ‘seen’ by its target(s) and anti-targets. A range of bioactivity assays are used by drug discovery scientists to assess the effects of compounds on targets and anti-targets (cell-based assays are also used to assess potential toxicity) and the QSAR field came into being to enable prediction of bioactivity from chemical structures.

It’s easy understand the pharmacological objectives of drug design which can be stated as “hit the targets” and “don’t hit the anti-targets”. Thinking in terms of concentration is a bit more difficult and it’s important to be aware that the concentration of a drug at its site(s) of action (usually referred to as exposure) is generally not something that you can measure unless the target(s) are in direct contact with plasma and I recommend that everybody working in drug discovery and chemical biology read the SR2019 study. Uncertainty in exposure for intracellular targets is also an issue in clinical development because failure to meet end points in a Phase 2 trial might simply be the result of inadequate exposure. I have argued in NoLE, HBD3 and this blog post that controllability of exposure should be seen as one of the objectives of drug design.

Not being able to measure the concentration of a drug at its site of action complicates drug discovery but is an issue that can be addressed. For example, we can invoke the free drug hypothesis (‘principle’ and ‘theory’ are also used in this context although I personally prefer ‘hypothesis’) by assuming that the concentration of the drug at its site(s) of action is equal to its unbound plasma concentration (which can be measured).  Some of this has been discussed in my post on the objectives of drug design and, in any case, I’ll be covering pharmacokinetic aspects of drug design in more detail when I post on the OpenADMET initiative.

This is a good point at which to conclude. I’ll examine OpenBind in the next post and OpenADMET in the post after that.  

PAINS and editorial policy

<< previous || next >>

I have blogged previously (1 | 2 | 3 | 4 | 5 ) on PAINS. In this post, I present the case against inclusion of PAINS criteria in the Journal of Medicinal Chemistry (JMC) Guidelines for Authors (viewed 22-Jan-2017) as given below:

"2.1.9. Interference Compounds. Active compounds from any source must be examined for known classes of assay interference compounds and this analysis must be provided in the General Experimental section. Compounds shown to display misleading assay readouts by a variety of mechanisms include, but are not limited to, aggregation, redox activity, fluorescence, protein reactivity, singlet-oxygen quenching, the presence of impurities, membrane disruption, and their decomposition in assay buffer to form reactive compounds. Many of these compounds have been classified as Pan Assay Interference Compounds (PAINS; see Baell & Holloway, J. Med. Chem. 2010, 53, 2719-2740 and webinar at bit.lyj/mcPAINS). Provide firm experimental evidence in at least two different assays that reported compounds with potential PAINS liability are specifically active and their apparent activity is not an artifact."

The term 'known classes of assay interference compounds' must be defined more precisely in order to be usable by both authors submitting manuscripts and reviewers of those manuscripts. Specifically, the term 'known classes of assay interference compounds' implies the existence of a body of experimental data in the public domain for which specific substructural features have been proven to cause the observed assay interference. The JMC Guidelines for Authors (viewed 22-Jan-2017) imply that any assay result for a compound with 'potential PAINS liability' should necessarily be treated as less informative than would be the case if the compound did not have 'potential PAINS liability'. I shall term this as 'devaluing' the assay result.              

The PAINS acronym stands for Pan Assay INterference compoundS and it was introduced in a 2010 JMC article (BH2010) that is cited in the Guidelines for Authors (viewed 22-Jan-2017). The PAINS filters introduced in the BH2010 study are based on analysis of frequent-hitter behavior in a panel of 6 AlphaScreen assays. Each PAINS filter consists of a substructural pattern and is associated with an enrichment factor that quantifies the frequent hitter behavior. Compounds that quench or scavenge singlet oxygen have the potential to interfere with AlphaScreen assays but individual PAINS substructural patterns were not evaluated for their likelihood of being associated with singlet oxygen quenching or scavenging. For example, the BH2010 study makes no mention of studies ( 1 | 2 | 3 | 4 ) linking singlet oxygen quenching/scavenging to the presence of a thiocarbonyl group which is a substructural element present in rhodanines.  

I argue that a high hit rate against a small panel of assays that all use a single detection technology is inadmissible as evidence for pan-assay interference.  I also argue that the results of screening against this assay panel can only be invoked to devalue the result from an AlphaScreen assay. In a cheminformatic context, the applicability domain of a model based on analysis of results from this assay panel is restricted to activity measured in AlphaScreen assays. Furthermore, it is questionable whether it is valid to invoke the results of screening against this assay panel to devalue a concentration response from an AlphaScreen assay because the results for each assay of the panel were obtained at a single concentration (i.e. no concentration response).

The BH2010 study does present some supporting evidence that compounds matching PAINS substructural patterns are likely to interfere with assays. In a cheminformatic context, this supporting evidence can be considered to extend the applicability domain of PAINS filters. However, supporting evidence is only presented for some of the substructural patterns and much of that supporting evidence is indirect and circumstantial. For example, the observation that rhodanines as a structural class have been reported as active against a large number of targets is, at best, indirect evidence for frequent hitter behavior which is characterized by specific compounds showing activity in large numbers of assays. There is not always a direct correspondence between PAINS substructural patterns and those used in analyses that are presented as supporting evidence. For example, the BH2010 study uses substructural patterns for rhodanines that specify the nature of C5 (either saturated or with exocyclic carbon-carbon bond).  However, the sole rhodanine definition given in the BMS2006 study specifies an exocyclic carbon-carbon double bond. This means that it is not valid to invoke the BMS2006 study to devalue the result of every assay performed on any rhodanine.

The data (results from 6 AlphaScreen assays and associated chemical structures) that form the basis of the analysis in the BH2010 study are not disclosed and must therefore be considered to be proprietary. Furthermore, some of the supporting evidence that compounds matching PAINS filters are likely to interfere with assays is itself based on analysis (e.g. BMS2006 and Abbott2007) of proprietary data. The JMC Guidelines for Authors (viewed 22-Jan-2017) make it clear that the use of proprietary data is unacceptable:

"2.3.5.2 Proprietary Data. Normally, the use of proprietary data for computational modeling or analysis is not acceptable because it is inconsistent with the ACS Ethical Guidelines. All experimental data and molecular structures used to generate and/or validate computational models must be reported in the paper, reported as supporting information, or readily available without infringements or restrictions. The Editors may choose to waive the data deposition requirement for proprietary data in a rare case where studies based on very large corporate data sets provide compelling insight unobtainable otherwise.

2.3.6 QSAR/QSPR and Proprietary Data. The following are general requirements for manuscripts reporting work done in this area:

(3) All data and molecular structures used to carry out a QSAR/QSPR study are to be reported in the paper and/or in its supporting information or should be readily available without infringements or restrictions. The use of proprietary data is generally not acceptable."

Given JMC's stated unacceptability of analysis based on proprietary data, to use such analysis to define editorial policy would appear to contradict that editorial policy.

To sum up:

• Analysis of the screening results for the BH2010 assay panel can only be invoked  invoked to devalue or otherwise invalidate the result from an AlphaScreen assay.

• Additional supporting evidence is only provided in BH2010 for some of the PAINS filters. In these cases, the evidence is not generally presented in a manner that would allow a manuscript reviewer to assess risk of assay interference in an objective manner.

• Most of the analysis presented in the BH2010 study has been performed on proprietary data. To base JMC editorial policy on analysis of proprietary data would appear to contradict the Journal's policy on the use of proprietary data.

I rest my case.

On open access

I’ve been meaning to write something about open access (OA) for some time and I do genuinely believe that true OA is a worthwhile (and achievable) ideal. This blog post is not intended as a critique of OA although I will suggest that its advocates do need to think a bit more carefully about how they present the benefits. I'll be writing from the perspective of authors of scientific journal articles and disclose, in the interests of openness, that all but one of my publications have been in paywall journals. One general observation that I will make is that the sanctimonious, holier-than-thou attitude of some OA advocates (who may occasionally avail themselves of the advantages of publishing in paywall journals) does tend to grate at times.  Although there is actually a lot more to OA than who pays for journal articles to be published, I'll not be discussing those issues in this post. The principal theme of this post is that it may be helpful for those who opine loudly on the benefits of OA to take a more author-centric look at the relevant issues and this is especially important for those advocates whose contributions to the scientific literature have been modest.  

One of my gripes is that OA is sometimes equated with open science and I believe that the openness of science is actually a much bigger issue than who pays for scientific articles to be published in journals. Drug discovery research is my primary focus and a certain amount of what (we think that) we know in this field comes from analysis of proprietary databases.  Some of this analysis falls short of what one might feel entitled to expect from journals that would describe themselves as premier journals and you’ll find some examples in this presentation.  I have argued here and here that studies of this genre would actually pack a much heavier punch if the data was made publicly available. However, it is not only datasets that are undisclosed in drug discovery papers.  In some cases, models built from the data are not disclosed either (this does not inhibit authors from comparing their models favorably with previously published models). If the science in a data modelling study is to be accurately described as open then the data and models associated with the study have to be disclosed. This is not just an issue for paywall journals and the models described in the following OA articles do not appear to be fully disclosed:

JCheminform2010:2:11 | JCheminform2012:4:29 | JCIM2014:54:944 
 JCIM2015:58:4066 | PLoSNTD2015:9:e0003878

The point that I'm making here is that a the science in a paywall journal is not necessarily less open than that in an OA journal. In my view, the openness of the science presented in a journal article is determined by how much its authors disclose rather than whether or not one needs to pay to read the article. It's also worth remembering that internet access is not free.

This is a good point to say say something about the economics of publishing articles in scientific journals. In economic terms, a journal article represents a service that the journal has provided for the author rather than the other way round (which many would regard as more sensible).  The author typically pays for the service by either signing over rights to paywall journal or by paying an article processing charge (APC) to a journal (OA or paywall) which means that the article can be freely viewed online and rights are (hopefully) retained by the author.  I need to note that not all OA journals levy APCs and, in these cases, it's a good idea to check how the journal is funded and whether or not articles are peer-reviewed. Payment of an APC implies that the journal article is technically an advertisement and this has interesting (and potentially even troubling) implications if the journal also publishes conventional advertisements. All articles in a paywall journal are usually considered (e.g. by download cost) to be of equal value and this is one of the bigger fictions in the scholarly publishing business.  The situation becomes more complicated with OA journals because the APC may vary with geographical region or be waived completely for some individuals. In the interest of openness and accountability, shouldn't open access journals disclose the actual APC paid for each article published?

There are compelling arguments that can be made in support of OA and, like many, I believe that the results of publicly-funded research should be fully visible to the public. This is probably a good point at which to raise the question of who should own intellectual property generated by publicly-funded research. OA broadens participation (by citizen scientists and emerging scientific communities in developing countries) and increases the likelihood of a study being read (and used) by the people with the most interest in it. OA greatly facilitates mining of the ever-expanding body of published literature which adds to its value and, unsurprisingly, some of the loudest voices calling for scientific literature to be made public property are those of informaticians and data scientists.  One question that OA advocates need to address is whether the results of mining and analysis of freely available literature should also be considered to be public property. While a strong and coherent case can be made for OA, it might not be such a great idea to invoke morality (can get a bit subjective) or fundamental human rights (people under siege in Syria might consider it less than tasteful that their suffering is being equated with their inability to access the latest articles in Nature).

There are a few hurdles that will need to be overcome if OA vision is to translate to OA reality. Advocates for OA typically demonize the paywall journals although the cynic might note that easily-recognized enemies are needed by the leaders of mass movements in order to galvanize the proletariat. Some declare that they will only review manuscripts for open access journals while others assert that they are boycotting Nature and Science (I scarcely dare to think of the mirth of that would result from me announcing to colleagues that I will no longer be submitting manuscripts to Nature). While demonizing paywall journals may make people feel good (and righteous), it misses the essential point that the OA movement needs to win hearts and minds so that authors (the creators of content) come to see OA journals as the most attractive option for publishing their studies. Put more bluntly, authors are as at least as much to 'blame' as the journals for their articles 'disappearing' behind paywalls. It's also worth remembering that journal editors are also usually authors as are the people on academic hiring committees who might use journal impact factor to to decide that one research output of one applicant is worth more than that of another applicant.

One nettle that the OA movement does need to grasp is that journal publishing is frequently a business and, from the perspective of the 'customer', the terms 'paywall' and 'open access' simply describe alternative business models.  I'm certainly not saying that it has to be that way and there are other areas, such as the provision of health care, education and religion, where a case can be made for the exclusion of commercial activity.  However, the issue remains and, like or not, the OA movement is effectively lobbying in favor of one set of commercial interests over another set of commercial interests. This means that OA advocates need to be clear (and open) about the sources of any funding that should come their way.  The other nettle that the OA movement needs to grasp is that it should be very careful about whom it allows to speak on its behalf.  OA propaganda sometimes takes the form 'open access good, paywall bad' and the message loses some of its bite if it turns out that Napoleon has been publishing in Nature while denouncing Beall's List as reactionary and counter-revolutionary.

So let's take a look at OA from the perspective of a scientist who is trying to decide where to send a manuscript.  Typically the decision will be made by the principal investigator (PI) who will be well aware that funding and, if relevant, tenure decisions will be influenced by which journals in which the PI's articles have been published.  In some areas (e.g. biomedical), academic research is particularly competitive and PIs are well aware that an article published in an elite journal is visible to the PI's peers and all those who will be making decisions about the PI's future.  As such, publishing in an OA journal can represent a risk without compensating benefits (in competitive academic science, knowing that everybody on the planet can see your article fits into the 'nice to have' category). Another factor that inhibits authors from publishing in OA journals is the paucity of options in some disciplines (e.g. chemistry). I don't believe that there is currently a credible OA equivalent to Journal of Medicinal Chemistry and would be concerned about the ability of a general purpose OA journal to identify and recruit specialist reviewers. I'm not a computational biologist but, if I were, I'd definitlely be thinking twice about submitting an article to a computational biology journal that published Ten Simple Rules of Live Tweeting at Scientific Conferences. Finally there is the issue of cost because publishing in OA journals (or paying for OA in a paywall journal) does not usually come cheap and, in some cases, funding needs to be specifically sourced (as opposed to simply paying out of one's grant). Those scientists for whom the APC represents a significant expense will also be aware that they are effectively subsidizing other scientists for whom the APC has been waived. Something that may be going through the mind of more than one scientist is the direction in which APCs might head should all scientific publishing be forced to adopt an OA business model.

I think this is a good point at which to wrap things up and I should thank you for staying with me.  If you're an OA opponent and had been hoping to see OA skewered then please accept my apologies for having disappointed you. If you're an OA supporter, you may well disagree with many (or all) of the points that I've raised. If, however, something in this post has prompted you to think about things from a different angle then it will have achieved its purpose.