Abstract
Metabolic networks are fundamental to cellular processes, driving energy production, biosynthesis, redox regulation, and cellular signaling. Recent advancements in metabolic research tools have provided unprecedented insights into cellular metabolism. Among these tools, the extracellular flux analyzer stands out for its real-time measurement of key metabolic parameters: glycolysis, mitochondrial respiration, and fatty acid oxidation, leading to its widespread use. This review provides a comprehensive summary of the basic principles and workflow of the extracellular flux assay (the Seahorse assay) and its diverse applications. We highlight the assay's versatility across various biological models, including cancer cells, immunocytes, Caenorhabditis elegans, tissues, isolated mitochondria, and three-dimensional structures such as organoids, and summarize key considerations for using extracellular flux assay in these models. Additionally, we discuss the limitations of the Seahorse assay and propose future directions for its development. This review aims to enhance the understanding of extracellular flux assay and its significance in biological studies.
Keywords: Electron transport chain, Extracellular flux assay, Glycolysis, Metabolism, Mitochondria, Seahorse assay
INTRODUCTION
The metabolic network, an intricate system of biochemical processes within cells, constitutes the cornerstone of life-sustaining functions. The metabolic network performs diverse functions, ranging from energy production to the biosynthesis of ribose sugars crucial for DNA/RNA synthesis, metabolites regulating redox status, and the modulation of protein functions through the addition of metabolic intermediates, such as acetyl or methyl groups during the post-translational modification step. Furthermore, the metabolites produced in the metabolic network play a role in facilitating cellular signaling (Lee and Kim, 2016, Li et al., 2019, Pavlova et al., 2022). Understanding the dynamics of cellular metabolism is pivotal not only for unraveling the fundamental principles of life but also for elucidating the pathophysiology of various diseases and developing novel therapeutic interventions, as altered metabolism is frequently associated with disease states. For example, sphingolipid metabolites, such as ceramide, increase susceptibility to metabolic dysfunctions (Green et al., 2021), alterations of NAD+ metabolism are associated with neurodegeneration and diabetes (Katsyuba et al., 2020), and diverse metabolic pathways, such as glycolysis, glutaminolysis, or lipid metabolism, are frequently altered in cancer (Pavlova et al., 2022).
Among various metabolic reactions, the O2-consuming reaction in mitochondria and lactate production occurring in glycolysis are key parameters for evaluating the metabolic state of cells. Thus, methods for quantifying oxygen consumption and lactate production have been conceived and utilized for research. Several methods have been used for measuring metabolic parameters, such as Warburg manometry, Clark electrodes, and chromatography coupled with mass spectrometry. However, these methods have distinct limitations. Warburg manometry measures oxygen consumption and lactate production in isolated tissues but is labor-intensive, requires large tissue samples, and lacks sensitivity for small-scale cellular measurements (Glickman et al., 1949). The Clark electrode, which uses an oxygen electrode to measure oxygen consumption in cell suspensions or tissues, is limited to measuring only oxygen consumption and requires invasive sample preparation (Li and Graham, 2012). Lastly, chromatography and mass spectrometry analyze metabolic intermediates and end-products but necessitate extensive sample preparation, involve expensive equipment, and provide only endpoint or specific time point measurements rather than real-time analysis (Jang et al., 2018).
The extracellular flux analyzer, also known as the Seahorse Extracellular Flux (XF) analyzer, was first introduced in 2006 by Seahorse Sciences and is extensively utilized in cellular metabolism research. It offers advantages over traditional methods by providing real-time, noninvasive measurements of key metabolic parameters (Caines et al., 2022, Gu et al., 2021). It is highly sensitive, suitable for small cell populations, and allows for comprehensive metabolic profiling of both mitochondrial respiration and glycolysis. The extracellular flux assay has been applied to evaluate mitochondrial function, different types of energy metabolism, drug effects and toxicity, cellular stress responses, and the metabolic characteristics of cells (Caines et al., 2022, Espinosa et al., 2021, Gu et al., 2021, Pan et al., 2022, Vander Heiden et al., 2009). By providing detailed insights into the metabolic states associated with various diseases, the extracellular flux assay has significantly advanced our understanding of cell biology and medical research through diverse experimental models. This review aims to provide an overview of the principles underlying extracellular flux assay technology and its wide-ranging applications in cellular metabolism research.
THE PRINCIPLE OF EXTRACELLULAR FLUX ANALYZER: REAL-TIME MONITORING OF CELLULAR ENERGY METABOLISM
Glycolysis and oxidative phosphorylation are pathways for adenosine triphosphate (ATP) production, serving as fundamental sources of cellular energy (Greene et al., 2022, Zheng, 2012). The extracellular flux assay quantitatively evaluates the glycolytic and oxidative phosphorylation capacities in various live or freshly prepared samples.
The process of glycolysis is a metabolic pathway in which 1 glucose molecule is converted into 2 pyruvate molecules, regardless of the presence of oxygen (Chandel, 2021). This process leads to ATP production and lactate formation, which acidifies the extracellular environment. Consequently, changes in extracellular pH can serve as a proxy for the glycolytic capacity of cells (Fig. 1). Oxidative phosphorylation, on the other hand, is the metabolic process through which ATP is synthesized by transferring electrons from nicotinamide adenine dinucleotide (hydrogen) (NADH) and flavin adenine dinucleotide (hydrogen) (FADH2) to oxygen via the electron transport chain (ETC). This electron transfer generates a mitochondrial membrane potential, which ultimately promotes ATP synthesis by the activity of ATP synthase (Nolfi-Donegan et al., 2020). Pyruvate enters the mitochondria and is processed to acetyl-CoA, which enters the tricarboxylic acid (TCA) cycle, generating NADH and FADH2. These molecules act as electron donors at complexes I and II of the ETC, respectively, creating a membrane potential gradient that drives ATP synthesis (Fig. 1). Oxygen acts as the final electron acceptor in the ETC and is continuously consumed while the ETC is active (Fig. 1). Therefore, quantifying oxygen consumption can serve as a proxy for mitochondrial respiration. Consequently, the Extracellular Acidification Rate (ECAR) and Oxygen Consumption Rate (OCR) are utilized to monitor cellular energy metabolism (Fig. 1).
Fig. 1.
ECAR and OCR as proxies of glycolysis and mitochondrial respiration in Seahorse assay. Glucose is metabolized to pyruvate through glycolysis. The pyruvate can be converted to lactate and protons (H+), leading to extracellular acidification. Fatty acyl-CoA is converted to acyl-carnitine by carnitine palmitoyltransferase-1 (CPT-1) to enter the mitochondria, where it is converted back to fatty acyl-CoA and undergoes FAO to produce acetyl-CoA. The acetyl-CoA generated from glycolysis and FAO enters the TCA cycle, leading to ATP production through oxidative phosphorylation, which consumes oxygen in mitochondria. Thus, the quantification of ECAR or OCR represents the capacity of glycolytic and mitochondrial respiration, respectively. (a) General patterns of the graph during testing the glycolytic functions using the Seahorse assay. (b) General patterns of the graph obtained during the Mitostress test for in-depth analysis of mitochondrial function. (c) General results for the measurement of FAO using the Seahorse assay.
The extracellular flux analyzer is a specialized instrument used to measure cellular metabolism in real-time (Caines et al., 2022, Wang et al., 2015). It operates by analyzing changes in the levels of protons (H+) and oxygen (O2) in the extracellular environment surrounding live cells. To facilitate these measurements, the XF analyzer utilizes a cartridge system. Each cartridge contains sensors necessary for monitoring both ECAR and OCR simultaneously, and four ports for reserving the inhibitors which are injected into the medium while running the extracellular flux assay for in-depth analysis of the glycolysis and mitochondrial respiration. The sensors embedded within the cartridge are designed to detect alterations in pH caused by the release of protons during glycolysis (ECAR measurement) and fluctuations in oxygen levels resulting from cellular respiration (OCR measurement) in real-time. Overall, the XF analyzer, with its cartridge system, enables the monitoring of cellular metabolism by measuring key parameters, such as ECAR and OCR, in real-time. This capability contributes to a better understanding of cellular energetics and metabolic pathways, such as glucose metabolism and lipid metabolism.
PRINCIPLES OF IN-DEPTH ANALYSIS OF GLYCOLYSIS, MITOCHONDRIAL RESPIRATION, AND FATTY ACID OXIDATION
Depending on the experimental model employed, the overall experimental timeline may vary, but assays with cells typically span 2 days if the concentration of drugs injected during the assay is optimized. Figure 2 outlines the general workflow of the extracellular flux assay, and detailed protocols are provided in previous methods papers (Caines et al., 2022, Gu et al., 2021, Wang et al., 2015, Zhang and Zhang, 2019). The primary function of the XF analyzer is to quantify protons and O2 levels in the media, enabling the assessment of basal glycolytic capacity and oxidative phosphorylation. However, for a more comprehensive analysis of glycolytic capacity or mitochondrial respiration, various drugs targeting glycolysis or components of the ETC are often utilized during the assay. This section aims to introduce the principles behind in-depth characterization of mitochondrial respiration, glycolytic capacity, and the fatty acid oxidation (FAO) pathway. Additionally, it will cover the common drug compounds required for this detailed analysis.
Fig. 2.
The workflow of Seahorse assay. The Seahorse assay process includes steps from optimization of conditions to data interpretation. The optimization step involves drug optimization (eg, oligomycin, FCCP) and optimization of sample quantity (eg, cell seeding number, quantity of mitochondria). Generally, 1 day before the assay, the cartridge is hydrated, and cells are seeded on the assay plate. On the day of the assay, the media is replaced with the assay media, drugs are prepared and loaded into the ports, and the program is run once the cartridge calibration is finished. When the program ends, cells or materials are subjected to further quantification for normalization. After the data are normalized, it is interpreted.
Measurement of Mitochondrial Respiration
The analysis of mitochondrial respiratory capacity is one of the primary purposes of utilizing the extracellular flux assay. Mitochondrial respiration capacity is reflected by the OCR, as oxygen is used as an electron acceptor at complex IV. For an in-depth analysis of mitochondrial function, the “Mitostress” test is employed. This test uses inhibitors of ETC components to determine how much OCR comes from mitochondrial and nonmitochondrial sources, how much OCR is consumed for making ATP, and the maximum capacity of mitochondrial respiration (Fig. 1b). The inhibitors used for the Mitostress test are as follows:
-
●
Oligomycin: Oligomycin is an inhibitor of complex V (ATP synthase) and prevents ATP formation in the ETC. After measuring the basal OCR, oligomycin from the port is injected into the medium, leading to a significant drop in OCR. The OCR value before oligomycin injection represents the sum of “basal respiration” and “nonmitochondrial respiration,” while the decrease in OCR after oligomycin injection represents “ATP-linked respiration”. The remaining OCR after oligomycin treatment indicates the portion of oxygen consumption that is not linked to ATP production, known as “uncoupled respiration” via proton leak.
-
●
FCCP (carbonyl cyanide-p-(trifluoromethoxy)phenylhydrazone): FCCP uncouples oxidative phosphorylation by dissipating the proton gradient. As a result, cells maximize their mitochondrial respiration capacity as they try to maintain the proton gradient. Therefore, the OCR values after FCCP treatment are referred to as the maximum OCR.
-
●
Rotenone and antimycin A: Rotenone and antimycin A are inhibitors of complexes I and III, respectively. Treatment with these drugs results in the complete deactivation of the electron transport system because complex IV can no longer receive electrons from complexes I and III. As a result, only nonmitochondrial O2-consuming reactions are detected, which are defined as “nonmitochondrial oxygen consumption.”
The assay medium for the Mitostress test contains general nutrients used for making the substrates for mitochondrial respiration via glycolysis and TCA cycle, such as pyruvate and glutamine, as well as glucose.
Measurement of Glycolytic Function
The capacity of the glycolytic pathway can be assessed by measuring the ECAR. ECAR values are obtained whenever experiments are conducted using the XF analyzer. Therefore, even when conducting experiments to measure mitochondrial capacity through the Mitostress test, information on ECAR can always be obtained. However, to measure glycolytic capacity more precisely, specific inhibitor compounds and assay media for glycolytic capacity measurement should be used. Unlike the Mitostress test, which begins with pyruvate and glucose in the basal media to activate mitochondrial respiration through the formation of NADH and FADH2 in the TCA cycle, the basal media for measuring glycolytic function lacks pyruvate and glucose. During the assay, glucose is supplemented, and oligomycin and 2-deoxyglucose (2-DG) are sequentially administered. The purpose of each compound injected during the measurement of glycolytic rate is indicated below (Fig. 1a).
-
●
Glucose: As glucose is the initial substrate of glycolysis, its addition initiates glycolysis, allowing the measurement of glycolysis-linked ECAR.
-
●
Oligomycin: Oligomycin blocks ATP synthase in the mitochondria's ETC, inducing maximum glycolytic activity in the cell to compensate for the blocked ATP production from the mitochondria. This allows for the measurement of the “glycolytic capacity” of the cells.
-
●
2-DG: 2-DG is a glucose analog with a hydroxyl group replaced by hydrogen at the second carbon. While it can be transported into cells similarly to glucose, it inhibits glycolysis. Once converted to 2-deoxy-D-glucose-6-phosphate within cells, it binds to hexokinase, an enzyme that uses glucose as a substrate in glycolysis, thereby halting glycolysis from progressing further. This effectively shuts down glycolysis-linked ECAR levels, confirming that the increased ECAR by the addition of glucose and oligomycin is indeed from the activity of glycolysis.
Measurement of fatty acid oxidation
FAO, also known as beta-oxidation, is a metabolic pathway that breaks down fatty acids into acetyl-CoA, which then enters the TCA cycle to produce NADH and FADH2 for ATP production via the ETC. This process occurs in the mitochondria and peroxisomes and is crucial for producing ATP, the cell's primary energy source. When free fatty acids enter the cell, they are converted into fatty acyl-CoAs in the cytosol. Fatty acyl-CoAs are then transported into the mitochondrial matrix through the carnitine shuttle, where they are converted into acyl-carnitine for transport across the mitochondrial membrane. Once inside the mitochondria, acyl-carnitine is converted back into fatty acyl-CoA, which undergoes beta-oxidation. This process produces acetyl-CoA, NADH, and FADH2. The acetyl-CoA generated from FAO enters the TCA cycle, where it is further processed to produce additional NADH and FADH2. These reducing equivalents are utilized in the ETC to generate ATP through oxidative phosphorylation (Rinaldo et al., 2002).
To measure mitochondrial respiration derived from FAO, palmitate-bovine serum albumin (BSA) and L-carnitine are added to the assay media, which does not contain other substrates such as glucose, pyruvate, and glutamine. Carnitine, an ammonium compound, facilitates the transport of fatty acyl-CoA into the mitochondrial matrix, which is essential for FAO (Mendoza et al., 2022). Using BSA-conjugated fatty acids instead of free fatty acids better mimics physiological conditions. Free fatty acids are hydrophobic and can form micelles or precipitate, causing inconsistent results. Conjugating them to BSA enhances solubility and stability. In the bloodstream, fatty acids naturally bind to albumin, so using BSA-conjugated fatty acids replicates physiological conditions more accurately and reduces toxicity (Spector, 1975, van der Vusse, 2009). This method ensures efficient uptake and accurate measurement of FAO in assays.
The assay media is prepared in 2 types: (1) BSA control and (2) palmitate-BSA. Palmitate, a 16-carbon chain fatty acid, serves as a substrate for β-oxidation. Samples in palmitate-BSA media exhibit higher OCR values compared to those in BSA control (Fig. 1c). This difference indicates that mitochondrial respiration driven by FAO can be measured separately from respiration mediated by glucose and other nutrients. This setup allows for the assessment of mitochondrial respiration specifically linked to FAO, providing insights into how cells utilize fatty acids as an energy source under different conditions. The following drugs are inhibitors used in the assay (Fig. 3):
-
●
Etomoxir: Etomoxir, an inhibitor of carnitine palmitoyltransferase I, blocks FAO by irreversibly preventing the formation of acylcarnitines (O'Connor et al., 2018). Etomoxir can be administered either during the assay run or in advance.
-
●
Oligomycin, FCCP, rotenone/antimycin A: Refer to “Measurement of Mitochondrial Respiration” for detailed descriptions of these compounds.
Fig. 3.
Targets of the drugs utilized in the Seahorse assay. In the measurement of OCR, oligomycin, FCCP, and rotenone/antimycin A are sequentially utilized. Oligomycin, an inhibitor of ATP synthase, allows the measurement of ATP-linked respiration. FCCP, a mitochondrial uncoupler, disrupts the proton gradient across the mitochondrial membrane, inducing maximal respiration. Rotenone and antimycin A are typically used together as inhibitors of complexes I and III, respectively. Their action results in the complete shutdown of the electron transport chain. In the FAO measurement, palmitate serves as the substrate for FAO, while etomoxir acts as an inhibitor that prevents FAO by blocking CPT-1. In the measurement of ECAR, glucose, oligomycin, and 2-DG are utilized. Glucose transporters (GLUTs) facilitates the transport of both glucose and 2-deoxyglucose (2-DG) across the plasma membrane. Glucose is the substrate for glycolysis. Oligomycin in ECAR measurements blocks the electron transport chain, maximizing glycolysis. 2-DG, a glucose derivative and competitive inhibitor of glycolysis, is phosphorylated to 2-deoxyglucose-6-phosphate (2-DG-6-P), which terminates glycolysis immediately by binding to hexokinase.
The measurement of FAO requires the following 4 conditions for an in-depth analysis of FAO: (1) BSA-control, -etomoxir, (2) BSA-control, +etomoxir, (3) palmitate-BSA, -etomoxir, (4) palmitate-BSA, +etomoxir. If FAO is not actively occurring in the sample, only a minimal difference between the OCR values in conditions (3) and (4) is observed. In some cases, condition (2) may be omitted (Nomura et al., 2016). This setup allows for a detailed analysis of the contribution of FAO to mitochondrial respiration by comparing the OCR values under these different conditions.
Above, we summarized the measurement of glycolysis, mitochondrial respiration, and FAO capacity using 3 widely used assays. However, the extracellular flux assay can be modified and applied for various purposes by altering the assay media, substrates, and drug combinations. For example, depending on the substrate provided, the assay can be further diversified. For instance, to measure energy production via glutamine in cells, OCR can be measured using glutamine as the substrate instead of glucose.
Normalization
The levels of OCR and ECAR values are correlated with the number of cells in each well. Therefore, a normalization step is required to minimize noise from errors introduced during cell seeding or sample preparation. There are multiple options for normalizing extracellular flux assay data. The most common and accurate approach is to normalize the values from each well to the number of cells in the corresponding well. Cytation5 is an automated imager that counts the cell number by staining the nucleus of cells with Hoechst immediately after running the assay. For the application of Cytation5 for cell counting, the nuclear staining dye, such as Hoechst, should be injected with the drugs during the last injection step of the assay. The normalized data can be obtained automatically within the Seahorse Wave Desktop Software (Agilent), making the analysis more convenient (Little et al., 2020). In addition to Cytation5, there are other methods available for cell number normalization. Manual counting using a hemocytometer or an automated cell counter can be employed before seeding the cells in the assay plate. Another alternative is using fluorescence microscopy combined with a nuclear stain (eg, 4',6-Diamidino-2-phenylindole (DAPI) or Hoechst) to count the number of cells after the assay manually or using automated image analysis software.
Normalization to cellular protein is also available. In this method, cells in each well that have been measured are lyzed in lysis buffer, such as radioimmunoprecipitation assay (RIPA) buffer, and then quantified using the Bradford or BCA protein assay (Brunton et al., 2020). However, it is important to keep in mind that “accurate” quantification of proteins can be challenging due to the relatively small number of cells required for the assay. The results of isolated mitochondria samples can be normalized based on the protein concentration measured before the assay. Mitochondria typically do not require postassay normalization because, unlike cells, they do not exhibit significant numerical differences before and after measurement (Underwood et al., 2020). For tissue samples or three-dimensional (3D) samples, normalization can be achieved based on weight (Miller et al., 2017). Overall, choosing the most effective normalization method according to the material used in the experiment will yield higher-quality data.
APPLICATIONS OF EXTRACELLULAR FLUX ASSAY IN VARIOUS MODEL SYSTEMS
The extracellular flux assay has been applied across a range of models and disciplines. This section outlines its diverse applications and underscores the importance of considering specific factors in different models or contexts. We delve into addressing the unique differences and considerations crucial for experimentation in each model (Fig. 4).
Fig. 4.
Applications of Seahorse assay. The Seahorse assay is applicable in various models, from cancer cells to model organisms such as C. elegans, for in-depth analysis of mitochondrial function and glycolysis in metabolism.
Cancer cells
Extracellular flux assay in cancer research is frequently used to analyze the energetic characteristics of various cancer cell lines (Caines et al., 2022). As with other cell types, the optimization of inhibitor concentrations used in the assay is required for each cancer cell line. Typically, treatment with antimycin A/rotenone at a concentration of 0.5–1 μM effectively inhibits complex I/III, resulting in a significant drop in OCR (Caines et al., 2022, Gu et al., 2021). However, the optimal concentration of oligomycin or FCCP varies depending on the cell line. Therefore, it is necessary to run optimization sets with various doses of oligomycin and FCCP. If the extracellular flux assay has been previously performed on the same cell line under similar conditions, the optimization step can be skipped by following the previously optimized conditions. For suspension cancer cell lines, it is important to seed the cells just before running the assay in the assay media on a plate precoated with poly-D-lysine (Erdem et al., 2022, Silic-Benussi et al., 2022). The poly-D-lysine helps ensure suspension cells attach well to the plate, improving the consistency and accuracy of OCR and ECAR measurements.
Immunocytes
Various types of immune cells, including macrophages and T-cells, can be used as samples for this application (Pereira et al., 2019, Valle-Casuso et al., 2019). However, for some sensitive cells, such as naïve T-cells, it is recommended to use alternative uncouplers such as BAM15, as FCCP may induce cytotoxic effects and negatively impact the data (Istomine et al., 2023, Kostic et al., 2018). Since immunocytes are suspension cells, precoated assay plates with poly-D-lysine should be utilized. The detailed protocol for the analysis of immunocytes is provided in a previous protocol paper (Scholler-Mann et al., 2021).
Caenorhabditiselegans
The C. elegans model is extensively employed, particularly in developmental and aging studies, due to its rapid lifecycle and ease of cultivation (Kenyon, 2010, Riddle et al., 1997). Additionally, it serves as an excellent model for investigating systemic metabolism and relevant diseases such as obesity, benefiting from its genetic manipulability (Soukas et al., 2009, Yue et al., 2021). The extracellular flux assay can effectively analyze the metabolic characteristics during development, aging, and diseases in this organism. However, utilizing C. elegans in the extracellular flux assay entails several differences and considerations compared to cell models. First, due to the limited uptake of compounds caused by the impermeability of its cuticle to non–water-soluble substances, C. elegans is deemed unsuitable for drug-related assays (Burns et al., 2010, Zheng et al., 2013). Consequently, in-depth analyses similar to the standard Mitostress test performed with cells are not feasible. Instead, the assay typically involves measuring basal OCR, injecting FCCP to measure maximal respiration, and directly treating the worms with a complex IV inhibitor to distinguish between nonmitochondrial and mitochondrial respiration (Koopman et al., 2016). Second, while the machine was initially developed for measuring the metabolic states of cells at 37°C, C. elegans is usually cultured at 20 to 25°C. Therefore, during the assay using C. elegans, it is essential to turn off the heater to maintain the appropriate temperature. Third, unlike cell assays where cells are seeded approximately 1 day before running the assay, C. elegans is placed in the plate just before the assay, with up to 30 worms per well in a 96-well format to ensure accurate assessments. Additionally, each well's data point should be normalized per worm or per μg protein. Detailed protocols for conducting the extracellular flux assay using C. elegans have been comprehensively summarized in previous literature (Haroon et al., 2018, Koopman et al., 2016).
Tissue
Fresh tissue segments obtained from mouse models or patient samples can be utilized in extracellular flux assays without dissociating the tissues into single cells. This approach avoids the stress induced during the process of single-cell preparation and allows for the measurement of the tissue's intrinsic metabolic state. Additionally, tissue samples, such as biopsy specimens, can be used for more systemic analysis in extracellular flux assays (Marty-Lombardi et al., 2024, Onodera et al., 2023). The key aspect of utilizing tissue in extracellular flux assays is the preparation of segments of consistent size, typically achieved using punch tools to create uniform sizes for placement in each well. However, several potential issues may arise during this process. First, if the tissue size is relatively large, the basal OCR may decrease due to substrate or oxygen depletion, necessitating caution during experimentation. Second, when dealing with tissues undergoing pathological processes, interpretation might be complicated by the infiltration of substantial numbers of immune cells into the tissue. Third, in cases such as brain tissue, the section from which the punch is obtained may impact the results, emphasizing the importance of consistently obtaining tissue from the same region to avoid interpretation errors. Additionally, tissue damage during the punching process should be taken into account when interpreting the data. References on the optimization process of extracellular flux assays using tissue can be consulted for further guidance (Marty-Lombardi et al., 2024, Onodera et al., 2023, Underwood et al., 2020).
Isolated mitochondria
If the exclusive activity of mitochondria needs to be measured or the activity of an enzyme or complex proteins in the mitochondria needs to be tested, mitochondria can be isolated from live cells or fresh tissues for analysis (Cao et al., 2022). For example, to confirm whether a specific substance directly affects mitochondria, extracellular flux assay data from cells can be compared with data from isolated mitochondria (Saxena et al., 2019). Several prior studies provide methods for isolating mitochondria and utilizing the obtained mitochondria for extracellular flux assays (Benador et al., 2018, Varkuti et al., 2020, Xia et al., 2024). A crucial aspect of this process is maintaining the mitochondrial membrane potential, which is essential for ETC function and thus for measuring mitochondrial respiration. The success of the experiment depends on how effectively the mitochondria are collected without damage and ensuring that all samples have a similar quality. Therefore, it is essential to optimize the conditions for extracting pure and intact mitochondria before performing the extracellular flux assay. One important consideration is to avoid using NaOH to adjust the mitochondrial isolation buffer, as sodium ions can disrupt the mitochondrial membrane potential. Instead, KOH should be used for adjustments. Additionally, the results should be normalized by measuring the protein content of the purified mitochondria in parallel with the samples used for the experiment.
3D structure (organoid, spheroid)
The extracellular flux assay is also applicable to samples with 3D structures, such as spheroids and organoids (Civenni et al., 2019, Huang et al., 2015, Morrone Parfitt et al., 2024). While applying the assay to organoid studies can be challenging due to the variability of technical and experimental replicates, as well as the potential solidification of Matrigel during the seeding process, optimization has been done to minimize errors in sample preparation. Detailed protocols are available in the literature (Ludikhuize et al., 2021). It is important to note that organoids are heterogeneous structures composed of different cell types. Therefore, the results of the assay reflect the average metabolic features of the multiple cell types within the organoid (Ludikhuize et al., 2021). For example, when studying the metabolic consequences of gene knockdown, it is crucial to consider the impact of this knockdown on cell differentiation.
DISCUSSION
The extracellular flux analyzer has transformed the study of cellular metabolism by enabling real-time measurements of key metabolic parameters, such as glycolysis, mitochondrial respiration, and FAO. This review has highlighted the principles and applications of the extracellular flux assay across various biological models, demonstrating its versatility and significance in metabolic research.
One of the primary advantages of the extracellular flux assay is its ability to provide dynamic, real-time measurements of cellular metabolic activities. This capability is particularly valuable for studying metabolic shifts in response to different stimuli or treatments. For example, by measuring OCR and ECAR, researchers can gain insights into the balance between glycolysis and oxidative phosphorylation in various cell types, including cancer cells, immunocytes, and primary tissues. The assay's adaptability to different experimental models, from isolated mitochondria to whole tissues and 3D structures, such as organoids, underscores its utility in diverse research contexts. This flexibility facilitates detailed metabolic profiling that can inform our understanding of disease mechanisms and therapeutic responses.
Despite these advantages, several challenges must be addressed to ensure accurate and reproducible results. The preparation of samples, whether cells, tissues, or isolated mitochondria, requires meticulous optimization to maintain their integrity and functionality. Normalization steps, such as by cell number, protein content, or tissue weight, are critical to minimize variability and ensure data accuracy. Additionally, the extracellular flux assay should be completed promptly, as prolonged exposure to non-CO2 incubators, assay media, and cytotoxic drugs can lead to cell death and erroneous data.
Another limitation is the necessity to use only fresh cells. Freezing and thawing can damage mitochondrial membranes and uncouple the electron transport system, complicating OCR measurements (Yamaguchi et al., 2007). While it is possible to measure electron transport system activity by isolating mitochondria from frozen tissues, there is a need for methods to recover mitochondria intact for immediate analysis. In some cases, metabolic activities unrelated to mitochondrial respiration, such as the actions of nitric oxide synthase (Carlstrom, 2021) or the sulfide oxidation pathway (Burns et al., 2010, Lin et al., 2023), can affect OCRs. The mitochondrial stress test helps distinguish pure mitochondrial OCR from other oxygen-consuming processes. If OCR does not decrease after treatment with rotenone and antimycin A, it may indicate the presence of other intracellular oxygen-consuming pathways.
In conclusion, the extracellular flux analyzer is a powerful tool for investigating cellular metabolism, offering valuable insights that can drive advancements in biomedical research. By addressing the challenges and optimizing experimental conditions, researchers can fully harness the potential of this technology to unravel the complexities of metabolic regulation and its implications for health and disease.
Author Contributions
N. Lee and J. Lee conceived the project. N. Lee, J. Lee, and I. Yoo created figures. N. Lee, I. Yoo, J. Lee, and I. Ahn wrote the manuscript.
Declaration of Competing Interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
We thank Brennon Berard (University of Massachusetts Chan Medical School) for proofreading our manuscript. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (RS-2024-00338061).
ORCID
I. Yoo, 0009-0009-2705-343X; I. Ahn, 0009-0001-2191-5408; J. Lee, 0009-0000-3188-2147; N. Lee, 0000-0002-2980-2758.
References
- Benador I.Y., Veliova M., Mahdaviani K., Petcherski A., Wikstrom J.D., Assali E.A., Acin-Perez R., Shum M., Oliveira M.F., Cinti S., et al. Mitochondria bound to lipid droplets have unique bioenergetics, composition, and dynamics that support lipid droplet expansion. Cell Metab. 2018;27(869-885) doi: 10.1016/j.cmet.2018.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brunton H., Caligiuri G., Cunningham R., Upstill-Goddard R., Bailey U.M., Garner I.M., Nourse C., Dreyer S., Jones M., Moran-Jones K., et al. HNF4A and GATA6 loss reveals therapeutically actionable subtypes in pancreatic cancer. Cell Rep. 2020;31 doi: 10.1016/j.celrep.2020.107625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burns A.R., Wallace I.M., Wildenhain J., Tyers M., Giaever G., Bader G.D., Nislow C., Cutler S.R., Roy P.J. A predictive model for drug bioaccumulation and bioactivity in Caenorhabditis elegans. Nat. Chem. Biol. 2010;6:549–557. doi: 10.1038/nchembio.380. [DOI] [PubMed] [Google Scholar]
- Caines J.K., Barnes D.A., Berry M.D. The use of Seahorse XF assays to interrogate real-time energy metabolism in cancer cell lines. Methods Mol. Biol. 2022;2508:225–234. doi: 10.1007/978-1-0716-2376-3_17. [DOI] [PubMed] [Google Scholar]
- Cao Y., Vergnes L., Wang Y.C., Pan C., Chella Krishnan K., Moore T.M., Rosa-Garrido M., Kimball T.H., Zhou Z., Charugundla S., et al. Sex differences in heart mitochondria regulate diastolic dysfunction. Nat. Commun. 2022;13:3850. doi: 10.1038/s41467-022-31544-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carlstrom M. Nitric oxide signalling in kidney regulation and cardiometabolic health. Nat. Rev. Nephrol. 2021;17:575–590. doi: 10.1038/s41581-021-00429-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chandel N.S. Glycolysis. Cold Spring Harb. Perspect. Biol. 2021;13 doi: 10.1101/cshperspect.a040535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Civenni G., Bosotti R., Timpanaro A., Vazquez R., Merulla J., Pandit S., Rossi S., Albino D., Allegrini S., Mitra A., et al. Epigenetic control of mitochondrial fission enables self-renewal of stem-like tumor cells in human prostate cancer. Cell Metab. 2019;30(303-318) doi: 10.1016/j.cmet.2019.05.004. [DOI] [PubMed] [Google Scholar]
- Erdem A., Marin S., Pereira-Martins D.A., Geugien M., Cunningham A., Pruis M.G., Weinhauser I., Gerding A., Bakker B.M., Wierenga A.T.J., et al. Inhibition of the succinyl dehydrogenase complex in acute myeloid leukemia leads to a lactate-fuelled respiratory metabolic vulnerability. Nat. Commun. 2022;13:2013. doi: 10.1038/s41467-022-29639-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Espinosa J.A., Pohan G., Arkin M.R., Markossian S. Real-time assessment of mitochondrial toxicity in HepG2 cells using the seahorse extracellular flux analyzer. Curr. Protoc. 2021;1 doi: 10.1002/cpz1.75. [DOI] [PubMed] [Google Scholar]
- Glickman I., Turesky S., Hill R. Determination of oxygen consumption in normal and inflamed human gingiva using the Warburg manometric technic. J. Dent. Res. 1949;28:83–94. doi: 10.1177/00220345490280011301. [DOI] [PubMed] [Google Scholar]
- Green C.D., Maceyka M., Cowart L.A., Spiegel S. Sphingolipids in metabolic disease: the good, the bad, and the unknown. Cell Metab. 2021;33:1293–1306. doi: 10.1016/j.cmet.2021.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greene J., Segaran A., Lord S. Targeting OXPHOS and the electron transport chain in cancer; molecular and therapeutic implications. Semin. Cancer Biol. 2022;86:851–859. doi: 10.1016/j.semcancer.2022.02.002. [DOI] [PubMed] [Google Scholar]
- Gu X., Ma Y., Liu Y., Wan Q. Measurement of mitochondrial respiration in adherent cells by Seahorse XF96 Cell Mito Stress Test. STAR Protoc. 2021;2 doi: 10.1016/j.xpro.2020.100245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haroon S., Li A., Weinert J.L., Fritsch C., Ericson N.G., Alexander-Floyd J., Braeckman B.P., Haynes C.M., Bielas J.H., Gidalevitz T., et al. Multiple molecular mechanisms rescue mtDNA disease in C. elegans. Cell Rep. 2018;22:3115–3125. doi: 10.1016/j.celrep.2018.02.099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Huang L., Holtzinger A., Jagan I., BeGora M., Lohse I., Ngai N., Nostro C., Wang R., Muthuswamy L.B., Crawford H.C., et al. Ductal pancreatic cancer modeling and drug screening using human pluripotent stem cell- and patient-derived tumor organoids. Nat. Med. 2015;21:1364–1371. doi: 10.1038/nm.3973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Istomine R., Al-Aubodah T.A., Alvarez F., Smith J.A., Wagner C., Piccirillo C.A. The eIF4EBP-eIF4E axis regulates CD4(+) T cell differentiation through modulation of T cell activation and metabolism. iScience. 2023;26 doi: 10.1016/j.isci.2023.106683. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jang C., Chen L., Rabinowitz J.D. Metabolomics and isotope tracing. Cell. 2018;173:822–837. doi: 10.1016/j.cell.2018.03.055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Katsyuba E., Romani M., Hofer D., Auwerx J. NAD(+) homeostasis in health and disease. Nat. Metab. 2020;2:9–31. doi: 10.1038/s42255-019-0161-5. [DOI] [PubMed] [Google Scholar]
- Kenyon C.J. The genetics of ageing. Nature. 2010;464:504–512. doi: 10.1038/nature08980. [DOI] [PubMed] [Google Scholar]
- Koopman M., Michels H., Dancy B.M., Kamble R., Mouchiroud L., Auwerx J., Nollen E.A., Houtkooper R.H. A screening-based platform for the assessment of cellular respiration in Caenorhabditis elegans. Nat. Protoc. 2016;11:1798–1816. doi: 10.1038/nprot.2016.106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kostic M., Katoshevski T., Sekler I. Allosteric regulation of NCLX by mitochondrial membrane potential links the metabolic state and Ca(2+) signaling in mitochondria. Cell Rep. 2018;25(3465-3475) doi: 10.1016/j.celrep.2018.11.084. [DOI] [PubMed] [Google Scholar]
- Lee N., Kim D. Cancer metabolism: fueling more than just growth. Mol. Cells. 2016;39:847–854. doi: 10.14348/molcells.2016.0310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li X., Sun X., Carmeliet P. Hallmarks of endothelial cell metabolism in health and disease. Cell Metab. 2019;30:414–433. doi: 10.1016/j.cmet.2019.08.011. [DOI] [PubMed] [Google Scholar]
- Li Z., Graham B.H. Measurement of mitochondrial oxygen consumption using a Clark electrode. Methods Mol. Biol. 2012;837:63–72. doi: 10.1007/978-1-61779-504-6_5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin H., Yu Y., Zhu L., Lai N., Zhang L., Guo Y., Lin X., Yang D., Ren N., Zhu Z., et al. Implications of hydrogen sulfide in colorectal cancer: mechanistic insights and diagnostic and therapeutic strategies. Redox Biol. 2023;59 doi: 10.1016/j.redox.2023.102601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Little A.C., Kovalenko I., Goo L.E., Hong H.S., Kerk S.A., Yates J.A., Purohit V., Lombard D.B., Merajver S.D., Lyssiotis C.A. High-content fluorescence imaging with the metabolic flux assay reveals insights into mitochondrial properties and functions. Commun. Biol. 2020;3:271. doi: 10.1038/s42003-020-0988-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ludikhuize M.C., Meerlo M., Burgering B.M.T., Rodriguez Colman M.J. Protocol to profile the bioenergetics of organoids using Seahorse. STAR Protoc. 2021;2 doi: 10.1016/j.xpro.2021.100386. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marty-Lombardi S., Lu S., Ambroziak W., Schrenk-Siemens K., Wang J., DePaoli-Roach A.A., Hagenston A.M., Wende H., Tappe-Theodor A., Simonetti M., et al. Neuron-astrocyte metabolic coupling facilitates spinal plasticity and maintenance of inflammatory pain. Nat. Metab. 2024;6:494–513. doi: 10.1038/s42255-024-01001-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mendoza A., Takemoto Y., Cruzado K.T., Masoud S.S., Nagata A., Tantipanjaporn A., Okuda S., Kawagoe F., Sakamoto R., Odagi M., et al. Controlled lipid beta-oxidation and carnitine biosynthesis by a vitamin D metabolite. Cell Chem. Biol. 2022;29(660-669) doi: 10.1016/j.chembiol.2021.08.008. [DOI] [PubMed] [Google Scholar]
- Miller A., Nagy C., Knapp B., Laengle J., Ponweiser E., Groeger M., Starkl P., Bergmann M., Wagner O., Haschemi A. Exploring metabolic configurations of single cells within complex tissue microenvironments. Cell Metab. 2017;26(788-800) doi: 10.1016/j.cmet.2017.08.014. [DOI] [PubMed] [Google Scholar]
- Morrone Parfitt G., Coccia E., Goldman C., Whitney K., Reyes R., Sarrafha L., Nam K.H., Sohail S., Jones D.R., Crary J.F., et al. Disruption of lysosomal proteolysis in astrocytes facilitates midbrain organoid proteostasis failure in an early-onset Parkinson's disease model. Nat. Commun. 2024;15:447. doi: 10.1038/s41467-024-44732-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nolfi-Donegan D., Braganza A., Shiva S. Mitochondrial electron transport chain: oxidative phosphorylation, oxidant production, and methods of measurement. Redox Biol. 2020;37 doi: 10.1016/j.redox.2020.101674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nomura M., Liu J., Rovira I.I., Gonzalez-Hurtado E., Lee J., Wolfgang M.J., Finkel T. Fatty acid oxidation in macrophage polarization. Nat. Immunol. 2016;17:216–217. doi: 10.1038/ni.3366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O'Connor R.S., Guo L., Ghassemi S., Snyder N.W., Worth A.J., Weng L., Kam Y., Philipson B., Trefely S., Nunez-Cruz S., et al. The CPT1a inhibitor, etomoxir induces severe oxidative stress at commonly used concentrations. Sci. Rep. 2018;8:6289. doi: 10.1038/s41598-018-24676-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Onodera T., Wang M.Y., Rutkowski J.M., Deja S., Chen S., Balzer M.S., Kim D.S., Sun X., An Y.A., Field B.C., et al. Endogenous renal adiponectin drives gluconeogenesis through enhancing pyruvate and fatty acid utilization. Nat. Commun. 2023;14:6531. doi: 10.1038/s41467-023-42188-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pan R.Y., He L., Zhang J., Liu X., Liao Y., Gao J., Liao Y., Yan Y., Li Q., Zhou X., et al. Positive feedback regulation of microglial glucose metabolism by histone H4 lysine 12 lactylation in Alzheimer's disease. Cell Metab. 2022;34(634-648) doi: 10.1016/j.cmet.2022.02.013. [DOI] [PubMed] [Google Scholar]
- Pavlova N.N., Zhu J., Thompson C.B. The hallmarks of cancer metabolism: still emerging. Cell Metab. 2022;34:355–377. doi: 10.1016/j.cmet.2022.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pereira M., Chen T.D., Buang N., Olona A., Ko J.H., Prendecki M., Costa A.S.H., Nikitopoulou E., Tronci L., Pusey C.D., et al. Acute iron deprivation reprograms human macrophage metabolism and reduces inflammation in vivo. Cell Rep. 2019;28(498-511) doi: 10.1016/j.celrep.2019.06.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riddle, D.L., Blumenthal, T., Meyer B.J., and Priess, J.R., eds. (1997). C. elegans II (Cold Spring Harbor, NY: Cold Spring Harbor Laboratory Press). [PubMed]
- Rinaldo P., Matern D., Bennett M.J. Fatty acid oxidation disorders. Annu. Rev. Physiol. 2002;64:477–502. doi: 10.1146/annurev.physiol.64.082201.154705. [DOI] [PubMed] [Google Scholar]
- Saxena S., Vekaria H., Sullivan P.G., Seifert A.W. Connective tissue fibroblasts from highly regenerative mammals are refractory to ROS-induced cellular senescence. Nat. Commun. 2019;10:4400. doi: 10.1038/s41467-019-12398-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scholler-Mann A., Matt K., Hochecker B., Bergemann J. Ex vivo assessment of mitochondrial function in human peripheral blood mononuclear cells using XF analyzer. Bio Protoc. 2021;11 doi: 10.21769/BioProtoc.3980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Silic-Benussi M., Sharova E., Ciccarese F., Cavallari I., Raimondi V., Urso L., Corradin A., Kotler H., Scattolin G., Buldini B., et al. mTOR inhibition downregulates glucose-6-phosphate dehydrogenase and induces ROS-dependent death in T-cell acute lymphoblastic leukemia cells. Redox Biol. 2022;51 doi: 10.1016/j.redox.2022.102268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Soukas A.A., Kane E.A., Carr C.E., Melo J.A., Ruvkun G. Rictor/TORC2 regulates fat metabolism, feeding, growth, and life span in Caenorhabditis elegans. Genes Dev. 2009;23:496–511. doi: 10.1101/gad.1775409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spector A.A. Fatty acid binding to plasma albumin. J. Lipid Res. 1975;16:165–179. [PubMed] [Google Scholar]
- Underwood E., Redell J.B., Zhao J., Moore A.N., Dash P.K. A method for assessing tissue respiration in anatomically defined brain regions. Sci. Rep. 2020;10:13179. doi: 10.1038/s41598-020-69867-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Valle-Casuso J.C., Angin M., Volant S., Passaes C., Monceaux V., Mikhailova A., Bourdic K., Avettand-Fenoel V., Boufassa F., Sitbon M., et al. Cellular metabolism is a major determinant of HIV-1 reservoir seeding in CD4(+) T cells and offers an opportunity to tackle infection. Cell Metab. 2019;29(611-626) doi: 10.1016/j.cmet.2018.11.015. [DOI] [PubMed] [Google Scholar]
- van der Vusse G.J. Albumin as fatty acid transporter. Drug Metab. Pharmacokinet. 2009;24:300–307. doi: 10.2133/dmpk.24.300. [DOI] [PubMed] [Google Scholar]
- Vander Heiden M.G., Cantley L.C., Thompson C.B. Understanding the Warburg effect: the metabolic requirements of cell proliferation. Science. 2009;324:1029–1033. doi: 10.1126/science.1160809. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Varkuti B.H., Kepiro M., Liu Z., Vick K., Avchalumov Y., Pacifico R., MacMullen C.M., Kamenecka T.M., Puthanveettil S.V., Davis R.L. Neuron-based high-content assay and screen for CNS active mitotherapeutics. Sci. Adv. 2020;6:eaaw8702. doi: 10.1126/sciadv.aaw8702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang R., Novick S.J., Mangum J.B., Queen K., Ferrick D.A., Rogers G.W., Stimmel J.B. The acute extracellular flux (XF) assay to assess compound effects on mitochondrial function. J. Biomol. Screen. 2015;20:422–429. doi: 10.1177/1087057114557621. [DOI] [PubMed] [Google Scholar]
- Xia W., Veeragandham P., Cao Y., Xu Y., Rhyne T.E., Qian J., Hung C.W., Zhao P., Jones Y., Gao H., et al. Obesity causes mitochondrial fragmentation and dysfunction in white adipocytes due to RalA activation. Nat. Metab. 2024;6:273–289. doi: 10.1038/s42255-024-00978-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yamaguchi R., Andreyev A., Murphy A.N., Perkins G.A., Ellisman M.H., Newmeyer D.D. Mitochondria frozen with trehalose retain a number of biological functions and preserve outer membrane integrity. Cell Death Differ. 2007;14:616–624. doi: 10.1038/sj.cdd.4402035. [DOI] [PubMed] [Google Scholar]
- Yue Y., Li S., Shen P., Park Y. Caenorhabditis elegans as a model for obesity research. Curr. Res. Food Sci. 2021;4:692–697. doi: 10.1016/j.crfs.2021.09.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhang J., Zhang Q. Using Seahorse machine to measure OCR and ECAR in cancer cells. Methods Mol. Biol. 2019;1928:353–363. doi: 10.1007/978-1-4939-9027-6_18. [DOI] [PubMed] [Google Scholar]
- Zheng J. Energy metabolism of cancer: glycolysis versus oxidative phosphorylation (Review) Oncol. Lett. 2012;4:1151–1157. doi: 10.3892/ol.2012.928. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zheng S.Q., Ding A.J., Li G.P., Wu G.S., Luo H.R. Drug absorption efficiency in Caenorhbditis elegans delivered by different methods. PLoS One. 2013;8 doi: 10.1371/journal.pone.0056877. [DOI] [PMC free article] [PubMed] [Google Scholar]