Advances in Single-Cell Biology Through Combined Data

Foundation Models are powerful tools that have gained a lot of attention lately in various fields, including biology. These models are very effective because they can learn important information from huge amounts of data. Inspired by advancements in language processing and computer vision, foundation models have also started to play a big role in biological research, especially in areas like single-cell biology. This area has become a focus because there are now plenty of accessible datasets available from single-cell RNA sequencing, which records the activity of genes in individual cells.

#The Rise of Single-Cell Biology

Single-cell biology examines the behaviors and characteristics of individual cells. This is crucial because it allows researchers to see how cells differ from one another, even when they belong to the same type. An important aspect of this research is single-cell RNA sequencing, which measures the expression of genes at the single-cell level. With larger datasets becoming available, foundation models can be applied to understand the complexities of biological data in single cells.

#Importance of Gene Representation

One of the main tasks in single-cell biology is to create representations of genes. Foundation models can learn how genes behave by looking at data from experiments, typically using gene expression counts to understand gene activity. However, there are other ways to represent genes, which can provide additional context. For example, using language as a representation is one approach that has emerged. Models like genePT aim to create representations of genes using information from Scientific Literature. This is vital as much of our knowledge about biological processes comes from research articles.

#The Role of Scientific Literature

Scientific literature contains a wealth of information about genes and their functions. Much of what we know has been shared through published studies, effectively locking away valuable insights in these texts. By incorporating this information, models can gain a better understanding of genes and their behaviors. This means that the knowledge contained in literature can enhance the representations that are learned from experimental data.

#Combining Experimental and Language-Based Approaches

In this study, we want to look at the effects of combining two different representations of genes when studying single-cell data. The first representation comes from the data collected during experiments, while the second type uses knowledge gathered from language sources like scientific literature. In particular, we are interested in how these two types of information can help predict the effects of genetic changes on how genes are expressed in cells.

#Types of Genetic Perturbations

Genetic perturbations refer to changes made to specific genes to see how they influence gene expression. There are different types of genetic perturbations, such as altering one gene at a time or tweaking multiple genes simultaneously. The goal is to understand how these changes affect the overall behavior of the cell.

In our research, we focus on two main categories of perturbations: one-gene and two-gene perturbations. A one-gene perturbation involves changing a specific gene, while a two-gene perturbation looks at the effects of changing two genes at once.

#Research Questions

To guide our examination, we have formed several research questions:

1. Can we create models that effectively learn structured biological information for specific tasks without embedding this information directly into the model?
2. Will using a combination of language and experimental data help us achieve better results?
3. How significant is the curation of the knowledge we integrate into the model?

#Methodology

To answer these questions, we started with a widely used foundation model called scGPT, which is designed to handle scRNA-seq data. We modified scGPT to incorporate language-based information at the gene level. Each gene now receives a language representation derived from different scientific sources. We began with summaries from the NCBI gene database and combined them with protein summaries from UniProt.

#Importance of Gene Representations

The goal of our approach is to combine both experimental data and language-derived knowledge to create a more powerful model. By introducing additional information from literature, we hope to improve the model's ability to predict changes in gene expression after perturbations.

#Experimenting with Sources of Information

In our tests, we explored various sources of gene-related information, including annotations from the Gene Ontology (GO) database, which provides insights into gene functions, processes, and locations within cells. We used embeddings generated by large language models (LLMs) to aggregate this knowledge effectively.

#Findings from Our Analysis

Our analyses reveal several key insights:

1. Additive Value of Textual Representations: Language-based representations can provide additional and complementary information alongside the biological representations learned from experimental data.

2. Different Types of Information: Various sources of scientific knowledge offer different advantages. For instance, information about where genes are located in cells (cellular components) helps more in single-gene perturbations, while protein summaries are more beneficial for two-gene perturbations.

3. Careful Curation Matters: By selectively choosing the language-based information we include, we can enhance the performance of our models, sometimes exceeding the results of models that rely on hardcoded biological knowledge.

#Model Architecture

In our modified model, called scGenePT, we combined gene expression data with additional representations obtained from language sources. For each gene, we calculated an overall representation that includes both its biological data and its textual representation. This allows the model to learn from multiple types of information simultaneously.

#Performance Evaluation

To evaluate the effectiveness of our model, we measured its ability to predict the effect of genetic perturbations. We used datasets containing examples of both single and two-gene perturbations. By comparing our approach against traditional models, we aimed to see if our combined method could improve predictions significantly.

#Results of Our Evaluation

When assessing performance, we found that:

1. Improved Predictions: The addition of language-based representations clearly improved the model's ability to predict changes in gene expression from perturbations.

2. Higher Impact in Complex Cases: The greatest improvements were noted in two-gene perturbations, which are inherently more challenging due to potential interactions between genes. Language-based knowledge provided a richer context for making these predictions.

3. Different Knowledge Sources Provide Unique Benefits: Our findings also suggest that certain types of knowledge from literature are particularly useful for different kinds of perturbations. For example, cellular component information was especially valuable for single-gene perturbations.

#Conclusion

The combination of data gathered from experiments and insights from scientific literature provides a powerful way to model gene behavior in single-cell biology. Our work highlights the importance of incorporating language-based knowledge in understanding genetic perturbations better. By leveraging this approach, we can enhance the predictive capabilities of models beyond relying solely on experimental data.

#Future Directions

Looking ahead, there are many exciting possibilities for further exploration. We can probe more into how different types of language-based information influence model performance and create strategies to curate knowledge more effectively. Additionally, testing our combined models on diverse datasets and more complex biological questions could yield valuable insights. The potential to improve our understanding of gene function through this multi-modal approach opens new avenues in biological research.

In summary, the integration of language and experimental data not only enhances model performance but also helps us uncover deeper biological insights, leading to significant advancements in the field of single-cell biology.