| # PDeepPP: A Comprehensive Protein Language Model Hub | |
| PDeepPP is a hybrid protein language model designed to predict post-translational modification (PTM) sites, analyze biologically relevant features, and support a wide range of protein sequence analysis tasks. This repository serves as the central hub for accessing and exploring various specialized PDeepPP models, each fine-tuned for specific tasks, such as PTM site prediction, bioactivity analysis, and more. | |
| The models in this repository can all be used on their corresponding datasets on GitHub ([https://github.com/fondress/PDeepPP]) | |
| --- | |
| ## Overview | |
| PDeepPP integrates state-of-the-art transformer-based self-attention mechanisms with convolutional neural networks (CNNs) to capture both global and local features in protein sequences. By leveraging pretrained embeddings from `ESM` and incorporating modular architecture components, PDeepPP offers a robust framework for protein sequence analysis. | |
| This repository contains links to multiple task-specific PDeepPP models. These models are pre-trained or fine-tuned on publicly available datasets and are hosted on Hugging Face for easy access. | |
| --- | |
| ## Key Features | |
| - **Flexible Architecture**: Combines self-attention and convolutional operations for robust feature extraction. | |
| - **Task-Specific Models**: Includes pre-trained models for PTM prediction, bioactivity classification, and more. | |
| - **Dataset Support**: Models are validated on datasets such as PTM and BPS, ensuring performance on real-world tasks. | |
| - **Extensibility**: Users can fine-tune the models on custom datasets for new tasks. | |
| --- | |
| ## Available Models | |
| ### General Models | |
| - [PDeepPP Main](https://huggingface.co/fondress/PDeepPP) | |
| ### Task-Specific Models | |
| #### Post-Translational Modifications (PTMs) | |
| - [PDeepPP Phosphorylation (Serine)](https://huggingface.co/fondress/PDeepPP_Phosphoserine) | |
| - [PDeepPP Phosphorylation (Tyrosine)](https://huggingface.co/fondress/PDeepPP_Phosphorylation-Y) | |
| - [PDeepPP Glycosylation (N-linked)](https://huggingface.co/fondress/PDeepPP_N-linked-glycosylation-N) | |
| - [PDeepPP Glycosylation (O-linked)](https://huggingface.co/fondress/PDeepPP_O-linked-glycosylation) | |
| - [PDeepPP Methylation (Lysine)](https://huggingface.co/fondress/PDeepPP_Methylation-K) | |
| - [PDeepPP Methylation (Arginine)](https://huggingface.co/fondress/PDeepPP_Methylation-R) | |
| - [PDeepPP SUMOylation](https://huggingface.co/fondress/PDeepPP_SUMOylation) | |
| - [PDeepPP Ubiquitin](https://huggingface.co/fondress/PDeepPP_Ubiquitin) | |
| - [PDeepPP S-Palmitoylation](https://huggingface.co/fondress/PDeepPP_S-Palmitoylation) | |
| - [PDeepPP N6-acetyllysine(K)](https://huggingface.co/fondress/PDeepPP_N6-acetyllysine-K) | |
| - [PDeepPP Hydroxyproline (P)](https://huggingface.co/fondress/PDeepPP_Hydroxyproline-P) | |
| - [PDeepPP Hydroxyproline (K)](https://huggingface.co/fondress/PDeepPP_Hydroxyproline-K) | |
| - [PDeepPP Pyrrolidone-carboxylic-acid (Q)](https://huggingface.co/fondress/PDeepPP_Pyrrolidone-carboxylic-acid-Q) | |
| #### Bioactivity Prediction | |
| - [PDeepPP ACE](https://huggingface.co/fondress/PDeepPP_ACE) | |
| - [PDeepPP BBP](https://huggingface.co/fondress/PDeepPP_BBP) | |
| - [PDeepPP DPPIV](https://huggingface.co/fondress/PDeepPP_DPPIV) | |
| - [PDeepPP Toxicity](https://huggingface.co/fondress/PDeepPP_Toxicity) | |
| - [PDeepPP Antimalarial (Main)](https://huggingface.co/fondress/PDeepPP_Antimalarial-main) | |
| - [PDeepPP Antimalarial (Alternative)](https://huggingface.co/fondress/PDeepPP_Antimalarial-alternative) | |
| - [PDeepPP Anticancer (Main)](https://huggingface.co/fondress/PDeepPP_Anticancer-main) | |
| - [PDeepPP Anticancer (Alternative)](https://huggingface.co/fondress/PDeepPP_Anticancer-alternative) | |
| - [PDeepPP Antiviral](https://huggingface.co/fondress/PDeepPP_Antiviral) | |
| - [PDeepPP Antioxidant](https://huggingface.co/fondress/PDeepPP_Antioxidant) | |
| - [PDeepPP Antibacterial](https://huggingface.co/fondress/PDeepPP_Antibacterial) | |
| - [PDeepPP Antifungal](https://huggingface.co/fondress/PDeepPP_Antifungal) | |
| - [PDeepPP Antimicrobial](https://huggingface.co/fondress/PDeepPP_Antimicrobial) | |
| - [PDeepPP Anti-MRSA](https://huggingface.co/fondress/PDeepPP_Anti-MRSA) | |
| - [PDeepPP Antiparasitic](https://huggingface.co/fondress/PDeepPP_Antiparasitic) | |
| - [PDeepPP Bitter](https://huggingface.co/fondress/PDeepPP_bitter) | |
| - [PDeepPP Umami](https://huggingface.co/fondress/PDeepPP_umami) | |
| - [PDeepPP Neuro](https://huggingface.co/fondress/PDeepPP_neuro) | |
| - [PDeepPP Quorum](https://huggingface.co/fondress/PDeepPP_Quorum) | |
| - [PDeepPP TTCA](https://huggingface.co/fondress/PDeepPP_TTCA) | |
| --- | |
| ## Model Architecture | |
| PDeepPP is built on a hybrid architecture that includes: | |
| - **Self-Attention Global Features**: Captures long-range dependencies in protein sequences. | |
| - **TransConv1d Module**: Combines transformer layers with convolutional layers for local feature extraction. | |
| - **PosCNN Module**: Incorporates position-aware convolutional operations to enhance sequence representation. | |
| --- | |
| ## How to Use | |
| To use any of the models, you need to install the required dependencies, such as `torch` and `transformers`: | |
| ```bash | |
| pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 | |
| pip install transformers | |
| ``` | |
| Here’s a quick example of how to load and use a model: | |
| ```python | |
| from transformers import AutoModel, AutoTokenizer | |
| # Load the model | |
| model_name = "fondress/PDeepPP_ACE" | |
| model = AutoModel.from_pretrained(model_name, trust_remote_code=True) | |
| # Example input | |
| protein_sequence = "VELYP" | |
| # Preprocess the sequence (refer to specific model documentation for preprocessing steps) | |
| # Forward pass | |
| outputs = model(input_ids=processed_input) | |
| logits = outputs.logits | |
| ``` | |
| ## Training and Customization | |
| You can fine-tune PDeepPP for custom tasks using your own datasets. The model supports: | |
| - **Custom PTM types**: Extend the model to predict additional post-translational modifications. | |
| - **Sequence classification tasks**: Adapt the model to classify protein sequences based on custom labels. | |
| - **Feature extraction for downstream analyses**: Use PDeepPP to generate embeddings for tasks like clustering or similarity calculation. | |
| Refer to the `PDeepPPConfig` class in the source repository for details on available hyperparameters and customization options. | |
| --- | |
| ## Citation | |
| If you use any of the PDeepPP models in your research, please cite the associated paper or repository: | |
| ``` | |
| @article{your_reference, | |
| title={`PDeepPP`: A Hybrid Model for Protein Sequence Analysis}, | |
| author={Author Name}, | |
| journal={Journal Name}, | |
| year={2025} | |
| } | |
| ``` |