...
- Initialize with a pre-trained model.
- Prepare a labeled dataset for the question-answering task
- Train the model using this dataset, adjusting all layers.
- Try different learning rates to avoid catastrophic forgetting and also to avoid over-fitting.
Types of Fine-tuning:
- Standard Fine-tuning as I mentioned its steps previously.
- Feature-Based Fine-Tuning: This involves using a pre-trained model as a feature extractor and then using a smaller model as a classifier. The advantage of this approach is it's computationally efficient and less prone to overfitting but also it might be weak in classification.
- There are other types of fine-tuning but they may not be suitable for your project
Learning Progress
- Natural Language Processing (NLP) course.
- Studying more about the Langchain framework
- Study React.js and Nest.js
- Learn more about Blockchain and
- Learn more about RAG and fine-tuning