Transformers

My attempt at trying to implement as many transformer architectures as I can until I reach a level of profound understanding of everything.

Model Implementations

Following is just the list of models that came to my mind. I have no particular order in which I'm going to implement them. I might end up doing something completely different as well. Just sailing with the flow.

Notations

✅: Completed
❌: Not started
⌛️: In progress

Model	Organization	Status
GPT-2	OpenAI	✅
LLaMA2 / LLaMA3	Meta	✅
ViT	Google	❌
BERT	Google	❌
T5	Google	❌
Mistral-7B-v0.1	Mistral AI	⏳
Mistral-7B-v0.2	Mistral AI	❌
DeepSeek-V2	DeepSeek	❌
DeepSeek-v3	DeepSeek	❌
Gemma-2	Google	❌
Gemma-3	Google	❌
Command-A	Cohere	❌