Welcome to my blog! I am Siddhesh Dosi, an M.Tech student at IIT Gandhinagar, pursuing Computer Science and Engineering under the mentorship of Dr. Mayank Singh.
Here, I’ll share my experiences, research projects, and insights as we delve into the exciting world of technology. IIT Gandhinagar, known for its excellence in education and research, provides the ideal environment for me to explore the limitless possibilities of computer science.
Current Work
My current research work is on optimizing the smaller variant of LLMs to get performance as equal as of its Larger variant.
| Size | Parameters |
|---|---|
| mini | 125 M |
| base | 1.3 B |
| standard | 6.7 B |
| large | 30 B |
| huge | 120 B |
Galactica is trained on a large set of scientic domains - Text - LaTex - Code - SMILES - AA Sequence - DNA Sequence
Continue pretraining of smaller size models mini or base on specific domian of scientific papers to make it as efficient as huge variant is on the domain.