07 Jul '25

DIYChatGPT Short Course for ANU Undergraduates

If you’re an ANU undergraduate student you can enrol in an upcoming (next week!) upcoming 3hr course I’ve created called DIY ChatGPT: LLMs as Information Processing Machines. It’s running for the first time next Thursday 17 July 10am–1pm (although it will run again later in the semester).

Here’s the blurb:

In this hands-on workshop you will train and use your own language model from scratch - with just pen and paper and a bit of dice rolling. Through interactive exercises and guided discussions, you’ll see how language models (even large language models like ChatGPT) are fundamentally information processing machines, turning language inputs into language outputs. The workshop builds from basic principles to more complex applications through an exploration of language modeling as a probabilistic process of predicting “what token comes next” in a sequence, and ends with a poetry slam (for real).

This course is a “Know the Nature Of Things” KNoT offered as part of the ANU’s McCusker Institute. They’re for-credit courses run by different folks all across campus, and a great way to explore some of the cool stuff going on at ANU.