Skip to content

Repository files navigation

makemore

Building makemore (a.k.a., generating more texts) using different language models from scratch.

Mostly inspired by the lectures from Andrej Karpathy: https://www.youtube.com/@AndrejKarpathy.

There are 4 models we consider:

Here are sample results.

Make more names using Bigram

ya
syahavilin
dleekahmangwnya
tryahdachen
ena
da
amiiae
a
keles
ly

Make more names using Trigram

se
emahle
em
jlbim
bannya
oryah
niewena
va
fajiah
fakeles

Make more names using MLP

yeesha
leilanni
camryn
maryda
trysten
hender
ellysoniah
abdijas
louanne
alivia

Make more Shakespeare work using Transformer

I which I long them-fares here surked,
Whil with pursection my faid do.
Shephered' and my stande couse.
All, I you dreece! the nature! If 'He even grong mise's suck which you more
You evens as thou somet. I do hear,
That at untery that longs doin the,
For my uncred-hen arming to the conked.
vire some, becurie 'tis queel!

QUEEN VI ELIVA:
Yill greaste at same insomest my my diest,
That's but comelful; she appayon'd take to Musire herse;
Too did and was for undom to by do eath that Edvil
He lornk to end not one breada master: hold,
inle your shighter thee of right
Hast onchemes mytredom of ac his dardient him.

AUTOLUMBERLA:
It lord?
But SBoht that you unsues I dare,
Hut stren tongentord of him distrain.
Thou do to his it; I to I know out to his lose grast,
Whom shame do lady the comegare. Where Cath wive, your prince the is incice
Look him, if with I will love!

...

UKE OF YORK:
Wife our will looke.

AUTOLYCUS:
Is them sidoim arming na

About

Building a few fundamental language models (and transformer) from scratch

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages