- Question - How do I make a neural network following Karpathy’s micrograd tutorial?
- Context - Wanted to learn more about neural nets. Before, I’d read about neural nets and fine-tuned them but hadn’t implemented from scratch. Karpathy’s tutorial looked cool.
- Primarily for personal learning or novel discovery? - Personal Learning
- What is the delta? - Before this experiment, I had only theoretical knowledge of neural networks and transformers without practical implementation experience. This experiment provided hands-on experience in implementing a neural network from scratch, enhancing my understanding of neural nets beyond theoretical knowledge.
- Observations -
- Stronger understanding of backpropagation, gradients, and neural networks. Understood the simplicity and complexity of neural networks, including the risk of overfitting with large training steps.
- I loved the use of scalars rather than vectors for each neuron’s weight and bias. Great choice for teaching neural nets.
- Made spaced repetition cards on learnings and these were helpful
Future changes
- I’d have timed how long this took me.
Tags
- personal mnemonic: neural network where every neuron is a cabbage. cabbage man
- type: #recreate-repo
- theme: #learn
- status: #done