skills · How large language models work
How LLMs Actually Work
See what a language model computes, by building one.
कोर्स के अंत तक, आप ये कर पाएँगे:
- A counting model that writes, and its loss on unseen text
- A byte-pair tokenizer trained from scratch
- Real token counts across languages
- Word vectors and nearest neighbors
- Masked multi-head attention in NumPy
- GPT-2's parameter count from its settings
- A training run that lowers loss
- A one-page note on how the model you use works
$9/माह से7.5 घंटे5 अध्याय20 पाठ
कोर्स अभी अंग्रेज़ी में हैं।
“Every reply is one guess at a time. Here is the guessing machine.”
इस कोर्स के बारे में
यह किसके लिए है
- You use chat assistants and want to know what they compute, not only how to prompt them
- You can read a short Python script and would rather run the idea than read about it
- You want to check claims about models (size, context, why they get things wrong) for yourself
आपको क्या चाहिए
- You can run a short Python script
- Python 3 on a laptop (NumPy and tiktoken are installed along the way)
आप कौन से कौशल अभ्यास करेंगे
- Language models
- Tokenization
- Byte-pair encoding
- Embeddings
- Attention
- Transformers
- Python
- NumPy
कितना समय लगता है
कुल मिलाकर लगभग 7.5 घंटे
- अध्याय 1 · Guessing the next token1.5 घंटे
- अध्याय 2 · Tokens1.5 घंटे
- अध्याय 3 · From ids to vectors1.5 घंटे
- अध्याय 4 · Attention1.5 घंटे
- अध्याय 5 · The whole machine1.5 घंटे
इसमें क्या शामिल है
- 20 पाठ
- 81 पठन
- 20 हल किए गए उदाहरण
- 40 अभ्यास
- 40 त्वरित जाँचें
- 20 आम भ्रम
- 20 क्विज़
- अध्याय के अंत की 5 जाँचें
पाठ्यक्रम
5 अध्याय, 20 पाठ
पूरा कोर्स: हर अध्याय और पाठ। विस्तार के लिए अध्याय पर टैप करें। शुरू करने पर पाठ खुलेंगे।
करियर
यह कोर्स आपको कहाँ तक ले जा सकता है
वे भूमिकाएँ जिनसे यह कोर्स जुड़ता है, और वे कौशल जिन्हें कोर्स पूरा करने के बाद आप अपने रिज़्यूमे और LinkedIn पर लिख सकते हैं।
वे कौशल जो आप बनाएँगे
कोर्स पूरा करने के बाद इन्हें अपने रिज़्यूमे या LinkedIn प्रोफ़ाइल में चिपकाएँ।
- Building and reading a bigram language modelExplain how a language model generates text one token at a time
- Decoding: greedy, sampling and temperatureExplain sampling temperature and its effect on model output
- Measuring data sparsity on held-out textEvaluate a model on held-out data
- Scoring a language model with loss and perplexityCompute cross-entropy loss and perplexity for a language model
- Comparing word, character and subword tokenizationExplain the trade-off between vocabulary size and sequence length
- Implementing byte-pair encodingImplement a BPE tokenizer in Python
- Counting tokens with tiktokenCount tokens for prompts and documents with a model's tokenizer
- Estimating token cost across languages and formatsEstimate token usage and cost for multilingual or structured inputs
- Building and sizing an embedding tableExplain how token ids map to embedding vectors
- Reading word-vector similarity with its limitsCompute cosine similarity and nearest neighbors in NumPy
- Explaining how position reaches a transformerExplain positional encodings in transformer models
- Computing logits, softmax and a baseline lossImplement softmax and cross-entropy loss in NumPy
- Computing attention as a weighted average of vectorsExplain how attention mixes token representations
- Implementing scaled dot-product attentionImplement query, key and value attention from the formula
- Applying and explaining the causal maskExplain autoregressive generation in decoder-style language models
- Sizing multi-head attention from a model configurationRead a transformer configuration for width and head count
- Reading a transformer block and counting a model's parametersEstimate model size from a published architecture configuration
- Training and scoring a small language modelExplain gradient descent and read a training log
- Reading chat templates and estimating prompt costExplain how a system prompt and chat history are sent to a language model
- Explaining language model behavior by mechanismExplain context windows, statelessness and hallucination to non-specialists
प्रश्न
प्रतिबद्ध होने से पहले
आज ही शुरू करें