Stanford Online
Stanford Online
  • 2 457
  • 38 398 897
Stanford CS25: V4 I Aligning Open Language Models
April 18, 2024
Speaker: Nathan Lambert, Allen Institute for AI (AI2)
Aligning Open Language Models
Since the emergence of ChatGPT there has been an explosion of methods and models attempting to make open language models easier to use. This talk retells the major chapters in the evolution of open chat, instruct, and aligned models, covering the most important techniques, datasets, and models. Alpaca, QLoRA, DPO, PPO, and everything in between will be covered. The talk will conclude with predictions and expectations for the future of aligning open language models. Slides posted here: docs.google.com/presentation/d/1quMyI4BAx4rvcDfk8jjv063bmHg4RxZd9mhQloXpMn0/edit?usp=sharing
All the models in the figures are in this HuggingFace collection: huggingface.co/collections/natolambert/lecture-artifacts-aligning-open-language-models-66197653411171cc9ec8e425
About the speaker:
Nathan Lambert is a Research Scientist at the Allen Institute for AI focusing on RLHF and the author of Interconnects.ai. Previously, he helped build an RLHF research team at HuggingFace. He received his PhD from the University of California, Berkeley working at the intersection of machine learning and robotics. He was advised by Professor Kristofer Pister in the Berkeley Autonomous Microsystems Lab and Roberto Calandra at Meta AI Research.
More about the course can be found here: web.stanford.edu/class/cs25/
View the entire CS25 Transformers United playlist: ua-cam.com/play/PLoROMvodv4rNiJRchCzutFw5ItR_Z27CM.html
Переглядів: 8 967

Відео

Stanford Seminar - The Human Factors of Formal Methods
Переглядів 1 тис.14 годин тому
April 19, 2024 Shriram Krishnamurthi, Brown University As formal methods improve in expressiveness and power, they create new opportunities for non-expert adoption. In principle, formal tools are now powerful enough to enable developers to scalably validate realistic systems artifacts without extensive formal training. However, realizing this potential for adoption requires attention to not onl...
Stanford CS236: Deep Generative Models I 2023 I Lecture 16 - Score Based Diffusion Models
Переглядів 2,7 тис.16 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 17 - Discrete Latent Variable Models
Переглядів 1,4 тис.16 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 18 - Diffusion Models for Discrete Data
Переглядів 4 тис.16 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 15 - Evaluation of Generative Models
Переглядів 1,3 тис.16 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 14 - Energy Based Models
Переглядів 72716 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 13 - Score Based Models
Переглядів 59916 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 12 - Energy Based Models
Переглядів 35716 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 11 - Energy Based Models
Переглядів 48316 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 10 - GANs
Переглядів 39216 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 9 - Normalizing Flows
Переглядів 38416 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 8 - GANs
Переглядів 36616 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 7 - Normalizing Flows
Переглядів 46416 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 6 - VAEs
Переглядів 54016 годин тому
For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai To follow along with the course, visit the course website: deepgenerativemodels.github.io/ Stefano Ermon Associate Professor of Computer Science, Stanford University cs.stanford.edu/~ermon/ Learn more about the online course and how to enroll: online.stanford.edu/courses/cs236-deep-generative-models To ...
Stanford CS236: Deep Generative Models I 2023 I Lecture 5 - VAEs
Переглядів 78016 годин тому
Stanford CS236: Deep Generative Models I 2023 I Lecture 5 - VAEs
Stanford CS236: Deep Generative Models I 2023 I Lecture 4 - Maximum Likelihood Learning
Переглядів 1,1 тис.16 годин тому
Stanford CS236: Deep Generative Models I 2023 I Lecture 4 - Maximum Likelihood Learning
Stanford CS236: Deep Generative Models I 2023 I Lecture 3 - Autoregressive Models
Переглядів 1,8 тис.16 годин тому
Stanford CS236: Deep Generative Models I 2023 I Lecture 3 - Autoregressive Models
Stanford CS236: Deep Generative Models I 2023 I Lecture 2 - Background
Переглядів 3,3 тис.16 годин тому
Stanford CS236: Deep Generative Models I 2023 I Lecture 2 - Background
Stanford CS236: Deep Generative Models I 2023 I Lecture 1 - Introduction
Переглядів 17 тис.16 годин тому
Stanford CS236: Deep Generative Models I 2023 I Lecture 1 - Introduction
Stanford CS25: V4 I Jason Wei & Hyung Won Chung of OpenAI
Переглядів 30 тис.16 годин тому
Stanford CS25: V4 I Jason Wei & Hyung Won Chung of OpenAI
Stanford Seminar - Towards trusted human-centric robot autonomy
Переглядів 1,7 тис.День тому
Stanford Seminar - Towards trusted human-centric robot autonomy
Information Session: Stanford Graduate Degrees, Certificates, and Courses I 2024
Переглядів 1,2 тис.14 днів тому
Information Session: Stanford Graduate Degrees, Certificates, and Courses I 2024
Information Session: Leading People, Culture, and Innovation Program
Переглядів 1 тис.14 днів тому
Information Session: Leading People, Culture, and Innovation Program
Stanford CS25: V4 I Overview of Transformers
Переглядів 37 тис.14 днів тому
Stanford CS25: V4 I Overview of Transformers
Stanford Seminar - Towards Safe and Efficient Learning in the Physical World
Переглядів 2,1 тис.21 день тому
Stanford Seminar - Towards Safe and Efficient Learning in the Physical World
Stanford EE274: Data Compression I 2023 I Lecture 18 - Video Compression
Переглядів 1 тис.21 день тому
Stanford EE274: Data Compression I 2023 I Lecture 18 - Video Compression
Stanford EE274: Data Compression I 2023 I Lecture 8 - Beyond IID distributions: Conditional entropy
Переглядів 58021 день тому
Stanford EE274: Data Compression I 2023 I Lecture 8 - Beyond IID distributions: Conditional entropy
Stanford EE274: Data Compression I 2023 I Lecture 5 - Asymptotic Equipartition Property
Переглядів 67621 день тому
Stanford EE274: Data Compression I 2023 I Lecture 5 - Asymptotic Equipartition Property
Stanford EE274: Data Compression I 2023 I Lecture 3 - Kraft Inequality, Entropy, Introduction to SCL
Переглядів 74321 день тому
Stanford EE274: Data Compression I 2023 I Lecture 3 - Kraft Inequality, Entropy, Introduction to SCL

КОМЕНТАРІ

  • @Roshan-tb3iz
    @Roshan-tb3iz 2 години тому

    IIT JEE 1984 top ten ranker. Gold medalist from IIT Kanpur, batch of 1988.

  • @srisaisubramanyamdavanam9912
    @srisaisubramanyamdavanam9912 3 години тому

    I think Laplace smoothing applied only during predictions. Why he is applying for parameters also??

  • @nerouchih3529
    @nerouchih3529 3 години тому

    28:00 A unique view at attention. In this image all 6 nodes are related with all 6 nodes in self-attention case. And in cross attention it would be like set A sends a message to nodes in set B. And voila, it's a fully-connected layer! But with tokens passed instead of values

  • @Lalala_1701
    @Lalala_1701 4 години тому

    Andrew ng also took same kind of example to explain LM.

  • @Lee-zo3dy
    @Lee-zo3dy 8 годин тому

    I think at 37:10 professor did not make it quite clear for probability = 0. The student confused probability with possibility. It is totally ok for thing A that is p(A) = 0 to happen to some extent. Am I right?

  • @DanBillings
    @DanBillings 8 годин тому

    Please put the subject of the talk in the title. You can then market the OpenAI speakers

  • @ChidinmaOnyeri
    @ChidinmaOnyeri 12 годин тому

    Hi. Can anyone recommend any textbook that can help in further study of this course. Thank you

  • @heyitsjoshd
    @heyitsjoshd 15 годин тому

    How do we know what is small vs large? For example, with emergent tasks, it highlights that more data could lead to more accuracy with enough compute. The small LM would have not seen accuracy improvements but the large LM did. For the tasks currently indicated as flat, couldn't we just not have enough compute now to know if these tasks would get more accurate?

  • @rucellegarciano4105
    @rucellegarciano4105 15 годин тому

    I could be wrong... But as I understand what mister Lamport is saying... This is just digital design... Combinational... Sequential circuits... I could also be wrong but... Clocks are more of combinational circuits... On the other hand, sequential circuits have clock circuits in them... 🤷

  • @zacharykosove9048
    @zacharykosove9048 19 годин тому

    The students were asking some great questions, no wonder I don't go to Stanford

  • @dodowoh3683
    @dodowoh3683 День тому

    Surprised by the amount of hair an AI scholar may have retained.

  • @AbdeeAwol
    @AbdeeAwol День тому

    100x😊

  • @hajerjm
    @hajerjm День тому

    Thank youuuu

  • @brashcrab
    @brashcrab День тому

    7217 1:07

  • @yuxingben399
    @yuxingben399 День тому

    Great introduction on deep generative models!

  • @arpitkumar592
    @arpitkumar592 День тому

    In the poker question, probability of A' is 42 options, right ? Since one of the 7 cards already on the table is A of clubs ?

  • @ahmad1239112
    @ahmad1239112 День тому

    Thanks for sharing this

  • @annalvarez3247
    @annalvarez3247 День тому

    This video is crazy 🔥🔥 interesting to know how crazy defi has grown and it’s great to be able to see this seminar through UA-cam it was an interesting chat to listen to defi is growing globally it’s not just in the us now it’s in other countries take in mind I am commenting from outside of the U.S crazy stuff excellent content 👍🏻

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w День тому

    Shouldn’t there be a different UA-cam channel for AI from Stanford.

  • @hedu5303
    @hedu5303 День тому

    Strange world. This dude is almost a kid and gives a lecture

    •  8 годин тому

      I am happy to learn from any kid :)

  • @joebobthe13th
    @joebobthe13th День тому

    Love the section on "kale divergence"! Thanks UA-cam auto-captioning! 😂

  • @chongsun7872
    @chongsun7872 День тому

    Great lecture. But some time a little faster pace than Christopher Manning.

  • @erichsiung9704
    @erichsiung9704 2 дні тому

    I am addicted to Prof. Jure's accent now😂!!!

  • @nivcohen5371
    @nivcohen5371 2 дні тому

    Very clear and interesting lecture

  • @CannabinatedFantasy
    @CannabinatedFantasy 2 дні тому

    omg his forehead

  • @tylerderdern9739
    @tylerderdern9739 2 дні тому

    it's a good idea, I like this subject

  • @Beverage21
    @Beverage21 2 дні тому

    1:03:51 that website what the Andrew doing there

  • @DougRigsby-tf2xw
    @DougRigsby-tf2xw 2 дні тому

    Great points. Maybe better for those starting out. Once you’ve been in a career with family, life has a way of creating boundaries. I’ve attempted many times to break out and do something different only to find myself unable to change. Interviewing for a career with experience in a different area yet same industry, likely you will be judged on your experience rather than your ability and willingness for change.

  • @elcanmhmmdli3305
    @elcanmhmmdli3305 2 дні тому

    Azerbaijan❤

  • @hajerjm
    @hajerjm 2 дні тому

    Thank you!

  • @SurajPrasad-mz1nx
    @SurajPrasad-mz1nx 2 дні тому

    Too Good to be Honest.

  • @Lee-zo3dy
    @Lee-zo3dy 2 дні тому

    Where can I find the problem sets? This is really import to me. Please someone help me!

  • @jonathanr4242
    @jonathanr4242 2 дні тому

    Thanks for sharing.

  • @aeroperea
    @aeroperea 2 дні тому

    wow

  • @fortuneolawale9113
    @fortuneolawale9113 3 дні тому

    thanks

  • @mahirturjo7509
    @mahirturjo7509 3 дні тому

    ❤❤❤❤❤❤

  • @nafikhan13-4-23
    @nafikhan13-4-23 3 дні тому

    I love 💓💓💓💓Stanford Online💓💓💓💓

  • @yuretenno1
    @yuretenno1 3 дні тому

    I firmly disagree. There must be an important distinction here: (a) expectation of growth by simple treasuring of commodities and currency; (b) expectation of growth by active exploration of the anticipated value by a third party in order to produce more value. The test must cover only option b, according to the reasons stablished in the video.

  • @Justjemming
    @Justjemming 3 дні тому

    The dice example for independence is wild! If event G sums to 7, it's independent from E or F but if it sums to a number less than 7 it's not? Would someone be able to explain this in some detail? Or provide some intuition? Thanks!

  • @TomTom-xh9tp
    @TomTom-xh9tp 3 дні тому

    Where to find the "other" videos that Andrew says the students can watch at home?

  • @atdt01410x
    @atdt01410x 3 дні тому

    This lecture is super useful. really appreciate.

  • @himanshusamariya9810
    @himanshusamariya9810 3 дні тому

    Just awesome 😊

  • @xX_BabooFrik_Xx
    @xX_BabooFrik_Xx 3 дні тому

    Love to maddie <3

  • @arjunkandaswamy
    @arjunkandaswamy 4 дні тому

    where is the full playlist?

  • @Beverage21
    @Beverage21 4 дні тому

    is this course still applicable in 2024 guys. after a lot advancements will this be sufficient to get started?

    • @akshat_senpai
      @akshat_senpai 4 дні тому

      No idea 😄 but I m looking friends 😅

  • @cheapearth6262
    @cheapearth6262 4 дні тому

    learning probability for 12th grade from standford lol

  • @forresthu6204
    @forresthu6204 4 дні тому

    two great minds of nowadays.

  • @user-my8vx3ls2u
    @user-my8vx3ls2u 4 дні тому

    Great presenter.

  • @MLLearner
    @MLLearner 4 дні тому

    00:10 Today's discussion is about supervised learning and locally weighted regression. 07:48 Locally weighted regression focuses on fitting a straight line to the training examples close to the prediction value. 16:15 Locally weighted linear regression is a good algorithm for low-dimensional datasets 22:30 Assumptions for housing price prediction 29:45 Linear regression falls out naturally from the assumptions made. 36:36 Maximum Likelihood Estimation is equivalent to the least squares algorithm 44:40 Linear regression is not a good algorithm for classification. 51:04 Logistic regression involves calculating the chance of a tumor being malignant or benign 58:30 Logistic regression uses gradient ascent to maximize the log-likelihood. 1:05:36 Newton's method is a faster algorithm than gradient ascent for optimizing the value of theta. 1:12:40 Newton's method is a fast algorithm that converges rapidly near the minimum. Crafted by Merlin AI.