(upbeat instrumental music)
Welcome back to the stage, Patrick Collison.
(upbeat instrumental music)
All right, good afternoon, folks.
I hope you've enjoyed the sessions
between now and when we last saw you this morning.
For this afternoon's keynote or fireside chat,
I suppose, I'm about to introduce somebody who needs little introduction.
Although, a fun fact
that you may not know about Jensen Huang is
that he's been a CEO of NVIDIA for 31 years this month,
making him the longest-serving CEO
in the technology industry.
And guess, therefore, logically that is...
(Audience cheers and whistles.)
So, John and I have only been doing it for a mere 14 years, so you know
even if we double that, we'll still be second to him.
And, Jensen, we'll talk about this on stage,
attended the Oneida Baptist Institute in Kentucky.
We'll definitely be asking about it.
Oregon State.
Worked as a waiter at Denny's, then...
(Audience laughs.)
Denny's close to here, actually.
LSI Logic, and then AMD, which is of course now run
by his first cousin once removed.
We'll definitely be asking about that.
Before he founded NVIDIA in 1993—
and NVIDIA's market cap was $8 billion
when Stripe launched in 2011,
and it is now, of course, more than 200 times that.
So, he's been busy since.
Please, welcome to the stage, Jensen Huang.
(Audience applauds.) (upbeat music)
Hey, everybody.
(Audience hoots.)
-So you watched the keynote earlier? -I did.
I've never seen a duet before.
You were so synchronized.
It seemed like the two of you knew each other.
It's incredible.
Some acquaintance.
But, okay, you've been doing keynotes a long time.
You are the keynotes GOAT, so...
Stop it.
So give us your... we don't have even a signature outfit yet.
We're just amateurs here.
So give us...
It's because you're still young.
(Audience laughs.)
Give us your keynote performance review. What'd you think?
I thought it was A+.
I thought it was... really.
(Audience cheers and applauds.)
You explained perfectly
the purpose of the company,
what inspires you guys, what keeps you guys up
what makes you work so hard, the ecosystem that you serve
the incredible platform you've built
the amazing contribution you make to the world's economy.
It's incredible. I thought it was great.
And there was a whole bunch of technology stuff,
feature stuff, money stuff.
I didn't understand any of that, but...
(Audience laughs.)
But something about a CYK or something.
What was that?
-KYC. -KYC. Yeah.
(Audience laughs.)
-I thought it was... -It's a big deal in our world.
Is that right? Kentucky Fried Chicken?
We take care of KYC
so that you can associate us with Kentucky Fried Chicken.
Okay, got it.
Did you...
Software-defined financial services
this idea, did that...
-Incredible. -Does that make sense to you?
Well, first of all, I think it's a giant idea.
Do you know where it came from?
You're going to tell me.
So Jensen and I were catching up, maybe...
But the part that I loved was how you realized, in the very beginning
that financial payments was about code, not finance.
I thought that was incredible.
And you explained that the first time we met.
So Jensen and I were catching up 18 months ago or so
and I guess it was a couple years since we'd last spoken.
So he was kind of asking for the update on Stripe.
And I was explaining, and you said
“Oh, so it's like software-defined networking
but for money.”
That was still ricocheting around in my mind.
So that's where we got to this idea for software-defined financial services.
So I hope we don't have to pay a licensing fee for that or something.
I got zero equity for that good idea.
All right, you guys are doing okay.
I was thinking about this.
Tesla's earnings were of course yesterday
and Elon announced that I think Tesla is going to have 85,000
H100s by the end of this year.
I was just reflecting on...
it's quite a success to sort of build a business where CEOs kind of compete
with each other to announce
who has spent more buying your product.
So, I think you've done something quite impressive.
But anyway, I actually want to start out talking a bit about...
All of my CEO friends, they all have the most.
(Audience laughs.)
I want to start out talking a little bit
about a remark you made at a Stanford event recently.
I was thinking of GSB, I think.
You said, “I wish upon you ample doses of pain and suffering.”
Elaborate.
(Jensen laughs.) (Audience laughs.)
Well, let's see.
There is a misunderstanding.
There's a phrase that said
“You should choose your career based on your passion.”
And usually, people connect passion with happiness.
I think there is something missing in that.
Nothing there is wrong, but there's something missing.
And the reason for that is because, if you want to do great things
and I know this to be true about you creating Stripe.
By the way, this is one of the world's finest CEOs.
Young as he may be.
(Audience applauds.)
You guys know I've met a lot of CEOs.
I've heard about a lot of companies
and this is genuinely one of the world's great visionary companies.
So anyways, I just wanted to say that.
And it's the reason why I'm here, I just love what...
No more compliments allowed. It makes us terribly uncomfortable.
I know. I could tell. I could see him.
He's starting to sweat.
(Audience laughs.)
And so, the thing is, when you want to build
something great
it's not easy to do.
And when you're doing something that's not easy to do
you're not always enjoying it.
I don't love every day of my job. I don't think every day brings me joy
nor does joy have to be the definition of a good day.
And every day, I'm not happy.
Every year I'm not happy about the company,
but I love the company every single second.
So I think that what people misunderstand is
somehow the best jobs are the ones that bring you happiness all the time.
I don't think that that's right.
You have to suffer.
You have to struggle. You have to endeavor.
You have to do those hard things and work through it
in order to really appreciate what you've done.
There are no such things that are great that were easy to do.
So by definition, I would say therefore
I wish upon you greatness, which by my way of saying it,
I wish upon you plenty of pain and suffering, and so.
(Audience applauds.)
Anything in your upbringing that taught you that idea?
Or is it just somehow innate to your makeup?
I didn't realize I had to lay down for this but...
(Audience laughs.)
I'm about to tell you things I've never told anyone, not even my family.
(Audience laughs.)
I was an immigrant.
And when I came in 1973, I was 9.
My older brother was almost 11.
This was a foreign country
and there was nothing easy about that.
We also grew up in a... really, really terrific parents,
but we weren't wealthy.
And so, they worked hard. They work hard today.
So they passed along a lot of life lessons
by working hard.
Now, I had all kinds of jobs.
We went to a school that included a lot of chores.
-It was in Kentucky. -Yeah. Kentucky, Oneida Baptist Institute.
And I...
I don't think it's the same as MIT
that “I” is not the same. It's the same word, but it's different.
It's a different type of institute.
But my institute required you to go to school,
and it was a dormitory, and so there were a lot of chores.
I was the youngest kid in school
and so all of the other kids got the hard work.
They had to work in the tobacco farm.
I got the easy job. I was 9 years old.
So after they left, I had to clean all the bathrooms.
I never felt that I got the easy job
because what they left behind was...
you can't unsee that kind of stuff.
(Audience laughs.)
But that was my job and so I did it delightfully.
Then I had plenty of other jobs, and Denny's was one of them.
I started out as a dishwasher and became a busboy and became a waiter.
And I loved every one of them. I loved every one of them.
Somehow, I've always found...
I want to say joy, but that's not quite right.
I just, everything that I was doing, I wanted to do the best I could.
And maybe that was kind of ingrained from the very beginning,
but I was definitely the best bathroom cleaner
the world's ever seen, I'm sure of it, yeah.
So if we fast-forward just a little bit
to the NVIDIA of today.
How large is your leadership team?
NVIDIA's leadership team is 60+, 60 people, yeah.
And they all report to you?
Yeah, they all report to me.
Sixty direct reports. -Sixty direct reports, yup.
Which is not conventionally considered a best practice.
(Audience laughs.)
I agree that may not be the best practice kind of...
I'm certain that's the best practice.
It's not conventional, but I am certain it's the best practice.
(Audience applauds.)
By the end of this, I'm going to convince all of you
to have 60 people on your direct reports.
(Audience laughs.)
The floor is yours.
First of all, the reason is because the layer of hierarchy
in your company really matters.
Information really matters.
I believe that your contribution
to the work should not be based
on the privileged access to information.
I don't do one-on-ones
and my staff is quite large.
Almost everything that I say I say to everybody all at the same time.
The reason for that is
because I don't really believe there's any information that I operate on
that somehow only one or two people should hear about.
“These are the challenges of the company”
or “This is the problem I'm trying to solve”
or “This is the direction we're trying to go into.
These are the new endeavors.” “This isn't working. That's working well.”
And so all of this type of information, everybody should be able to hear.
I love that everybody's working off of the same song sheet.
I love that there is no privileged access to information.
I love that we're able to all contribute to solving a problem.
And when you have 60 people in a room
and oftentimes, my staff meetings are once every other week,
it's all based on issues, whatever issues we have.
Everybody's there working on it at the same time.
Everybody heard the reasoning of the problem.
Everybody heard the reasoning of the solution.
Everybody heard everything.
And so that empowers people.
I believe that when you give everybody equal access
to information, it empowers people.
And so, that's number one, empowering.
Number two
if the CEO's direct staff is 60 people,
the number of layers you've removed
in a company is probably something like 7.
Depending on how it is.
Is it 60 at every layer?
As in, if I'm one of the fortunate 60,
do I also have 60 direct reports?
-No. -Okay.
I don't think that that's scalable downward.
And the reason for that is because you need more
and more supervision depending on certain levels.
And at the E-staff level,
if you're so unfortunate to be serving on NVIDIA's E-staff,
it's very unlikely you need a lot of managerial.
So I rarely find myself having to...
stand up for conventional wisdom.
But if I were to steel man the other side
I'd say, “Well, one-on-ones are where you provide coaching,
where you maybe talk through goals together,
personal goals, career advancement, what have you.
Where maybe you give feedback on something
that you see somebody systematically not doing so well and so forth.”
And there is all these things that one is, again
conventionally supposed to do in the one-on-one.
Do you not do those things, or do you do them in a different way?
Really good question.
I do it right there.
Right there in...
I give you feedback right there in front of everybody.
In fact, this is really a big deal.
First of all, feedback is learning.
Feedback is learning.
For what reason are you the only person who should learn this?
Now, you created the conditions
because of some mistake that you made
or silliness that you brought upon yourself.
(Audience laughs.)
We should all learn from that opportunity.
So you created the conditions, but we should all learn from it.
Does it make sense? And so, for me to explain to you
why that doesn't make sense or how I differ from it
half the time, I'm not right.
But, for me to reason through it
in front of everybody
helps everybody learn how to reason through it.
So the problem I have
with one-on-ones and taking feedback aside is
you deprive a whole bunch of people that same learning.
Learning from mistakes,
other people's mistakes, is the best way to learn.
Why learn from your own mistakes?
You know, why learn from your own embarrassment?
You got to learn from other people's embarrassment.
That's why we have case studies, and isn't that right?
We're trying to read from other people's disasters,
other people's tragedies.
Nothing makes us happier than that.
(Audience laughs.)
Have you succeeded in getting other leaders
at NVIDIA to adopt this practice?
Or is that difficult?
I give people the opportunity to decide for themselves
but I really discourage one-on-ones.
I really discourage one-on-ones.
Nothing is worse than the idea that somebody says
“Oh, Jensen wants us to do this.”
Why does that have to be said to anybody? Everybody should know.
Or somebody said,
“That E-staff said that.”
Nothing drives me nuttier than that.
You once told me that you really didn't like firing people
and very seldom did it.
Can you elaborate on that?
Well, I'd rather improve you than give up on you.
When you fire somebody, you're kind of saying...
Well, a lot of people say
“Well, it wasn't your fault,” or “I made the wrong choice.”
There are very few jobs.
Look, I used to clean bathrooms, and now I'm the CEO of a company.
I think you could learn it.
I'm pretty certain you can learn this.
There are a lot of things in life that I believe you can learn
and you just have to be given the opportunity to learn it.
I had the benefit of watching a lot of smart people do a lot of things.
I'm surrounded by 60 people that are doing smart things all the time.
They probably don't realize it
but I'm learning constantly from every single one of them.
So I don't like giving up on people, because I think they could improve.
So there's... It's kind of tongue in cheek,
but people know that I'd rather torture them into greatness.
That was the phrase that I was hoping to uncover.
Yeah, I remember you mentioned that.
Yeah. So I'd rather torture you into greatness
because I believe in you.
And I think coaches that really believe in their team
torture them into greatness.
And oftentimes they're so close.
Don't give up. They're so close.
Greatness, it comes all of a sudden.
One day it's like, “I got it.” Do you know what I'm saying?
That feeling that you didn't get it yesterday
and all of a sudden one day something clicked, and “Oh, I got it.”
Could you imagine if you gave up just that moment
right before you got it?
So I don't want you to give up on that.
So let's just keep torturing you.
How is...
How's your work-life balance?
(Audience laughs.)
Well, it depends on who you ask.
I think my work-life balance is really great.
It's really great.
I work as much as I can.
(Audience laughs.)
I feel like he's judging me.
(Audience laughs.)
You know, I'm older than you. I have more wisdom than you.
So what I...
(Audience laughs.)
These are all the highlights from our conversations
that I think more people should get to hear, so.
Well, I work from the moment I wake up
to the moment I go to bed.
And I work seven days a week.
When I'm not working, I'm thinking about working.
And when I'm working, I'm working, and so.
I sit through movies, but I don't remember them
because I'm thinking about work.
(Audience laughs.)
You know?
And so that's... but my work is not as, you know...
It's not “working” as in...
There's this problem, and you're trying to solve this problem.
You're thinking about what the company can be
and are there things that we could do even better.
Or sometimes, it's just trying to solve a problem.
But sometimes you're imagining the future
and boy, if we did this and that.
And it's working.
You're fantasizing, you're dreaming.
I mean, that's incredible.
Well, so yeah, to concretize this a little bit
and then, we will get to talk about AI,
which I hear is a thing these days, but...
-It's a thing. -Yeah. Officially a thing, TM.
But, to concretize this a bit,
what does a day in Jensen's life look like?
-Like when do you wake up? -Well, I used to wake up at 5.
These days I wake up at 6 because of my dogs.
And the reason why, 6 is somehow we decided
that 6 o'clock is when they should wake up.
And I don't know what it is.
I don't mind waking anybody up,
but I feel guilty when I wake the puppies up.
(Audience laughs.)
It actually burdens me.
So I don't want to move, it might...
they pick up on any vibration
in the house and it wakes them up.
So we stay in bed
and I just read in bed until 6 o'clock and it's time.
But you're thinking about GPUs?
Oh, yes, yeah, yeah, sure.
I'm obsessed about GPUs. I mean, what can you do?
I'm constantly... no, I'm just...
Then the day is all, I guess, group meetings
because it can't be one on one meetings.
Yeah. I get my work done before I go to work.
And then when I get to work...
(Audience laughs.)
How many meetings in a typical day?
Pretty much all day long.
So I select the meetings that are really important to me.
I try not to have regular meetings
regular operational meetings
because I've got amazing people in the company
who are doing regular operational meetings.
So we're pinch hitters.
CEOs are pinch hitters.
We should be working on the things that nobody else can or nobody else is.
So you're jumping into projects that are stuck
or offtrack.
-That's right. -Or new ideas.
Wherever we can move the needle.
No reporting. No reporting meetings.
I hate reporting meetings.
They don't want to report to me, and just problem meetings.
So problem meetings or idea meetings
or brainstorming meetings or creation meetings
or whatever it is.
Those are the meetings I go to.
Usually I call them.
I try really hard not to have Outlook manage my life.
We purposefully decide what kind of things
that we want to do, we want to work on.
So I try to live a life of purpose and I manage my time accordingly.
You used a phrase once
“Zero-billion-dollar markets,”
that zero-billion-dollar markets are your favorite markets.
-Yeah. -What do you mean?
If you take a step back, our purpose, almost all of our purposes should be
to go and do something that has never been done before
that is insanely hard to do—
that if you achieve it, it could make a real contribution.
-I know your company does that. -I try to do that.
If that's the case, it hasn't been done before
it's incredibly hard to do.
It's probably, and it's never been done before.
That market is probably zero billion dollars in size
because it has never been done before.
I'd rather be a market maker, market creator than a market taker.
You know, to create something new
that never existed before versus thinking about share.
I don't love thinking about share. I don't like the concept of share.
The reason for that is
because if you think about it in the big picture,
Stripe existed out of thin air.
You vaporized. You created something out of vapor.
It wasn't as if there was another...
something else.
So I'd like to think that we can come up
with something that is zero billion dollars.
A zero-billion-dollar market is
a good way to cause the company to think
about how to go create something for the first time.
So our mission is to grow the GDP of the internet.
I mean, the GDP of the internet—
a clause in that usually gets most of the attention.
But I think the most important part is just the verb grow.
Yeah.
Because, to your point we shouldn't be thinking about, well...
which are the transactions that are already happening
or which are the businesses that already exist.
We should be thinking about which are the transactions that don't exist.
-That's right. -Which are the businesses
-that don't exist. -Exactly.
The GDP of the world is around $100 trillion
but it doesn't have to be $100 trillion.
It could be $200 trillion or $1,000 trillion.
That's exactly right. That's exactly right.
Most of the value we're going to create
over the next several decades
are likely not limited by physical things.
So this is a pretty extraordinary time.
So with this concept of zero-billion-dollar markets
if I'm, again, at NVIDIA
am I coming to you with some proposal for some project
and maybe there's several billion dollars of CapEx involved
or, you know, it's a many-year pursuit or something
and there are no customers for it today.
There's no demand that I can demonstrate for it.
And you guys are just making a gut call to say that
“Yes, nobody's doing this today. We think they could.
We think they should. And therefore, we're going to pursue it.”
Really close. Yeah, it's kind of like that.
It's a gut call in the sense
that your intuition says something as a starting thesis
but then you have to reason through it.
And the reasoning of it is much, much more important to me
than a spreadsheet.
I hate spreadsheets
because you can make spreadsheets do whatever you want.
You can make any chart you want out of a spreadsheet.
You just got to type in some numbers.
So I don't love spreadsheets for that reason.
I love words for that reason.
Words are reasoning. Tell me, how did you reason through this?
What's our intuition? Why do we believe that matters?
Why do we think it's hard?
I like hard things, because it takes a long time to do.
And if it takes a long time to do,
a lot of people who are less committed probably won't do it.
If it's really, really hard to do, it takes a long time to do
it takes us a really resilient
and a really dedicated, really committed person to go after it.
And if it also takes a long time to do
you can kind of flounder around for a couple years
nobody notices.
So I could be incompetent for several years
and everybody goes, well, who saw it?
And where did CUDA come from?
CUDA came originally from two ideas.
One is called...
I hate to get technical, but we created—
we pioneered this idea called accelerated computing.
Accelerated computing is like an IO device,
something that you sit on PCI Express, if anybody's in the computer business,
an IO device that allows
the application to interact with that IO device in such a way
as to accelerate parts of the application.
And UDA was an invention in 1993, and this really profound invention allows
the software programmer to directly program
an IO device, write an application directly to the IO device
because the IO device is virtualized
and it's...
architecturally compatible across multiple generations.
Anyways, we invented this idea called accelerated computing
and that was, we called it Unified Driver Architecture
for whatever reason.
And then, several years later
we thought we could make our GPUs more programmable
to high-level programming languages
and we invented this idea called CG.
C for graphics.
C for graphics processors.
That opened up some really exciting opportunities.
And we thought, you know what?
This is going to work, but CG,
the programming model, wasn't exactly right.
And so we extended, we invented CUDA,
which is compute with... So anyways, that's how.
It's a horrible story, frankly. (Audience laughs.)
Anyways, we invented this idea called accelerated computing.
We pioneered this approach.
I guess the real question is, was it a smash hit overnight?
No, it was a...
It was an incredible disaster overnight.
And it kind of went like this.
So this is one of your zero- billion-dollar markets you went after.
Yeah.
-And it was a disaster. -Yeah.
Because it was a zero billion dollar we went after,
but it cost so much to go after that zero-billion-dollar market
it actually crushed the $1 billion market we were enjoying.
So, and the reason for that is
because CUDA
added a ton of cost into our chips.
But there were no applications. And there are no applications.
Customers don't value the product, and they won't pay you a premium for it.
And if people aren't willing to pay you for it
but your cost went up, then your gross margins get crushed.
And we got...
Our market cap was low
and it went down to really low.
It was like, I think our market cap went down
to like a billion dollars or something like that.
-I wish I bought it, but anyways... -Okay.
So therefore, you immediately canceled CUDA
and went back to the old strategy.
No, no, I believed in CUDA
because you reasoned about it.
You reasoned about it. Look, we really believe
that accelerated computing was going to be able to solve
problems that normal computers couldn't.
And if we wanted to extend
the architecture to be much more general purpose
we had to make that sacrifice.
So I deeply believed in the mission of our company.
I deeply believed in its opportunities.
And so were analysts...
And I deeply, deeply believed that people were wrong.
They just didn't appreciate what we built.
I deeply believed it.
And so, weren't analysts and the board and employee
like, “You've torpedoed this existing revenue stream.
You've this hyped thing that...
you're selling a lofty dream around that nobody seems to actually want.
The business is really suffering.”
Talk us through that.
You believed.
You just go something like this. “Oh gosh, they're so dumb.”
(Audience laughs.)
Something like that.
You know, denial. No, I'm just kidding.
No, you go back to what you believe.
And if you believe something...
Did the board put pressure on you during this?
I'd start every conversation with what I deeply believed.
And they believed it
because they saw me deeply believe it.
And I reasoned about it.
It wasn't like it was a spreadsheet
and therefore, you've got to believe the spreadsheet.
They had to believe the reasoning, the words.
How long did it take it to start working?
Probably 10 years, yeah.
-Yeah. It wasn't that long. -Yeah.
Ten years. It comes and goes.
-Ten years. -Less than a third of your tenure.
Yeah, it comes and goes. It was, I barely remembered it.
The suffering, I barely remembered it.
Could NVIDIA be as successful in AI without CUDA?
No. Impossible.
It is potentially one of the most important inventions
in modern computing.
We invented this idea called accelerated computing.
And the idea is so simple, but deeply profound.
It says the vast majority...
a small percentage of the code of programs
occupies, consumes 99.999% of the runtime.
This is true for a lot of very important applications.
That small little kernel
or, you know, some, several kernels...
can be accelerated.
And they tend...
It's not all just parallel processing. It's not as simple as that.
But the idea is that we can take that kernel
that piece of software, that part of the software
and accelerate the living daylights out of it.
And today, when Moore's Law has run its course
and CPU scaling is basically stopped.
If we don't accelerate every software
you're going to see extraordinary computation inflation.
Because the amount of computation the world does is
doubling every year still, and yet, if CPUs
and general-purpose computers are not increasing
in performance because it's stopped,
then what's your alternative?
Or your cost of computing is going to keep going up exponentially.
So the time has come for us to do that.
So everyone here runs a business, and...
-Accelerate everything. -And you heard it here first.
And probably everyone has...
some version of CUDA
or a thing that they think really makes sense
for the sector or makes sense for their technology or what have you
but where the market doesn't see it yet.
Do you think it's possible to extract any kind of generalizable principles
around when you should really doggedly trust that vision
and when perhaps it's worth reconsidering in a fashion that, yeah
we could extrapolate from, in the case of CUDA
and other CUDAs that have existed over the course of NVIDIA's history?
Yeah, the question is determination
and commitment versus stubbornness.
And that line is fuzzy.
Look, I gut-checked against my core beliefs every day.
I still do. And you gut-check against it.
The first principles by which you reasoned
about your strategies
the first principles by which you reasoned about your strategies
those first principles are easy to remember.
It's not a long list.
Now the question is, did those principles...
did they change in some fundamental way?
Are external conditions such that they no longer matter
as much as before?
Did somebody else solve the problem
and therefore, that problem has now disappeared?
Is it, there will never be any need?
You gut-check it constantly
to the extent that that's number one, gut-check.
You have to, first of all, you really have to be careful to distill down
the first principle instead of “I want something.”
That's stubbornness.
You can't reason about it. I just want it.
We're not 5-year-olds, right?
So you got to reason about it, number one.
Number two, you have to be clever.
The fact of the matter is
there are a lot of new companies being created here.
It's amazing how many great companies are in the audience
and young companies in the audience.
You have to be clever.
So we found ways to monetize
even in a small way, CUDA.
And so, we found app, we looked everywhere for applications.
We found an application with CT reconstruction.
We found an application with seismic processing.
We found another application with molecular dynamics.
And so we're constantly looking for applications.
They didn't make it a home run
but it sustained us just enough, just enough
and bought us time for it to really happen.
Okay, so let's talk about AI.
Maybe just going to do some math to ground things here.
Let's just say that the total compute capacity
of all GPUs in the world today is X.
What do you think...
What multiple of X will we be at in five years?
First of all, you know that I'm going to regret saying this.
And this is...
I'm a public company, you crazy person.
(Audience laughs.)
See, this is... how nice is it to be private?
(audience laughs)
(Audience applauds.)
Safe to say considerably more?
Well, let's reason about it, shall we?
Okay, so let's reason about it. Let's reason our way through, okay?
So first of all, it goes like this.
The world has installed
about a trillion dollars worth of data centers.
Those trillion dollars worth of data centers used
as general-purpose computing.
General-purpose computing has run its course.
We cannot continue to process that way.
And so, the world is going to accelerate everything
data processing, you name it, okay?
So we're going to accelerate everything.
When we accelerate everything, every single data center,
every single computer will be an accelerated server.
Well, there's about a trillion dollars worth
of computers if we don't grow at all
over the next, call it four years that we have to go replace.
Four years, six years, pick your number of years.
But, if the computer industry continues to grow
at some 20% or so,
we'll probably have to replace over the course of next...
pick your number of years, about $2 trillion worth of computers
with accelerated computing.
So, just make that GPUs, okay?
That's number one. And this is the second part.
This is the reason why all of you, Stripe
you're onto something just absolutely monumental.
This idea called, and...
you've heard me say an industrial revolution.
Lemme tell you why.
We are producing something for the very first time
that has never been produced before.
And we're producing it in extremely high volume.
And the production of this thing requires
a new instrument that never existed before.
It's a GPU.
The thing that we're producing
for the very first time, for the mathematicians
and all the computer scientists in the room, for all of you
you know that we're producing tokens.
We're producing floating point numbers at high volume
for the first time in history.
The floating-point numbers have value.
The reason why they have value is because it's intelligence.
It's artificial intelligence.
You can take these floating-point numbers,
you reformulate it in such a way that it turns into English,
French, proteins, chemicals, graphics, images,
videos, robotic articulation, steering wheel articulation.
We're producing tokens at extraordinary scale.
Now, we've discovered a way through all of the work that we do
with artificial intelligence to produce tokens
of almost any kind.
So now, the world is going to produce
an enormous amount of tokens.
Now these tokens are going to be produced
in new types of data centers.
We call them AI factories.
Back in the last industrial revolution,
water comes into a machine, you light the water on fire.
Turn it into steam
and then it turns into electrons.
Atoms come in; electrons go out.
In this new industrial revolution,
electrons come in and floating-point numbers come out.
And just like the last industrial revolution,
nobody understood why this electricity is so valuable
and is now sold, marketed kilowatt hours
per dollar.
And so, now we have million tokens per dollar.
That same logic is as incomprehensible
to a lot of people as the last industrial revolution,
but it's going to be completely normal in the next 10 years.
Well, these tokens are going to create
new products, new services,
enhanced productivity on whole slew of industries,
a hundred trillion dollars worth of industries on top of us.
So this industry is going to be gigantic.
In order to monetize that, transact that
you're going to need Stripe, and so.
(Audience applauds.)
I got to tell you: this is one of my favorite companies.
The first time I met Patrick, he had to explain Stripe to me.
I was, first of all, it was so complicated.
Because it's complicated.
We tried to refine the descriptions over time.
But you got an early version. -No, you're in a complicated business.
no matter what.
But nonetheless, I was so inspired by it. Incredible what you guys have built.
Are we going to get you migrated to Stripe Billing
now that we have usage-based billing?
I wish I had a business that required billing.
I think your public filing suggests you're doing a lot of billing.
(Audience laughs.)
-We'll follow up on it. -All right, so.
It's only 10 transactions, just so you know.
(Audience laughs.)
Your economics serving us is like nothing
It's like 10 transactions.
Remember, we'd happily take the 2.9%, but anyway.
(Jensen laughs)
We can discuss that separately.
So...
-Done. -Think about this token.
You can't say that. You're a public company.
(Audience laughs.)
So thinking about these...
token factories, I feel like a big question right now is
whether the models saturate in the sense that, you know
we demoed the Sigma Assistant on stage earlier.
And you can write some natural language
and we convert that to SQL.
And going from, you know, maybe a 7 billion parameter model
to a 70 billion parameter model
or something like that, there might be a significant kind of...
consequential improvement in query accuracy for the user
for the typical kind of queries that people tend to construct.
But, maybe going to a model
of 10x larger than that is sort of unnecessary.
Like, at some point, you get too good enough
you can reliably convert the natural language in SQL.
I think there's a question of...
for the use cases for which LLMs are being deployed,
what does that saturation curve look like
and for how many use cases does one need a trillion-parameter model
or a 10 trillion-parameter model?
Or do we simply reach a point where
some number that is say less than 100 billion is sufficient?
Do you have any point of view on that?
Or is that even...
a reasonable way to look at the question in the first place?
Okay, let's break it down. Let's reason about it.
-In public appropriately. -In almost everything
every question I get, let's break it down, let's reason about it.
So let's start with an example.
In 2012, AlexNet was computer vision
ImageNet, image recognition, 82% or something like that, accuracy.
Over the next...
almost not quite 10 years, I think it was like 7 years,
every single year, the accuracy error reduced in half.
Every year, the error reduced in half, or otherwise known as Moore's Law.
Okay?
So you doubled the performance, you double the accuracy,
and you double its believability every single year.
Over the course of seven years, it's now superhuman.
Same thing with speech recognition.
Same things with natural language understanding.
We want to know, we want to believe, not know.
We want to believe that the answer that's being predicted to us is accurate.
We want to believe that.
And so the industry is going to chase
that believability or that accuracy
and double its accuracy
2x every year.
I believe that's going to be the same thing
with natural language understanding.
And, of course, the problem space is a lot more complicated.
But I have every certainty that we're going to double its accuracy
every single year to the point where it is so accurate.
We've largely tested
across many of your examples when you interact with it
that you go, “You know what? This is really, really good.
I believe the answer that it's producing for me.”
That condition is very important.
The second thing is this:
today's language models, today's AI
and everything that we've shown are one shot.
And yet, you and I both know that there are many things that we think about
that are not one shot.
You have to iterate.
So how do you come up, how do you reason about a plan?
How do you come up with a strategy to solve a problem?
Maybe you need to use tools.
Maybe you have to look up some proprietary data.
Maybe you have to do some research, in fact.
Maybe you have to ask another agent.
Maybe you have another, ask another AI.
Maybe you have to be a human in the loop, ask a human.
Trigger an event, send an email to somebody
or text to somebody, get a response
before you can move on to the next step of that plan.
And so, a large language model has to iterate
and think of a plan.
That's not a one-shot thing.
And once it comes up with a plan, as it traverses that graph
there's a whole bunch of language models
that are going to get instantiated and initiated.
So I think your future models are going to iterate.
So instead of a one-shot model,
it's going to be a planning model
with a whole bunch of other models around it
that are particularly good at particular skills.
And so, I think we have long ways to go.
Meta garnered a lot of attention last week for the release of Llama 3,
which seems to be the most impressive open-source model
thus far.
Any thoughts on open-source models?
If you ask me what are the top most important events
in the last couple of years,
I would tell you, of course, ChatGPT
reinforcement learning, human feedback, grounding into human values
and having the technology necessary to do that
obviously a breakthrough and democratized computing.
It made it possible for everybody to be a programmer.
Everybody's now doing amazing things with it.
ChatGPT, the work that OpenAI did.
Greg and Sam and the team, really proud of them.
The second thing that I would say,
that it that is
just as important, I would say is Llama,
not Llama 1, but Llama 2.
Llama 2
activated just about every industry to jump into working on generative AI.
And it opened the floodgates
of every industry being able to access this technology.
Health care, financial services, you name it
manufacturing, you name it, customer service
retail, all kinds.
I think Llama 2 and Llama 3,
because it's open sourced, it engaged research
and engaged startups, engaged industry.
It made generative AI accessible.
I think that's a very big deal.
And so, I think ChatGPT democratized computing.
I think Llama democratized generative AI. Does that make sense?
And I think without it, it's very hard to have activated
all of the research on safety
and all of the different ways of chains of thoughts
and all the reasoning technology that's now being developed
and all the reinforcement learning stuff.
That stuff would've been very hard to have activated without Llama.
Dario Amodei was on Ezra Klein's podcast two weeks ago
and he, as many others have, many others in particular who are...
involved with Frontier Labs, was predicting
AGI in the relatively near term, conceivably the next couple of years,
years like 2027 and so on are frequently thrown around.
Thoughts?
Depending on how you define AGI.
Now, first of all, as an engineer
you know that we can only solve a problem
ultimately if you can measure it.
And so, you have to express the problem statement,
the mission somehow in some measurable way.
If you told me that AGI is the list of benchmarks we currently use,
they're math tests and English comprehension tests
and reasoning tests, and you know.
You got medical exams and bars.
You make your list of all of the tests that you want.
It doesn't matter what it is. Just make your list.
If you make your list
I am certain we will achieve excellent results
in a very nominal amount of time.
And if that's the definition of AGI,
I'll make a guess
it's probably, definitely, within the next five years.
So all of the tests that we currently measure these models with
their accuracy or their error rate is reducing
in half every six months.
So there's no reason why we shouldn't expect it
all to be superhuman pretty soon.
So again, everyone in this audience...
But that doesn't meet the standard.
Just be clear. That doesn't meet the standard
of a normal person thinking it's AGI.
Does that make sense?
A on-the-street person, hey, AGI, that's probably not what they're thinking
what I defined it as.
The way I defined it is simply an engineering way
of defining it so that you can answer that question.
The second way of answering the question is
when can you achieve AGI in an undefined way?
If it's undefinable, then how long would you know...
How long would it take? Undefinable.
So everyone in this audience, again, runs a business.
And so, a practical question they/we all face is
how do you know if you are
in the face of the kinds of changes you just depicted,
how does one know, how can one know whether one is responding appropriately
sufficiently in the right ways, etcetera?
Any advice?
If you're not engaging AI actively
and aggressively, you're doing it wrong.
You're not going to lose your job to AI.
You're going to lose your job to somebody who uses AI.
Your company is not going to go out of business because of AI.
Your company is going to go out of business
because another company used AI.
There's no question about that.
And so, you have to engage AI as quickly as possible.
You have to engage AI as quickly as possible
so that you could do
things that you think cost too much to do.
For example, if the marginal cost of intelligence was
practically zero, there are a lot of things
that you would do now
that you wouldn't have done otherwise.
And so, notice how often we do search
and these days, notice how often we ask questions.
I mean, any random question I'll be asking Perplexity, right?
And so, why not? Just in case.
Aravind just gave a talk here at Sessions.
Okay. I love using it.
And even if I know the answer, I'll just ask it anyways.
You know, just to see what it comes up with.
And so I think we want that to happen.
We want the marginal cost of these types of activities to be
as low as possible so that you use it in abundance.
Second, if you could use AI to be productive
you know that productive companies leads
to higher earnings.
Higher earnings leads to more employment.
More employment leads to more social growth.
So there is a lot of reasons to want to drive
productivity into companies.
And apart from just changing your manufacturing plans
and your CapEx plans,
how has AI changed how NVIDIA works internally?
We were one of the first technology companies to invest
in our own AI supercomputers.
We can't design a chip anymore without AI.
At night, our AIs are exploring design spaces
vast and wide that we would never do ourselves
because it costs too much money to explore it.
And so we... I...
Our chips are so much better.
Because of an AI, we could reduce the amount
of energy used for our chips as higher performance.
Our software, we can't write software without AI anymore.
We have to explore all the...
The design space of optimizing compilers is too large.
We use AIs to file bugs.
So our bugs database actually tells you
who's... what's wrong with the code
who's likely involved
and activates that person to go fix it, you know?
And so, I think I want everybody,
every organization in our company to use AI very aggressively.
I want to turn NVIDIA into one giant AI.
How great would that be?
And then, I'll have work-life balance.
(Audience laughs.)
Are there any favorite examples you've heard of businesses
and maybe in some kind of unexpected sector
or some unexpected use case
where you feel they kind of can serve as a poster child for some of the dynamics
you're describing where they've really realized
some of this opportunity?
Well, the biggest surprise of AI
that shouldn't be a surprise
for a lot of people is
that when we say, “It's a large language model,”
the word language doesn't mean human language only.
And it doesn't mean English only or French only
or Irish only, that's a whole different language but...
Is there a large language model for Irish?
-I've tried it. -That works?
Yeah. It works well.
John and I spent most of our education
in Ireland being taught in Irish.
So these models are some of the first people
I've had the chance to have a dialogue with
as Gaeilge, in...
-Very surprising. -Many years.
And actually I've been enjoying, have you played with Suno?
-Suno? -Suno.
Suno is an app for creating music.
-Synthetic music. -Okay.
-And I've been enjoying creating... -Irish music.
I, of course, tested it on that.
And Celtic dubstep is a thing that it can do.
Fantastic. Okay. Makes sense.
Like, if it could do that,
then of course they could learn the language of life.
Of course, they could learn
and if a language model can understand sound,
which is a sequence time series, it's a sequence
why can't it learn robotics articulation,
which is a sequence?
You just have to figure out how to tokenize it.
So the idea that all of a sudden
“Oh hey, look, listen, I could also learn SQL.
I could learn ABAP, I could learn Lightning.
I could learn all these proprietary languages.
I could learn Verilog,” I could learn, right?
So, all of a sudden, you realized, hang on a second
I can put a Copilot on top of every tool on the planet.
Well, and to this point, and you know, NVIDIA being one big AI
is the future one of 100,000 models
or 100 million models
or is the future one of one model
and there's just like a model that does all the things.
I think that it would be great to have...
It would be great to have supermodels
that help you reason about things in general, but...
For us, for all companies
that have very specific, domain-specific expertise,
we're going to have to train our own models.
And the reason for that is because we have a proprietary language.
That difference between 99% and 99.3% is
the difference between life and death for us.
So it's too valuable to us. No different than fraud detection for you.
-I was going to say... -It's too important to you.
-That's been exactly our experience. -Yeah. It's too important to you.
However good the general model is,
you're going to want to take that and fine-tune it
and improve it into perfection because it's just too important to you.
So we're going to shortly run out of time here
and there's a whole bunch of questions I haven't gotten to yet.
I've exercised poor discipline on the time management front.
So there's a bunch that I think are...
I was told I definitely had to ask you
but there's a couple that I really wanted to ask
and it's only us up here, so.
Lisa Su is your first cousin once removed?
Yes. She's terrific. She's amazing.
-And then, AMD is now... -She's the CEO of AMD, by the way.
Yeah. And AMD is now one of your competitors in the GPU space?
No.
(Audience laughs.)
We're family.
-Okay. -We're all in the industry.
-We're in the industry. -One of your partners in the industry.
Yeah. Yeah. We buy from AMD.
What's going on in the water?
How did we end up with two of the...
arguably the two most important GPU companies
being run by close relatives?
What's going on?
(Audience laughs.)
You got to keep it close to the family.
No, I just, I have no idea how it happened.
We didn't grow up together and we didn't know each other.
That makes it even more interesting, right?
Yeah. We didn't even know each other until she was at IBM.
And her career is incredible.
She's really quite extraordinary, yeah.
I think this question requires further study.
-Yeah. -So.
Okay, you've been operating in Silicon Valley
since the early '90s. -Yes.
How has Silicon Valley culture changed in that time?
Oh, wow.
I haven't thought about this in a long time.
I guess in a lot of ways, in a lot of ways, probably...
Okay, here's one.
When I first started NVIDIA
I was 29 years old and...
I was 29 years old with acne.
And...
you go talk to your... go recruit
law firms and VCs
and you know, I got a big zit on my forehead.
I don't have one today
so I feel comfortable talking about it.
But it could happen.
And so anyways
you feel rather insecure, because most of CEOs back then wore suits
and they're quite accomplished, and they sound like adults
and they use big words
and they talk about business and things like that.
So when you're young, you feel rather intimidated.
You're surrounded by a bunch of adults.
Well, you know, now, if you don't have acne,
I don't think you deserve to start a company.
(Audience laughs.)
I just, that's one big difference.
Acne.
You know, the takeaway from Jensen's speech.
I just, what it means is really...
we've enabled younger people to be extraordinary.
I think that the young generation of CEOs
the type of things that you guys know
at such a young age is really quite extraordinary.
I mean, it took me decades to learn it, and so.
Last question.
That was a compliment. See how he quickly, he changed it?
I wasn't saying you have acne. I was just say you were smart.
(Audience laughs.)
NVIDIA has a market cap of roughly $2 trillion dollars
and you're...
and you're now within spitting distance
of Apple and Microsoft.
And I just checked, and they have 220,000
and 160,000 employees respectively.
NVIDIA has 28,000 employees.
So, you know, less than a fifth
of the smaller of the two there.
And then, you just said, when we were chatting backstage
and I jotted this down
“You can achieve operational excellence through process
but craft can only be achieved with tenure.”
And so, NVIDIA is considerably smaller
than any of the other giants.
And you seem to think that tenure really matters
and I guess that craft really matters.
Say a little bit more there.
The, I think, extraordinary thing...
I think a lot of good things could be made.
Good things are made with operational excellence.
But you can't make extraordinary things
through just operational excellence.
And the reason for that is because a lot of the great things
in your body of work
and the products that you make, the company you created,
the organizations you've nurtured,
it takes loving care.
And you can't even put it in words.
How do you put loving care in an email?
And for people to go, “Oh, I know exactly what to do.”
You can't put that...
you can't put that in a business process, loving care, and...
Is love and care kind of an NVIDIA catchphrase?
Well, I use love fairly abundantly
and care, I use abundantly.
At Stripe, we talk a lot about craft and beauty.
Yeah, right. You have to use these words
because, that in a lot of ways, there are no other words to describe it.
You can't put it in numbers.
You can't write it in the product specification.
The product specification says, “I want you to build something great
that's incredibly beautiful, in great, great craft.”
You can't specify these things, and so.
But I'm sure there is people at Stripe who think,
“Patrick's always yammering on
about craft and beauty, and it's this kind of...”
I never yammer. I just want to let you know that.
I don't even know what that sounds like.
-Okay, well, yeah. -Yammering on.
-Go ahead. Go ahead. -Yeah, you're more lucid than I am.
I just babble. But hey, so Patrick is always going on
about this craft and beauty stuff
and wants things to have this particular ineffable character
but it doesn't directly serve some customer need and so forth.
Like, customers aren't coming to us and saying
“I want the product to be more beautiful.”
They're saying, “I want it to feature X or feature Y.”
And yet, we believe that the craft and beauty really matters.
It sounds like you're getting at something similar.
Why do you think it matters?
Actually, your customers, even though they didn't say it
they might not have the words to say it, but when they experience it, they know it.
There's no question.
Look, I...
Look, Stripe's work has beauty, has elegance,
has simplicity.
Simplicity is not simple as you guys know.
Simplicity and simple are not the same thing.
And it has elegance and it solves the problem
but just enough.
It burdens you, but not too much. You know?
And so that... and that balance is hard to find.
And you can't specify that. You just feel your way there.
And when you have a team that's with you
that feels the way they're together, in a lot of ways
we've codified, we've encoded
the magic of the company
in a way that no words can describe.
And you don't want to lose that. You don't want to lose that.
You want to take that and take it to the next level next time.
And so I don't want to reset.
I don't like working with new people for that reason
because I've encoded, I've embodied, I've deposited
so much pain, suffering, joy, knowledge, right?
All that experience, life experience
you've encoded it in, all the people that you've worked with.
You want to carry it on. You want to take it to the next level.
And that's really the reason why I really deeply believe in tenure.
And because of that, small teams could do great things.
And NVIDIA is kind of a small team, we're 28,000 people.
People think we punch well above our weight
because of that reason.
And so, it's amazing what you guys have done
and how incredibly small you are, 7,000 people supporting a trillion dollars
worth of ecosystem and industry and...
and economy, and who knows how far you guys can go.
-So, I'm very proud of you. -Jensen, thank you.
(Audience applauds.)
(upbeat music)