I'd like to uh welcome our second and final uh plenary.
Uh to the stage.
Um up next is Yan lukan.
Uh he's a chief AI scientist at meta and professor at.
NYU.
Now Yan was the founding director of meta and of the NYU.
Uh Center I should say for data science.
He works Prim primarily in a number of fields machine learning computer vision.
Uh mobile Robotics and computational.
Euro science.
In 2019 Yan won the prestigious ACM touring award for his work on AI and he's of course a member.
Of uh the US nationaly.
And the French Academy the sance a warm welcome to you Yan good to have.
You.
Thank you very much a real pleasure to be here.
Uh last time must have been before covid or something um.
Okay um there's going to be some uh connection a little bit with what Bernard.
Just talked about.
Um and what I'm going to talk about is all the stuff that Mark Jordan earlier today told you you shouldn't be.
Working.
On.
Um.
So as a matter of fact we do need human level.
AI.
Um and it's not just because it's an interesting scientific question.
It's also sort of a product need.
Um we are going to be.
Uh wearing.
Smart devices like smart glasses and things of that type in the future and in in those smart.
U devices we'll be able to.
Um access AI assistants that will be with us at all times and we'll be interacting with them either through voice or.
Through U.
Uh electron.
Um electrogram.
CMG.
Um the glasses will eventually have displays although currently they don't.
And.
And we need those system to have human level intelligence because that's what we're the most familiar.
Um um interacting with we're familiar with interacting with other humans.
Uh we are familiar with the level of intelligence that we expect in a in a human.
And uh it would be more you know easier to.
Interact with systems that have kind of similar forms of intelligence.
Um so you know those ubiquitous.
Assistants.
Um are going to mediate all of our interactions with the digital world.
And um that that's why that's why we we we need them to be easy to use for a wide.
Population that is not necessarily familiar with um using.
Technology.
Okay but the problem is machine learning.
Sucks.
Compared to what we observe in humans and animals.
Uh we don't really have the techniques that would um allow us.
To build machines that have the the same type of.
Uh.
Learning abilities and Common Sense and.
Understanding of the physical world.
Um so animals and humans.
Um have background knowledge that allows them to.
Learn new tasks extremely quickly.
Understand how the world Works.
Um being able to reason and plan.
And that's based on what we call common sense it's not a very well- defined concept.
Um and and.
Our behavior and behaviors of animals are driven by objectives.
Essentially.
So.
I'm going to argue that the type of AI systems that we uh we have at the moment.
Um or or that everybody is you know playing with almost everybody is playing with.
Uh do not have the right characteristics.
Uh for for what we want.
And the reason is uh they basically.
Produce.
One token after the other autor regressively.
Right so you have a sequence of tokens which are subo units but it doesn't matter what they are a sequence of.
Symbols.
And then you have a predictor that is repeated over the sequence that Bic basically take a window.
Of previous tokens and predict the next.
Token.
Um and the way you train those system is that you put the sequence at the at the input and I really.
Apologize for this I'm going to perhaps.
Change the.
Resolution.
Of the.
Screen.
That.
We don't have this constant.
Flashing.
Hang on just one second.
Okay.
Not sure this is better but.
Hopefully.
All right.
So the way those things are trained is you take a sequence and you basically train the system to just reproduce its.
Input on its output and because it has a causal structure.
Um it cannot cheat and use a particular input to predict itself it has to only look at the symbols that are.
To the left of it that's called causal.
Architecture.
So that's very efficient this is you know what people people call a GPT general purpose Transformer but you don't have to.
Put Transformers in it this could be anything it's just a caal.
And I'm afraid I haven't fixed the flashing anyway.
Um so the the the way you train the uh those systems uh then you can use it to generate text by.
Just Auto aggressively.
Producing a token shifting it into the input and then producing the second token shifting that in ETC that's Auto prediction Not.
A New Concept at all obviously.
Um and there's an issue with this which is that um.
The.
U.
The that process is basically.
Divergent.
Every time you produce a token there is some chance that the token is not.
Within the set of reasonable answers and take you outside a set of reasonable answers and if it does that there is.
No way to fix it afterwards.
Um and if there is if you assume there is some probability for that you know wrong token.
Uh for wrong tokens to be generated and the errors are independent which of course they're not.
Um then you get exponential.
Divergence.
Uh which is why you know we have.
With those models hallucination.
Issues.
Um but we're missing something really big because.
Uh you know never mind trying to reproduce human intelligence we can even reproduce cat intelligence or rat intelligence let alone dog.
Intelligence.
They can do amazing feits they understand the physical world.
Um you know any house cat can plan very highly complex.
Um actions.
Um and they have causal models of of the world.
Some of them know how to open doors and and Taps and things of that type.
Um and in humans you know a 10-year-old can.
Clear up the dinner table and fill up the dishwasher without learning zero shot the first time you ask a 10-year-old to.
Do it.
Um yeah she will do it any 17-year-old can learn to drive a car in 20 hours of practice.
But we still don't have robots that can act like a cat we don't have domestic robots that can clear up the.
Dinner table and we don't have level five cell driving cars despite the fact that we have hundreds of thousands if not.
Millions of hours of supervis training data.
Okay so that tells you we're missing something really.
Big.
Um yet we have systems that can pass the bar exam do math problems prove theorems.
But.
No domestic robots.
So we keep bumping into this Paradox called Mor Paradox right things that we take for granted.
Because humans and animals can do it we think it's not complicated it's actually very complicated.
And the stuff that we think is uniquely human like manipulating and generating language playing chess playing go playing poker.
Producing.
Poetry and this kind of stuff turn that to be easy.
Relatively.
Okay and perhaps the reason for this is this very simple.
Calculation.
Um a typical llm nowadays is trained on on the order of 30 trillion tokens.
Three 10 to the 13.
Uh tokens.
That's two to the 13 words roughly.
Each token is about three bytes.
Um so the data volume is roughly.
10 to the 14 bytes.
Uh it would take any of us uh almost half a million years to read through all that material it's basically all.
The publicly available text on the.
Internet.
Now consider her human child.
A four-year-old.
Has been awake a total of 16,000.
Hours.
Which by the way is only 30 minutes of YouTube.
Uploads.
Um we have 2 million optical nerve fibers Each of which carries about 1 B per second maybe a bit less but.
It doesn't matter.
So the data volume is about 10 to the 14 in four years.
A four-year-old child has seen as much.
Data.
As the biggest llm in the form of visual perception.
And for blind children is touch it's the same kind of.
Bandwidth.
Uh that tells you.
A number of things.
We're never going to get to human level intelligence by just turning on text it's not just not.
Happening.
Despite what you know some people who are have a vested interest in this happening are telling us we're going to reach.
You know PhD level intelligence by next year it's just not happening.
We might have PhD level in some subfield.
In some area some.
Problems like chess playing you know but more of them.
Um as long as.
We train those systems specifically for for those problems as um as Bernard was explaining with the visual Illusions.
Um there are a lot of problems of this type when you formulate a problem you pose a problem to an llm.
And if the problem is kind of a standard puzzle the answer will be regurgitated in just a few seconds if you.
Change the statement of the problem a little bit the system will still produce the same answer that it had before because.
It has no real mental model what goes on.
Um in the in the puzzle.
So how do um.
Humans.
Infants.
Learn how the world works.
And you know infants accumulate a huge amount of background knowledge about the world in the first few months of life.
Notions like object permanence.
Solidity.
Rigidity.
Natural categories of objects.
Before children understand language they do understand the difference between the table and the chair.
Um that kind of develops.
Naturally.
And they.
Understand intuitive physics notion like gravity inertia and things of that type around the age of nine.
Months.
Um so it takes a long time uh observation mostly.
Um until four months because babies don't really have any influence on the on the world before that.
But then uh through interactions.
But the amount of interaction that's that's required is astonishingly.
Small.
If we want.
AI system that can reach eventually reach human level might take a while.
Um we call this Advanced machine intelligence at meta we don't like the term AGI artificial general intelligence the reason being that.
That human intelligence is actually quite specialized.
And so calling it AGI is kind of a misnomer.
Um so we call this Ami we actually pronounce it Ami which means friend in French.
Um so we need systems that um learn well models from sensory input basically mental models of how the world works that.
You can manipulate in your mind learning 2D physics.
Um from video let's say systems that have persistent memory systems that can plan.
Actions.
Uh possibly.
Hierarchically.
So as to fulfill an objective and systems that can.
Reason.
Um and then systems that are controllable and safe.
By Design.
Not by fine-tuning which is the the case for llms.
Now the only way I know to build systems of this type is to change the type of of inference.
Um that um current.
AI systems perform so right now the way an llm.
Uh performs inference is by running through a fixed number of layers.
Of anet a transformer then producing a token injecting that token on the input and then running through a fixed number of.
Layers again.
And the problem with this is that.
If you ask a simple question or complex question and you ask the system to answer by yes or no.
Like does 2 and two equal four.
Yes or no.
Or does p equal NP yes or no it's going to spend the exact same amount of computation to answer those two.
Questions so people have been kind of cheating.
And telling the system system will explain.
You know the Chain of Thought trick you you basically have the system produce more tokens so that is going to spend.
More competition answering the question but that's kind of a hack.
The way um a lot of inference.
In statistics for example that's going to make Mike happy actually um the way inference works is is not that way in.
Uh In classical AI in statistics.
Uh in structure prediction a lot of different domains.
The way it works is that you have a function that measures the degree of compatibility or incompatibility.
Between your observation.
And a proposed output.
And then the inference process consist in finding.
The value of an output.
That minimizes this incompatibility.
Measure okay let's call it an energy function so you have an energy function.
Okay represented by the square box here on the right.
Um when it doesn't.
Disappear.
And the system just do performs optimization.
For doing inference now if the inference.
Uh problem is more difficult the system will just spend more time performing inference in other words they will think about complex.
Problems.
For longer than simple ones for which the answer is pretty.
Obvious.
Um and this is really a very classical thing to do in classical classical AI is all about reasoning and uh search.
Therefore optimization.
Pretty much any.
Computational.
Problem can be reduce an optimization problem essentially or search problem.
Uh it's also very classical in.
Probabilistic.
Modeling like probabilistic graphical models and things of that type.
So this type of inference would be more akin to what psychologists.
Call system two in uh sort of human.
U mind if you want system two is.
When you think about what action.
Or sequence of actions you're going to take before you you you take them you think about something before doing it and.
The system one is when you can do the thing without thinking about it you know it becomes sort of subconscious.
So llms are system one what I'm proposing is system two.
Um and then the appropriate.
Um sort of.
Semi theoretical framework to.
Um explain this is energy based models which I'm not going to have time to get into too much detail but basically.
You capture the dependency between variables let's say observations X and.
Uh outputs.
Uh y through an energy function that takes low value where when X and Y are compatible and then larger values when.
X and why are not compatible.
You don't want to just compute y from X as we just saw you just want an energy function that measures the.
Degree of incompatibility.
And then you know given an X find a y that has low energy for that.
X.
Okay so now let's.
Go a little bit into the details of how this type of architecture can be built.
So essentially.
And how it kind of relates to um.
Thinking or planning.
Uh so a system would look like this um you you get observation from the world it go through a perception module.
That produces an estimate about the state of the world but of course.
The state of the world is not completely observable so you may have to combine this with a memory the content of.
A memory that constit you know contains your idea of the state of the world you don't uh currently.
Perceive.
And the combination of those two goes into.
A world model so what is a world model World model is given given a current estimate of the state of the.
World.
Which is in an abstract representation space.
And given an action sequence that you imagine.
Taking.
Uh your world model predicts.
The the resulting state of the world that will.
Um occur after you take that sequence of actions.
Okay that's what a world model is if I tell you imagine a cube floating in the air in front of you.
Okay now rotate this Cube by 90 degrees around a vertical axis.
Um what does it look like it's very easy for you to kind of have this metal model of a cube rotating.
All.
Right.
Let's hope this will.
Be more stable.
50 Herz not 60.
HZ.
Okay so uh what you can do now.
Is.
Uh feed.
Okay hang.
Make a more radical Choice here.
Okay this doesn't look like it was a good.
Idea.
Oh I know what I.
That's.
Nice.
Okay I think we're going to have human level intelligence before we have audio video that actually.
Works.
Okay um so so if we have this world model which is able to predict the.
Result of a sequence of actions.
We can feed it to an objective which is a task objective that measure to what extent the predicted final State.
U satisfies a goal that we set for ourselves.
It's just a cost function.
Um and we also can set some uh guardrail.
Objectives.
Think of them as constraints that need to be satisfied.
For.
The system to behave in a safe manner right so those guardes will be explicitly.
Implemented.
And the way the system proceeds is by optimization it's looking for an action sequence that minimizes.
The task objective.
The uh guard rail objectives at runtime.
Okay we're not talking about learning here we're just talking about.
Inference.
Um and that will guarantee the safety of the system because uh the guard rails guarantee safety and there is no way.
You can Jailbreak that system by giving it a prompt that will you know have it ES Escape its guardwire objectives the.
Guard objectives would be just.
Hardwired.
They might be trained.
But hardwired.
Now.
A sequence of actions should probably use.
A single World model that you repeat you use repeatedly over multiple time steps.
Okay so you have a one model if you did the first action it predicts the next state and the second action.
Predicts the second next state you can have guard R cost and objective.
Uh task uh.
Task objectives along the trajectory.
The ad specifying what optimization algorithm we can use it doesn't really matter for the discussion that we have.
Um if the world happens not to be completely deterministic and predictable.
The world model may need to have latent variables to account for all the things about the world that we do not.
Observe.
And that uh you know makes our prediction basically.
Um inexact.
And ultimately what we want is a system that can plan hierarchically so something that may have several levels of abstraction.
In such a way that.
At the low level we plan low level actions.
Like basically muscle control but at a high level we can plan abstract.
Macro action.
Where the world model predicts at longer time steps but in a representation space that is more abstract.
And therefore contains fewer detail so if I want if I'm sitting at my office at NYU and I decide to go.
To Paris.
Um I can decompose that task into two sub tasks go to the airport and catch a plane.
Okay now I have a sub goal going to the airport.
Um I'm in New York city so going to the airport consist in going down on the street and haing a taxi.
How do I go down in the street.
Well I need to uh get to the elevator push the button go down go out the building how do I go.
To the elevator well I need to stand up for my chair pick up my bag open the door walk to the.
Elevator.
Avoid all the obstacles and then at some point I get to a level where I don't need to plan I can.
Just take the actions.
Um but we do those type of this type of hierarchical planning absolutely all the time and I tell you we have.
No idea how to do this with learning.
Machines.
Almost every robot does hierarchical planning but the.
The representations at every level of the hierarchy are hand.
Handcrafted.
What we need is to train an architecture perhaps of the type that I'm describing here so that it can learn.
Repres abstract representations.
Not just of the state of the world.
But also prediction World models that predict what's going to happen but also abstract.
At levels of abstraction so we can do this hierarchical planning.
Animals do this.
Humans do this very well.
We're completely incapable of doing this withm today.
If you're starting a PhD.
Great topic.
Might take more than three.
Years.
I with all those Reflections about 3 years ago I wrote a long paper where I kind of explained sort of where.
Where I think AI research should be focusing on so this so before the whole GP CH GPT craze.
Um I haven't changed my mind about this CH GPT hasn't Chang anything we wereing Els before that so we knew what.
Was coming anyway.
Um this is the paper um a path towards autonomous machine intelligence that we now call Advanced machine intelligence because autonomous just.
Scares people.
Um and it's on open review it's not on archive.
And there's various versions of this talk that I've I've given various ways okay so very natural idea for for getting systems.
To understand how the world.
Is um using the same.
Process that we used to.
To to train system for natural language and apply this to let's say video okay if a system is capable.
Of predicting what's going to happen in a video you show it A short segment of video and you ask it to.
Predict what's going to happen next presumably it would have.
Understood the underlying.
Structure of the world.
Um and so training it to make that prediction might actually cause the system to understand the annoing structure of the.
It works for text.
Because predicting words is relatively simple.
Why is predicting words simple because words um there's only a finite number of possible words certainly a finite number of possible.
Tokens.
And so we can't predict exactly which word will follow another word.
Or what what word is missing in the text but we can produce a probability distribution or score for every possible word.
In the dictionary.
We cannot do this for.
Images for video frames we do not have good ways of representing distributions of our video frames.
Um every attempt to do this uh.
Basically bumps into mathematical intract.
Abilities.
Um and so.
You could try to get around the problem using you know um statistics and and the math that was invented by by.
Physicists you know vial.
Inference and all that stuff but in fact it's better to just throw away the entire idea of doing probabilistic modeling and.
Just just say I just want to learn this energy function that tells me whether.
My output is compatible with my input and I don't care if this energy function is a negative log of some.
Distribution.
Um and so the reason we need to do this of course is because we cannot predict exactly what's going to happen.
In the world there is a whole set of possible things that may happen and if we train a system to just.
Predict one frame it's not going to do a good job.
Um so the solution to that problem is an AR a new architecture I call John tedding predictive architecture or.
Jepa.
And that's because generative architecture.
Simply do not work.
For producing videos.
You may have seen video generation systems.
That produce pretty amazing stuff there's a lot of hacks that go be Beyond them uh behind them and they don't really.
Understand.
Physics.
Um they don't need to they just need to to predict pretty pictures they don't need to actually have kind of accurate.
Model of the world okay so here's what the JEA is the idea is that you run both the observation.
And the output.
Which is the next observation.
Into an encoder.
So that the prediction.
Does not consist in predicting pixels.
But basically predicting an abstract.
Representations.
Of what goes on in the video.
Video or anything okay so let's compare those two.
Architectures.
On the left you have generative.
You run X the observation to an encoder and perhaps to a predictor or decoder.
And you make a prediction for y.
Okay that straightforward.
Prediction.
And then on the right this jeta architecture you run both X and Y through and codos which may be identical or.
Different.
And then you predict the representation of Y from the representation of X in this abstract space.
What this will cause the system to basically learn.
An encoder that eliminates all the stuff you cannot.
Predict.
And this is really what we do there's no way that you know if if I observe the left part of this.
Room here and I kind of pan the camera towards the right.
There's no way any video prediction system including humans can predict what every one of you looks like or predict the texture.
On the wall or the texture of the wood.
U on the on the hardwood floor.
Um there's a lot of things that we just simply cannot predict.
And so instead of insisting that we should.
Make a probabilistic prediction about stuff that we cannot predict let's just not predict it.
Learn a representation in which all of those details are essentially eliminated so that the prediction is much simpler it may still.
We need to be.
Non- deterministic but at least we simplify the problem so there's various flavors of those jads which I'm not going to go.
Into some of which have latent variables some of which have are action conditioned so I'm going to talk about the action.
Condition because that's uh the the most interesting one because they really are World models right so you have an encoder X.
Is current state of the world or current observation.
XX is current state of the world you feel an action to a predictor which you imagine taking and the predictor which.
Is a world model predicts the representation of the next state of the world.
And that's how you can do planning.
Okay so um you need to we need to train those systems and we need to figure out how to train those.
Jepa architectures and tells that to not be completely trivial.
Because.
You need to train the the cost function in this JEA architecture that measures the.
The Divergence.
Between the representation of Y and the predicted representation of Y we need this to be low.
On the training data but we need also needed to be large outside the training set.
Okay so this is you know this kind of energy function here that has kind of uh Contours of equal.
Equal energy.
We need to make sure the energy is high outside of the manifold of data.
And I only know two classes of methods for this one set of method is called contrastive.
It consists in um having uh data points which are those those blue dark blue dots.
Pushing the down the energy of those and then generating you know those flashing green dots and then pushing the energy up.
The problem with this type of method Contra method is that they don't scale very well in high dimension.
If you have too many dimensions in your space of Y you're going to need to push up in lots of different.
Places.
And um it it doesn't.
Work so well you need a lot of contrastive samples for this to work there's another set of method that um called.
Regularized method and what they do is.
They use a regularizer on the energy so as to minimize the volume.
Of space that can take low energy.
Okay so.
That leads to two different.
Types of learning procedure one one learning procedure which is contrastive you need to generate those contrastive points and then push their.
Energy up to some loss function.
And the other one is some regularizer that is going to sort of shrink wrap the the manifold of data.
Um so as to make sure that the energy is Tire outside.
So there's a number of techniques to do this um I'll describe just just a handful and the way um we we.
Started testing them several years ago.
Maybe.
Five six years ago.
Was.
To train them to learn.
Representations of images so you take one image.
You corrupt it or transform it in some ways and you run the original image and the corrupted version in identical encoders.
And you train a predictor to predict the representation of the original image from the corrupted.
One.
Once you're done training the system you remove the predictor and you use a representation at the output of the encoder as.
Input to a simple.
Um like a linear classifier or something of that type that you train supervised.
Uh so as to verify that the representations that are learned are good and this idea is very old it goes back.
To the 198 90s and things like uh we used to call SES.
Networks.
Um and some more recent work on on those joint embedding architectures and then adding the predictor is more is more.
Recent.
So s clear which is from from Google is a contrastive method derived from s.
Nets.
Um but again the dimension is is restricted so the regularized method.
Uh worked the following way you try to estimate.
Have some sort of estimate of the information content coming out of the encoders.
And what you need to do is prevent.
The encoder from collapsing this a trivial solution of training a a Jeeter architecture.
Where the encoder basically ignores the input produces a constant output and another the prodction error is zero all the time okay.
And obviously that's a collapsed solution that is uh not interesting so you need a system you need to prevent the system.
From collapsing.
Which is the regularization method I was talking about earlier and an indirect way of doing this is.
Maintain the information content coming out.
Of the encoder.
Okay so so you're going to have a training objective function which is a negative.
Information content if you want because we minimize in machine learning we don't maximize.
Uh one way to do this.
Is to basically take the.
Vectors representation vectors that come out of the encoder over a batch of.
Samples.
Um and make sure they contain information how you can you do this you can take that.
Matrix of representation vectors and compute the product of that matrix.
By its transposed you get aarian.
Matrix.
And you try to make that coari Matrix equal to.
Identity.
There's a bad news with this which is that this.
Basically.
Approximates the information content.
By making very strong assumptions about the the nature of the dependencies between the variables.
And in fact it's an upper bound on information content and we're pushing it up crossing our fingers that the actual information.
Contain which is below is going to follow.
Okay so it's slightly.
Irregular.
Uh theoretically but but it works.
All right so again.
Uh you have a matrix coming out of your encoder.
It's got a number of samples.
Um and each Vector is a separate variable what we're going to try to do is going to try to make each.
Variable.
Individually.
Informative so we're going to try to prevent the the variance of the variable from going to to zero force it to.
Be one for example.
And then we're going to decorrelate the variables.
With each other and that means Computing The coverance Matrix of this Matrix is transpose.
Multiply by itself and then try to make the resulting coar Matrix as close to the identity.
Uh Matrix as possible.
There are other methods that try to.
Make the samples.
Uh orthogonal not the not the variables.
Um and those are contrasting sample contrasting methods.
Um but they don't work in high dimension and they require large batches.
Uh so we have um a method of this type called viag.
That means variance in variance Co variance regularization.
And it's got particular loss functions for this ciance Matrix.
Um there been kind of similar methods proposed by.
Uh yima and his team called MCR squar and then another method by.
Uh some colleagues from NYU.
Called.
Mmcr.
From neuroscience.
So that's one set of methods and I really like those methods and I I think and they work really well I.
Expect to see more of them in the future.
But there is another set of method that to some extent has been slightly more successful over the last.
Couple years and those are based on distillation so again you have two encoders it's still a joint Ting productive architecture you.
Have two encoders.
They kind of share the same weights but not really so the encoder on the right.
Uh gets a version of the weights of the enod on the left that are obtained through a.
Exponential moving.
Average.
Okay a moving average so basically you force the encoder.
On the right.
To.
Uh change its weights more slowly than the one on the left and for some reason that prevents collapse.
There's some theoretical work on this um in fact uh this one that jum just finished writing.
Um but it's a little bit mysterious why this works and frankly I'm a little uncomfortable with this method but we have.
Um accept the fact that actually works.
Um if you if you're careful.
You know real Engineers.
Buildings.
Without necessarily knowing why they work.
That's good.
Engineers.
And then the usual joke in France that everybody here should should learn.
Is that students that come out of e poly technique.
When they build something it doesn't work but they can tell you.
Why.
Sorry about that.
I didn't study here you can tell um.
Okay let me uh switch ahead.
Skip ahead a little bit in interest of time because we wasted a bit of time.
Okay so there's a particular way of implementing this AIO distillation called IA there's another one called called Dino.
Or Dino.
Uh which I I skipped a little bit um and um so Dino.
Um is V2 people are working on on V3 this is a method produced by some some of my colleagues at at.
Fair Paris.
Um team led by Max Maximo cab.
Um and then a slight different version um called IA V.
JEA.
By also Fair.
People in in Montreal and Paris mostly.
So no need for negative samples there and those those kind of those systems learn generic features that you can then learn.
For any Downstream task and the features are really good.
Um so this works really well I'm not going to bore you with details because I don't have time uh more recently.
We worked on a version of this for video so this is a system that takes a a chunk of 16 frames.
From video.
And you corrupt you you take those 16 frames run them to an encoder and then you corrupt those 16 frames by.
Masking some parts of it run them to the same encoder and then train a predictor to predict.
The U representation of a full video from the one that is partially masked or corrupted.
And the U.
So again this.
Group of researchers at at Fair in Paris and Montreal.
Um and this works really well in the sense that uh you learn features that you can then feed to A system.
That can classify actions in videos and you get really good results with the with this these these methods again I'm not.
Going to bore you with details but here is a really interesting thing this is a paper that we just.
Submitted.
Um if you show that system.
Videos where.
Something really strange.
Happens.
That system actually is capable of telling you my prediction error is going through the roof there is something strange going on.
In that window so you you take a you take a video and you take the 16 video Frame Window you slide.
It over the video and you measure the prediction error.
Of the system.
And if something really strange happen like an object spontaneously disappears or change.
Shape.
Um the prediction error shoots up.
So what that tells you is that that system despite its Simplicity has learned some level of Common Sense he can tell.
You if something really strange in the world is happening.
Lots of experiments to show this in various contexts for various types of intuitive physics but I'm not going to I'm to.
Skip to this uh latest work uh D Dino.
World model.
Um so this is using Dino features and then training a predictor on top of it which is action condition so that.
It's a world model that we can use for planning.
Um and this is a a paper that is on archive there's a website also that you can uh you can look.
At the URL is at the top here.
So basically.
Uh train a predictor.
Using.
You know a picture of the world that you run through a dino.
Encoder.
And then.
An action that maybe a robot.
Um takes so you get the next.
Frame.
Uh of that of that video.
Next image from the world run this to the dino encoder and then train your predictor to just predict what's going to.
Happen given the action that was taken.
Okay very simple to do planning.
Um You observe an initial state run into the doo encoder.
Then run your world model multiple time steps with imagined.
Then you have a Target state which is represented by a Target image for example you run it to the encoder and.
Then you compute the distance in.
State space between the predicted State and the the the state representing the the target.
Image.
And the planning consists in just through optimization.
Finding a sequence of actions that minimizes that cost at runtime okay reference time.
You know people are excited about.
Um you know test time computation and blah blah blah as if it was something new.
This is completely classical in optimal control this is called Model preductive control it's been around with us.
About the same time that I've been around.
Um the first paper is.
You know planning using using models of this type using optimization are from the early 60s.
Um the the ones that actually learned the model are more recent they're more from the 70s from France.
Actually.
It's called.
Edcom.
Um some people in optimal control might know about this.
But you know it's very simple concept this works amazingly well so let me skip to the video.
Okay so let's say you have this uh Little T shape.
And you want to push it into a particular.
Um position.
And so you know which position it has to go to because you put an image of that position run to the.
Enod and that gives you a Target state in representation.
Space.
Let me play that video.
Again.
Okay so at the top you see what actually happens in the real world when you take a sequence of actions that.
Is planned and what you see at the bottom is the internal mental prediction.
Of what the system of the sequence of actions the system was planning.
And this is run to a decoder that produces a pictorial representation of the internal state but that is trained separately there's.
No image generation.
Um let me skip to the more interesting one so here is one where you have an initial state which is a.
Bunch of Blue Chips.
Randomly.
Thrown.
On the floor.
And the target state is at the top and what you see here are.
The actions that are resulted from planning.
And the robot like accomplishing those actions the Dynamics of this environment is actually fairly complicated because those blue Chiefs kind of.
Interact with each other and and everything.
Um the system has just learned this through you know observing.
A bunch of uh uh State action next state.
Um and this works in a lot of situations for you know arms and moving through mazes and pushing a te around.
And and things like that so.
Okay and I'm not sure where I came.
Back.
Um we've applied kind of similar idea to navigation but interest of time I'm just going to.
Skip.
Um so this is you know basically sequences the videos where.
A frame is uh is taken at one time and then the robot moves and you know through odometry you know by.
How much the robot has moved you get the next frame and so you just train a system to predict what the.
World is going to look like if you take a particular motion.
Uh action and what you can do next is you can tell a system like you know navigate to that point.
Um and it.
It will it will do it and you know avoid obstacles.
On the way this is a very new.
Work.
But let me go to the conclusion so I'm having a number of uh.
Recommendations abandon generative models.
The most popular method today that everybody is working on startop working on this you work on jads those are not generative.
Models they predict in representation.
Probably seek models because it's.
Intractable.
Use energy based.
Models.
M have had like a 20.
Year.
Contentious discussion about this um abandon contractive methods.
In favor of those regularized methods abandon reinforcement learning but that I've been saying for a long time we know it's inefficient.
You have to use reinforcement learning really as a last result when your model is inaccurate or or your cost function is.
Inaccurate.
Um but if you are interested in human level AI just don't work on llm there's no point.
I mean in fact if you are in Academia don't work on LM.
Because you're in competition with like hundreds of people.
With tens of thousands of gpus like there's nothing you can bring to the table.
Do something else.
Um there's a number of problems to solve U training those things with you know large scale data blah blah blah planning.
Algorithms are kind of inefficient we have to come up with better methods.
So if you are like into optimization applied math it's great.
Um J with latent variables planning under uncertainty.
Hierarchical planning which is completely unsolved.
Um learning cost module because probably most of them you can't build by hand you need to learn them and then there.
Is issues exploration Etc okay so in the future we'll have.
Universal virtual assistants.
They'll be with us at all times they will mediate all our interaction with the digital world.
We cannot afford to have those systems come from a handful of companies.
From the west coast of the US or China.
Uh which means the platforms on top of which we build those systems need to be open source and widely available.
They are expensive to train.
But once you have a foundation model fun tuning it for a particular application is relatively cheap and a lot of people.
Afford to do this so the platforms need to be shared they need to speak all the the world languages.
Understand all the world's cultures.
All the value systems all the centers of Interest.
No single entity in the world can train a foundational model of this type this probably will have to be done in.
A collaborative fashion or distributed fashion again some work for Applied mathematicians who are interested in distributed.
Algorithms for large scale.
Optimization.
Um and so open source AI platforms are necessary.
The danger I see.
Um in Europe and in other places is that.
Geopolitical.
Rivalry.
Will.
U entice governments.
To basically make the release of Open Source model illegal.
Because there are under the impression that a country will stay ahead if he keeps uh its science secret that's that would.
Be a huge mistake.
When you do research in secret you fall behind that's.
Inevitable.
What will happen is that the rest of the world we go up and and will overtake you that's currently what's what's.
The open source models are.
Overtaking.
Uh slowly but surely.
Uh proprietary.
Thank you very much.