- 24
- 84 952
cwkx
United Kingdom
Приєднався 9 січ 2011
Chris G. Willcocks
Associate Professor
Computer Science, Durham University
Website: cwkx.com
Twitter: @cwkx
Associate Professor
Computer Science, Durham University
Website: cwkx.com
Twitter: @cwkx
Deep Learning 2: Mathematical principles and backpropagation
Slides: cwkx.github.io/data/teaching/dl-and-rl/dl-lecture2.pdf
Colab: colab.research.google.com/gist/cwkx/dfa207c8ceed5999bdad1ec6f637dd47/distributions.ipynb
Twitter: cwkx
Next video: ua-cam.com/play/PLMsTLcO6etti_SObSLvk9ZNvoS_0yia57.html
Foundational Statistics
- probability density function
- joint probability density function
- marginal and conditional probability
- expected values
Foundational calculus
- derivative of a function
- rules of differentiation
- partial derivative of a function
- rules of partial differentiation
- the Jacobian matrix
Mathematics of neural networks
- neural network functions
- computational graphs
- reverse mode of differentiation
#statistics #calculus #probability #deeplearning #jointprobability #marginal #conditional #derivatives #partialderivative #jacobian #neuralnetworks #computationalgraphs
Colab: colab.research.google.com/gist/cwkx/dfa207c8ceed5999bdad1ec6f637dd47/distributions.ipynb
Twitter: cwkx
Next video: ua-cam.com/play/PLMsTLcO6etti_SObSLvk9ZNvoS_0yia57.html
Foundational Statistics
- probability density function
- joint probability density function
- marginal and conditional probability
- expected values
Foundational calculus
- derivative of a function
- rules of differentiation
- partial derivative of a function
- rules of partial differentiation
- the Jacobian matrix
Mathematics of neural networks
- neural network functions
- computational graphs
- reverse mode of differentiation
#statistics #calculus #probability #deeplearning #jointprobability #marginal #conditional #derivatives #partialderivative #jacobian #neuralnetworks #computationalgraphs
Переглядів: 2 808
Відео
Reinforcement Learning 9: Model-based methods
Переглядів 2,5 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture9.pdf Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Model-based RL - taxonomy - overview - the simulation cycle - characteristics Integrated learning and planning - Dyna-Q - characteristics - Monte Carlo tree search - simulated policy learning #reinforcementlearning #modelbased #MCTS #planni...
Reinforcement Learning 10: Extended methods
Переглядів 8873 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture10.pdf Atari: ua-cam.com/play/PL34t13IwtOXUNliyyJtoamekLAbqhB9Il.html Twitter: cwkx Playlist: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Distributed and recurrent RL - DQN characteristics - recurrent replay in distributed RL - R2D2 performance Exploration vs exploitation - approaches Intrinsic rewards - NGU: intri...
Reinforcement Learning 8: Policy gradient methods
Переглядів 1,4 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture8.pdf Code: github.com/higgsfield/RL-Adventure-2 Theory: lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Policy-based methods - definition - characteristics - deterministic vs stochastic policies Policy gradients - gradien...
Reinforcement Learning 6: Temporal-difference methods
Переглядів 4,4 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture6.pdf Colab: colab.research.google.com/gist/cwkx/54e2e6d59918a083e47f19404fe275b4/temporal-difference-learning.ipynb Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Temporal-difference learning - dopamine and reward predictor error - definition - behaviour example SARSA (on-policy TD control) ...
Reinforcement Learning 4: Dynamic programming
Переглядів 8 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture4.pdf Colab: colab.research.google.com/gist/cwkx/670c8d44a9a342355a4a883c498dbc9d/dynamic-programming.ipynb Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Introduction - definition - examples - planning in an MDP Policy evaluation - definition - synchronous algorithm Policy iteration - policy...
Reinforcement Learning 7: Function approximation
Переглядів 3,6 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture7.pdf Code: github.com/higgsfield/RL-Adventure Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Function approximation - introduction - definition - challenges Incremental methods - SGD for prediction - SGD for control - convergence Batch learning - experience replay - model freezing with doubl...
Reinforcement Learning 5: Monte Carlo methods
Переглядів 4 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture5.pdf Colab: colab.research.google.com/gist/cwkx/a5129e8888562d1b4ecb0da611c58ce8/monte-carlo-methods.ipynb Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Introduction - history of Monte Carlo methods - definition Monte Carlo prediction - overview - definition - incremental means - prediction...
Reinforcement Learning 3: OpenAI gym
Переглядів 8 тис.3 роки тому
Guest lecture by Adam Leach Colab: colab.research.google.com/gist/qazwsxal/6cc1c5cf16a23ae6ea8d5c369828fa80/gym-demo.ipynb The last 20mins of this video can be skipped by most watchers (contains specifics of how to use one of Durham's GPU servers. Feel free to watch if interested, otherwise just use Colab) Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Content: OpenAI gym C...
Reinforcement Learning 2: Markov Decision Processes
Переглядів 8 тис.3 роки тому
This lecture uses the excellent MDP example from David Silver. Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture2.pdf Colab: colab.research.google.com/gist/cwkx/ba6c44031137575d2445901ee90454da/mrp.ipynb Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Content: Markov Chains - markov property - state transition matrix - definition and example...
Reinforcement Learning 1: Foundations
Переглядів 6 тис.3 роки тому
This is based on David Silver's course but targeting younger students within a shorter 50min format (missing the advanced derivations) more examples and Colab code. Slides: cwkx.github.io/data/teaching/dl-and-rl/rl-lecture1.pdf Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6ettgmyLVrcPvFLYi2Rs-R4JOE.html Introduction - definition - examples - comparison A Brief History - learnin...
Deep Learning 10: Meta learning and manifold learning
Переглядів 8973 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/dl-lecture10.pdf Twitter: cwkx Playlist: ua-cam.com/play/PLMsTLcO6etti_SObSLvk9ZNvoS_0yia57.html Manifold learning - NLDR with DNNs - t-SNE and UMAP on DNNs - designing tailored embeddings - Jonker-Volgenant assignment Meta learning - thinking in distributions - the distribution of all data... - ...and of all tasks - definition - the me...
Deep Learning 9: Flow models and implicit networks
Переглядів 9183 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/dl-lecture9.pdf GON: cwkx.github.io/data/GON/ SIREN: vsitzmann.github.io/siren/ Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6etti_SObSLvk9ZNvoS_0yia57.html Flow models - definition - the determinant - the change of variables theorem Normalising flows - definition - triangular Jacobians - normalising flow layers Implicit representa...
Deep Learning 7: Energy-based models
Переглядів 8 тис.3 роки тому
Slides: cwkx.github.io/data/teaching/dl-and-rl/dl-lecture7.pdf Colab: colab.research.google.com/gist/cwkx/6b2d802e804e908a3ee3d58c1e0e73be/dbm.ipynb Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6etti_SObSLvk9ZNvoS_0yia57.html Manifolds Energy-based models - definition - GANs as energy-based models - clustering as an energy-based model - softmax and softmin - exact likelihood Co...
Deep Learning 3: PyTorch programming (coding session)
Переглядів 2 тис.3 роки тому
This video has minor problems with audio but fixes itself later Colab 1: colab.research.google.com/gist/cwkx/441e508d3b904413fd3950a09a1d3bd6/classifier.ipynb Colab 2: colab.research.google.com/gist/cwkx/3a6eba039aa9f68d0b9d37a02216d385/convnet.ipynb Twitter: cwkx Next video: ua-cam.com/play/PLMsTLcO6etti_SObSLvk9ZNvoS_0yia57.html #PyTorch #programming #livecoding #deeplearning #vis...
Deep Learning 4: Designing Models to Generalise
Переглядів 1,3 тис.3 роки тому
Deep Learning 4: Designing Models to Generalise
Interactive GPU active contours for segmenting inhomogeneous objects
Переглядів 4606 років тому
Interactive GPU active contours for segmenting inhomogeneous objects
ABI: Automatic 3D Billboard Imposters
Переглядів 9 тис.10 років тому
ABI: Automatic 3D Billboard Imposters
Great! Thanks
Interesting example on 14:00. Watched your 2 series. Ty very much!
looks like the colab code is outdated step # setup the environment, and record a video every 50 episodes gives: AttributeError: module 'gym.wrappers' has no attribute 'Monitor' also a lot of deprecation warnings
Thank you for the classes! Incredible so far!
This is seriously very underrated, the best
You extract just the key things needed in sutton's book. I follow his book while following your videos and bunch others as well 😁 thank you
Why is the action q* going from class3 to the pub 9.4?
I have watched several lectures about RL and you have been the best one
same here.
saaame
i am the 2k th person added in your subscriber list
Thanks. Was good to hear there is lack of consistency in the notation used in papers as that was confusing!
thank you prof, for this elegant explanation....
Incomplete list of timestamps: 0:00 - Introduction 3:01 - Installing Conda 13:25 - Using Torch (tensors) 23:00 - Using Torch (data parsing) 32:15 - Using backprop 52:00 (ish) - Using convolution
The best lectures on RL
it was perfect thank you so much
thank you so much for not assuming students knowing everything 😭. You explain it so clear!
Chapter 12 Sutton and Barto. This is what I was looking for. Thanks for beautiful the explanation!
Thank you so much ! rich content for free.
Good Explanation, thank you so much
Possible for you to share the latex template for the presentation?
riight
thanx
How do we conclude that ->success: {stats/episode} would always be monotonically increasing? And how does it indicate convergence with value as 0.54?
Hi I can't remember what I said here, it was over 2 years ago - we go into depth on the convergence properties for this in our practicals, which are on my github: github.com/cwkx/materials/raw/main/reinforcement-learning/rl-answers4.pdf and github.com/cwkx/materials/blob/main/reinforcement-learning/rl-answers5.pdf etc
Thanks for a well explained topic.
Thank you for doing these videos... I found them really very helpful
Amazing lecture, thanks for uploading
I really recommend this video! Just perfect! I wish POMDP was also well explained in the same way!
sassy 💅
good lecture
Great job !
Very very useful and informative thank you very much!
35:02 implicit networks
You started not knowing what you are talking about.
As a newcomer who has already learned pytoch, this class has allowed me to consolidate the actual combat practice of torch and understand the magic of vision. In addition, I would like to ask Willcocksr: How should we learn more about using pytoch to build our own network?
could you speak louder in your next video
On Langevin Dynamics in Machine Learning - Michael I. Jordan (Video Link): ua-cam.com/video/QTnjqdxG99c/v-deo.html&ab_channel=InstituteforAdvancedStudy
The example of car accident got me
Excellent explanation, well done
Thank you..
By far one of the most thorough and helpful explanations I've encountered! THANKS!
Hi great lectures! i would like to know if you can recommend me more courses and resources to advance in the field
I'd just recommend just getting into a habit of reading the latest papers from the top venues such as ICLR, NeurIPS and CVPR when they've been reviewed, e.g. find a sorted ranked list of the best papers and read the abstracts and Ctrl+F any terms interesting to you - e.g. tanelp.github.io/neurips2021/ and papers.labml.ai/papers/iclr_2022?sort_by=conference_score&dsc=0
Very nice explanation sir.Thank you
It was amazing, this lecture has made my life much easier. thank you.
Your video helped me a lot! Very informative and easy to understand. Thank you!
Amazing lecture, thank you so much!
It was an amazing video, you explained the issues in the best way. Thank you
Recently I followed the teacher’s second class on and off, involving some basic theoretical knowledge related to backpropagation. The main difficulty is still in the formula described in English, but I can feel the improvement of listening. I also manually offline deduced the back-propagation algorithm and implemented it in python, keep going!
This is gem. Great content.
Would you please check the calculation for 0.34? It is 0.3125 in my calculation. Thank you.
0.34375 = 0.25*0.0625 (left) + 0.25*0.0625 (up) + 0.25*0.25 (down) + 0.25*1 (right) - you can see this clearly if you go to the colab notebook in the comment, then in the Policy evaluation section, where it first says "# evaluate this policy ... V = policy_evaluation(env,policy,draw=False)" change draw=True and you'll see all the intermediate steps and you can see 0.34375.
@@cwkx so you use the updated valves in the same episode. I thought they will be used in the next episode.
@@johnnassour Yes there must be mistake in video's calculations. If you look algorithm, the array is updated when all states are gone through. But still very good video series, thanks for that.
Deep learning is indeed developing faster and faster, which requires us to understand it in essence, including biological perspectives, historical perspectives, etc. I hope I can build a more systematic understanding of deep learning through this course. By the way, Teacher Willcocks's English expression is very fluent, and it doesn't sound particularly strenuous.
Hello Dear Sir Interesting Video, Please did it possible to have the code for this video
Hi Mamadou, all code where available are in the video descriptions (Colab links).
@@cwkx thank you
Hi there,big fan! I learned that you also had expertise in security, is there any chance you will consider teaching them in youtube ?
Hi Ali, many thanks for the kind comment - unfortunately I don't think I can get permission for this due to the sensitive nature of some of the security materials/discussions/exploits/real-world stories not covered in the slides.