Posts

I gave two talks on my recent papers on adaptive optimizers, one for Google Brain a couple of weeks ago and another for the MIT Operations Research Center today. The slides of my presentation can be accessed using this link.

Today I’m giving an online talk at AIDRC Seminar Series of Technology Innovation Institute. The talk announcement can be found on the seminar’s website along with the abstract. In short, The topic of my presentation is our 2022 ICML paper ProxSkip and its several extension that were done by other authors.

My paper on Regularized Newton got accepted for publication at SIAM Journal on Optimization (SIOPT). The main result of this work is to show that one can globalize Newton’s method by using regularization proportional to the square root of the gradient norm. The corresponding method achieves global acceleration over gradient descent and it converges with the $O(1/k^2)$ rate of cubic Newton.

I’m taking part in the 2022 Workshop on Federated Learning and Analytics on 9 and 10 November. I am giving a talk about our work on Asynchronous SGD in the mini-workshop on Federated Systems at Scale on 9 November.

I’m giving a talk at Séminaire Parisien d’Optimisation on 10 October. I will be presenting my work on second-order optimization, including the Super-Universal Newton paper.

This week I’m participating in the Learning and Optimization in Luminy (LOL2022) workshop. Yesterday I gave a short presentation of our Super-Universal Newton paper.

We recently released two new papers that are available on arxiv as preprints. The first work considers the global convergence of regularized Newton method (short summary in this twitter thread) and the second work is about adaptive stepsizes for stochastic gradient descent.