View this talk on talks.cam (opens in new tab)<\/span><\/a>Opens in a new tab<\/span><\/p>\n Frontiers in Artificial Intelligence<\/span> is a series of public lectures at Microsoft Research Cambridge featuring leading researchers in the field, focusing on the cutting edge topics at the intersection of machine learning, statistics, and artificial intelligence. Students, scientists, and engineers in academia and industry are all welcome to join us for these exciting talks and the opportunity to socialize with the Cambridge AI\/ML community.<\/p>\n Unsupervised learning, in particular learning general nonlinear representations, is one of the deepest problems in machine learning. Estimating latent quantities in a generative model provides a principled framework, and has been successfully used in the linear case, e.g. with independent component analysis (ICA) and sparse coding. However, extending ICA to the nonlinear case has proven to be extremely difficult: A straight-forward extension is unidentifiable, i.e. it is not possible to recover those latent components that actually generated the data. Here, we show that this problem can be solved by using temporal structure. We formulate two generative models in which the data is an arbitrary but invertible nonlinear transformation of time series (components) which are statistically independent of each other. Drawing from the theory of linear ICA, we formulate two distinct classes of temporal structure of the components which enable identification, i.e. recovery of the original independent components. We show that in both cases, the actual learning can be performed by ordinary neural network training where only the input is defined in an unconventional manner, making software implementations trivial. We can rigorously prove that after such training, the units in the last hidden layer will give the original independent components. [With Hiroshi Morioka, published at NIPS2016 and AISTATS2017.]<\/p>\n \u00a0<\/strong>Opens in a new tab<\/span><\/p>\n Arguably, most excitement about deep learning revolves around the performance of convolutional neural networks and their ability to automatically extract useful features from signals. In this talk I will present work from AMLAB where we generalize these convolutions. First we study convolutions on graphs and propose a simple new method to learn embeddings of graphs which are subsequently used for semi-supervised learning and link prediction. We discuss applications to recommender systems and knowledge graphs. Second we propose a new type of convolution on regular grids based on group transformations. This generalizes normal convolutions based on translations to larger groups including the rotation group. Both methods often result in significant improvements relative to the current state of the art.<\/p>\n Joint work with Thomas Kipf, Rianne van den Berg and Taco Cohen.<\/p>\n Visit the event page<\/a><\/p>\n \u00a0<\/strong>Opens in a new tab<\/span><\/p>\n","protected":false},"excerpt":{"rendered":" 21 Station Road Cambridge CB1 2FB View the whole series on talks.cam (opens in new tab) View this talk on talks.cam (opens in new tab)Opens in a new tab Frontiers in Artificial Intelligence is a series of public lectures at Microsoft Research Cambridge featuring leading researchers in the field, focusing on the cutting edge topics […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_startdate":"2017-05-03","msr_enddate":"2017-05-03","msr_location":"Microsoft Research Cambridge UK","msr_expirationdate":"","msr_event_recording_link":"","msr_event_link":"","msr_event_link_redirect":false,"msr_event_time":"13:00 - 14:00","msr_hide_region":true,"msr_private_event":false,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[13556],"msr-region":[239178],"msr-event-type":[197944],"msr-video-type":[],"msr-locale":[268875],"msr-program-audience":[],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-376433","msr-event","type-msr-event","status-publish","hentry","msr-research-area-artificial-intelligence","msr-region-europe","msr-event-type-hosted-by-microsoft","msr-locale-en_us"],"msr_about":"\n\n 21 Station Road View the whole series on talks.cam (opens in new tab)<\/span><\/a><\/p>\n View this talk on talks.cam (opens in new tab)<\/span><\/a>Opens in a new tab<\/span><\/p>\n Frontiers in Artificial Intelligence<\/span> is a series of public lectures at Microsoft Research Cambridge featuring leading researchers in the field, focusing on the cutting edge topics at the intersection of machine learning, statistics, and artificial intelligence. Students, scientists, and engineers in academia and industry are all welcome to join us for these exciting talks and the opportunity to socialize with the Cambridge AI\/ML community.<\/p>\n Unsupervised learning, in particular learning general nonlinear representations, is one of the deepest problems in machine learning. Estimating latent quantities in a generative model provides a principled framework, and has been successfully used in the linear case, e.g. with independent component analysis (ICA) and sparse coding. However, extending ICA to the nonlinear case has proven to be extremely difficult: A straight-forward extension is unidentifiable, i.e. it is not possible to recover those latent components that actually generated the data. Here, we show that this problem can be solved by using temporal structure. We formulate two generative models in which the data is an arbitrary but invertible nonlinear transformation of time series (components) which are statistically independent of each other. Drawing from the theory of linear ICA, we formulate two distinct classes of temporal structure of the components which enable identification, i.e. recovery of the original independent components. We show that in both cases, the actual learning can be performed by ordinary neural network training where only the input is defined in an unconventional manner, making software implementations trivial. We can rigorously prove that after such training, the units in the last hidden layer will give the original independent components. [With Hiroshi Morioka, published at NIPS2016 and AISTATS2017.]<\/p>\n \u00a0<\/strong>Opens in a new tab<\/span><\/p>\n Arguably, most excitement about deep learning revolves around the performance of convolutional neural networks and their ability to automatically extract useful features from signals. In this talk I will present work from AMLAB where we generalize these convolutions. First we study convolutions on graphs and propose a simple new method to learn embeddings of graphs which are subsequently used for semi-supervised learning and link prediction. We discuss applications to recommender systems and knowledge graphs. Second we propose a new type of convolution on regular grids based on group transformations. This generalizes normal convolutions based on translations to larger groups including the rotation group. Both methods often result in significant improvements relative to the current state of the art.<\/p>\n Joint work with Thomas Kipf, Rianne van den Berg and Taco Cohen.<\/p>\n Visit the event page<\/a><\/p>\n \u00a0<\/strong>Opens in a new tab<\/span><\/p>\n","tab-content":[{"id":0,"name":"About","content":"Frontiers in Artificial Intelligence<\/span> is a series of public lectures at Microsoft Research Cambridge featuring leading researchers in the field, focusing on the cutting edge topics at the intersection of machine learning, statistics, and artificial intelligence. Students, scientists, and engineers in academia and industry are all welcome to join us for these exciting talks and the opportunity to socialize with the Cambridge AI\/ML community.\r\n![](\"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2017\/03\/Aapo_web.jpg\")
<\/h3>\nNonlinear ICA using temporal structure: a principled framework for unsupervised deep learning<\/h3>\n
Prof. Aapo Hyvarinen, Gatsby Computational Neuroscience Unit, University College London<\/h4>\n
![](\"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2017\/03\/Max-Welling_web.jpg\")
Generalizing Convolutions for Deep Learning<\/h3>\nProf. Dr. Max Welling, University of Amsterdam<\/h4>\n
\nCambridge
\nCB1 2FB<\/p>\n<\/h3>\nNonlinear ICA using temporal structure: a principled framework for unsupervised deep learning<\/h3>\n
Prof. Aapo Hyvarinen, Gatsby Computational Neuroscience Unit, University College London<\/h4>\n
Generalizing Convolutions for Deep Learning<\/h3>\nProf. Dr. Max Welling, University of Amsterdam<\/h4>\n
<\/h3>\r\nNonlinear ICA using temporal structure: a principled framework for unsupervised deep learning<\/h3>\r\n
Prof. Aapo Hyvarinen, Gatsby Computational Neuroscience Unit, University College London<\/h4>\r\nUnsupervised learning, in particular learning general nonlinear representations, is one of the deepest problems in machine learning. Estimating latent quantities in a generative model provides a principled framework, and has been successfully used in the linear case, e.g. with independent component analysis (ICA) and sparse coding. However, extending ICA to the nonlinear case has proven to be extremely difficult: A straight-forward extension is unidentifiable, i.e. it is not possible to recover those latent components that actually generated the data. Here, we show that this problem can be solved by using temporal structure. We formulate two generative models in which the data is an arbitrary but invertible nonlinear transformation of time series (components) which are statistically independent of each other. Drawing from the theory of linear ICA, we formulate two distinct classes of temporal structure of the components which enable identification, i.e. recovery of the original independent components. We show that in both cases, the actual learning can be performed by ordinary neural network training where only the input is defined in an unconventional manner, making software implementations trivial. We can rigorously prove that after such training, the units in the last hidden layer will give the original independent components. [With Hiroshi Morioka, published at NIPS2016 and AISTATS2017.]\r\n\r\n\u00a0<\/strong>"},{"id":1,"name":"Past speakers","content":"
Generalizing Convolutions for Deep Learning<\/h3>\r\nProf. Dr. Max Welling, University of Amsterdam<\/h4>\r\nArguably, most excitement about deep learning revolves around the performance of convolutional neural networks and their ability to automatically extract useful features from signals. In this talk I will present work from AMLAB where we generalize these convolutions. First we study convolutions on graphs and propose a simple new method to learn embeddings of graphs which are subsequently used for semi-supervised learning and link prediction. We discuss applications to recommender systems and knowledge graphs. Second we propose a new type of convolution on regular grids based on group transformations. This generalizes normal convolutions based on translations to larger groups including the rotation group. Both methods often result in significant improvements relative to the current state of the art.\r\n\r\nJoint work with Thomas Kipf, Rianne van den Berg and Taco Cohen.\r\n\r\nVisit the event page<\/a>\r\n\r\n\u00a0<\/strong>"}],"msr_startdate":"2017-05-03","msr_enddate":"2017-05-03","msr_event_time":"13:00 - 14:00","msr_location":"Microsoft Research Cambridge UK","msr_event_link":"","msr_event_recording_link":"","msr_startdate_formatted":"May 3, 2017","msr_register_text":"Watch now","msr_cta_link":"","msr_cta_text":"","msr_cta_bi_name":"","featured_image_thumbnail":null,"event_excerpt":"Frontiers in Artificial Intelligence is a series of public lectures at Microsoft Research Cambridge featuring leading researchers in the field, focusing on the cutting edge topics at the intersection of machine learning, statistics, and artificial intelligence. Students, scientists, and engineers in academia and industry are all welcome to join us for these exciting talks and the opportunity to socialize with the Cambridge AI\/ML community. Nonlinear ICA using temporal structure: a principled framework for unsupervised deep…","msr_research_lab":[199561],"related-researchers":[],"msr_impact_theme":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-opportunities":[],"related-publications":[],"related-videos":[],"related-posts":[],"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/376433","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-event"}],"version-history":[{"count":2,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/376433\/revisions"}],"predecessor-version":[{"id":1147176,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/376433\/revisions\/1147176"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=376433"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=376433"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=376433"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=376433"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=376433"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=376433"},{"taxonomy":"msr-program-audience","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-program-audience?post=376433"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=376433"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=376433"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}