{"id":147350,"date":"2008-03-01T00:00:00","date_gmt":"2008-03-01T00:00:00","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/msr-research-item\/avancees-theoriques-sur-la-representation-et-loptimisation-des-reseaux-de-neurones\/"},"modified":"2018-10-16T20:20:39","modified_gmt":"2018-10-17T03:20:39","slug":"avancees-theoriques-sur-la-representation-et-loptimisation-des-reseaux-de-neurones","status":"publish","type":"msr-research-item","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/publication\/avancees-theoriques-sur-la-representation-et-loptimisation-des-reseaux-de-neurones\/","title":{"rendered":"Avanc\u00c3\u00a9es th\u00c3\u00a9oriques sur la repr\u00c3\u00a9sentation et l&#8217;optimisation des r\u00c3\u00a9seaux de neurones"},"content":{"rendered":"<div class=\"asset-content\">\n<p>Les r\u00e9seaux de neurones artificiels ont \u00e9t\u00e9 abondamment utilis\u00e9s dans la communaut\u00e9 de l\u2019apprentissage machine depuis les ann\u00e9es 80. Bien qu\u2019ils aient \u00e9t\u00e9 \u00e9tudi\u00e9s pour la premi\u00e8re fois il y a cinquante ans par Rosenblatt [68], ils ne furent r\u00e9ellement populaires qu\u2019apr\u00e8s l\u2019apparition de la r\u00e9tropropagation du gradient, en 1986 [71]. En 1989, il a \u00e9t\u00e9 prouv\u00e9 [44] qu\u2019une classe sp\u00e9cifique de r\u00e9seaux de neurones (les r\u00e9seaux de neurones \u00e0 une couche cach\u00e9e) \u00e9tait suffisamment puissante pour pouvoir approximer presque n\u2019importe quelle fonction avec une pr\u00e9cision arbitraire : le th\u00e9or\u00e8me d\u2019approximation universelle. Toutefois, bien que ce th\u00e9or\u00e8me e\u00fbt pour cons\u00e9quence un int\u00e9r\u00eat accru pour les r\u00e9seaux de neurones, il semblerait qu\u2019aucun effort n\u2019ait \u00e9t\u00e9 fait pour profiter de cette propri\u00e9t\u00e9. En outre, l\u2019optimisation des r\u00e9seaux de neurones \u00e0 une couche cach\u00e9e n\u2019est pas convexe. Cela a d\u00e9tourn\u00e9 une grande partie de la communaut\u00e9 vers d\u2019autres algorithmes, comme par exemple les machines \u00e0 noyau (machines \u00e0 vecteurs de support et r\u00e9gression \u00e0 noyau, entre autres). La premi\u00e8re partie de cette th\u00e8se pr\u00e9sentera les concepts d\u2019apprentissage machine g\u00e9n\u00e9raux n\u00e9cessaires \u00e0 la compr\u00e9hension des algorithmes utilis\u00e9s. La deuxi\u00e8me partie se focalisera plus sp\u00e9cifiquement sur les m\u00e9thodes \u00e0 noyau et les r\u00e9seaux de neurones. La troisi\u00e8me partie de ce travail visera ensuite \u00e0 \u00e9tudier les limitations des machines \u00e0 noyaux et \u00e0 comprendre les raisons pour lesquelles elles sont inadapt\u00e9es \u00e0 certains probl\u00e8mes que nous avons \u00e0 traiter. La quatri\u00e8me partie pr\u00e9sente une technique permettant d\u2019optimiser les r\u00e9seaux de neurones \u00e0 une couche cach\u00e9e de mani\u00e8re convexe. Bien que cette technique s\u2019av\u00e8re difficilement exploitable pour des probl\u00e8mes de grande taille, une version approch\u00e9e permet d\u2019obtenir une bonne solution dans un temps raisonnable. La cinqui\u00e8me partie se concentre sur les r\u00e9seaux de neurones \u00e0 une couche cach\u00e9e infinie. Cela leur permet th\u00e9oriquement d\u2019exploiter la propri\u00e9t\u00e9 d\u2019approximation universelle et ainsi d\u2019approcher facilement une plus grande classe de fonctions. Toutefois, si ces deux variations sur les r\u00e9seaux de neurones \u00e0 une couche cach\u00e9e leur conf\u00e8rent des propri\u00e9t\u00e9s int\u00e9ressantes, ces derniers ne peuvent extraire plus que des concepts de bas niveau. Les m\u00e9thodes \u00e0 noyau souffrant des m\u00eames limites, aucun de ces deux types d\u2019algorithmes ne peut appr\u00e9hender des probl\u00e8mes faisant appel \u00e0 l\u2019apprentissage de concepts de haut niveau. R\u00e9cemment sont apparus les Deep Belief Networks [39] qui sont des r\u00e9seaux de neurones \u00e0 plusieurs couches cach\u00e9es entra\u00een\u00e9s de mani\u00e8re efficace. Cette profondeur leur permet d\u2019extraire des concepts de haut niveau et donc de r\u00e9aliser des t\u00e2ches hors de port\u00e9e des algorithmes conventionnels. La sixi\u00e8me partie \u00e9tudie des propri\u00e9t\u00e9s de ces r\u00e9seaux profonds. Les probl\u00e8mes que l\u2019on rencontre actuellement n\u00e9cessitent non seulement des algorithmes capables d\u2019extraire des concepts de haut niveau, mais \u00e9galement des m\u00e9thodes d\u2019optimisation capables de traiter l\u2019immense quantit\u00e9 de donn\u00e9es parfois disponibles, si possible en temps r\u00e9el. La septi\u00e8me partie est donc la pr\u00e9sentation d\u2019une nouvelle technique permettant une optimisation plus rapide.<\/p>\n<\/div>\n<p><!-- .asset-content --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Les r\u00e9seaux de neurones artificiels ont \u00e9t\u00e9 abondamment utilis\u00e9s dans la communaut\u00e9 de l\u2019apprentissage machine depuis les ann\u00e9es 80. Bien qu\u2019ils aient \u00e9t\u00e9 \u00e9tudi\u00e9s pour la premi\u00e8re fois il y a cinquante ans par Rosenblatt [68], ils ne furent r\u00e9ellement populaires qu\u2019apr\u00e8s l\u2019apparition de la r\u00e9tropropagation du gradient, en 1986 [71]. En 1989, il a [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"","msr_doi":"","msr_arxiv_id":"","msr_s2_paper_id":"","msr_mag_id":"","msr_pubmed_id":"","msr_other_authors":"Le Roux, Nicolas","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_original_fields_of_study":"","msr_release_tracker_id":"","msr_s2_match_type":"","msr_citation_count_updated":"","msr_published_date":"2008-03-01","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"http:\/\/www.iro.umontreal.ca\/~lisa\/pointeurs\/LeRouxNicolasThese.pdf","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_s2_pdf_url":"","msr_year":2008,"msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_match_confidence":0,"msr_microsoftintellectualproperty":true,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13547],"msr-publication-type":[193725],"msr-publisher":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-147350","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2008-03-01","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"208284","msr_publicationurl":"http:\/\/www.iro.umontreal.ca\/~lisa\/pointeurs\/LeRouxNicolasThese.pdf","msr_doi":"","msr_publication_uploader":[{"type":"file","title":"LeRouxNicolasThese.pdf","viewUrl":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2016\/02\/LeRouxNicolasThese.pdf","id":208284,"label_id":0},{"type":"url","title":"http:\/\/www.iro.umontreal.ca\/~lisa\/pointeurs\/LeRouxNicolasThese.pdf","viewUrl":false,"id":false,"label_id":0}],"msr_related_uploader":"","msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[{"id":0,"url":"http:\/\/www.iro.umontreal.ca\/~lisa\/pointeurs\/LeRouxNicolasThese.pdf"},{"id":208284,"url":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2016\/02\/LeRouxNicolasThese.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"nicolasl","user_id":33093,"rest_url":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=nicolasl"},{"type":"text","value":"Le Roux","user_id":0,"rest_url":false},{"type":"text","value":"Nicolas","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[],"msr_project":[],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"phdthesis","related_content":[],"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/147350","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/147350\/revisions"}],"predecessor-version":[{"id":526971,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/147350\/revisions\/526971"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=147350"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=147350"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=147350"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=147350"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=147350"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=147350"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=147350"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=147350"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=147350"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=147350"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=147350"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=147350"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=147350"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}