We consider the problem of reconstructing a signal from multi-layered (possibly) non-linear measurements. Using non-rigorous but standard methods from statistical physics we present the Multi-Layer Approximate Message Passing (ML-AMP) algorithm for computing marginal probabilities of the corresponding estimation problem and derive the associated state evolution equations to analyze its performance. We also give the expression of the asymptotic free energy and the minimal information-theoretically achievable reconstruction error. Finally, we present some applications of this measurement model for compressed sensing and perceptron learning with structured matrices\/patterns, and for a simple model of estimation of latent variables in an auto-encoder.<\/p>\n","protected":false},"excerpt":{"rendered":"
We consider the problem of reconstructing a signal from multi-layered (possibly) non-linear measurements. Using non-rigorous but standard methods from statistical physics we present the Multi-Layer Approximate Message Passing (ML-AMP) algorithm for computing marginal probabilities of the corresponding estimation problem and derive the associated state evolution equations to analyze its performance. We also give the expression […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"user_nicename","value":"Andre Manoel","user_id":"40504"},{"type":"text","value":"Florent Krzakala","user_id":0},{"type":"text","value":"Marc Mezard","user_id":0},{"type":"text","value":"Lenka Zdeborova","user_id":0}],"msr_publishername":"IEEE","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"","msr_pages_string":"2098\u20132102","msr_page_range_start":"2098","msr_page_range_end":"2102","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"2017 International Symposium on Information Theory","msr_doi":"10.1109\/ISIT.2017.8006899","msr_arxiv_id":"","msr_mag_id":"2582278104","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_release_tracker_id":"","msr_highlight_type":"","msr_date_display_format":"","msr_main_download_label":"","msr_external_link_label":"","msr_doi_label":"","msr_published_date":"2017-01-25","msr_startdate":"","msr_presentation_date":"","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_year":2017,"msr_month":1,"msr_day":25,"msr_microsoftintellectualproperty":false,"msr_pub_id":"","msr_publication_uploader":[{"type":"doi","viewUrl":"false","id":false,"title":"10.1109\/ISIT.2017.8006899","label_id":243106,"label":0},{"type":"url","viewUrl":"false","id":false,"title":"https:\/\/hal-cea.archives-ouvertes.fr\/cea-01447203\/document","label_id":243132,"label":0},{"type":"url","viewUrl":"false","id":false,"title":"http:\/\/ui.adsabs.harvard.edu\/abs\/2017arXiv170106981M\/abstract","label_id":243109,"label":0},{"type":"url","viewUrl":"false","id":false,"title":"https:\/\/arxiv.org\/abs\/1701.06981","label_id":243109,"label":0},{"type":"url","viewUrl":"false","id":false,"title":"https:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1701.html#ManoelKMZ17","label_id":243109,"label":0},{"type":"url","viewUrl":"false","id":false,"title":"https:\/\/ieeexplore.ieee.org\/document\/8006899\/","label_id":243109,"label":0}],"msr_related_uploader":[],"msr_original_fields_of_study":["Machine learning","Computer science","Artificial intelligence","Matrix (mathematics)","Algorithm","Expression (mathematics)","Latent variable","Simple (abstract algebra)","Message passing","Perceptron","Compressed sensing","Information theory","Energy (signal processing)"],"msr_s2_paper_id":"","msr_s2_pdf_url":"","msr_citation_count_updated":"","msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13556],"msr-publication-type":[193716],"msr-publisher":[],"msr-publication-cta":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[246904,246694,258160,246691,258166,248725,258163,249538,246685,246862,256957,258157,256267],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-760723","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-field-of-study-algorithm","msr-field-of-study-artificial-intelligence","msr-field-of-study-compressed-sensing","msr-field-of-study-computer-science","msr-field-of-study-energy-signal-processing","msr-field-of-study-expression-mathematics","msr-field-of-study-information-theory","msr-field-of-study-latent-variable","msr-field-of-study-machine-learning","msr-field-of-study-matrix-mathematics","msr-field-of-study-message-passing","msr-field-of-study-perceptron","msr-field-of-study-simple-abstract-algebra"],"msr_publishername":"IEEE","msr_edition":"","msr_affiliation":"","msr_published_date":"2017-01-25","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"2098\u20132102","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":0,"msr_main_download":"","msr_publicationurl":"","msr_doi":"10.1109\/ISIT.2017.8006899","msr_publication_uploader":[{"type":"doi","viewUrl":"false","id":"false","title":"10.1109\/ISIT.2017.8006899","label_id":"243106","label":0},{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/hal-cea.archives-ouvertes.fr\/cea-01447203\/document","label_id":"243132","label":0},{"type":"url","viewUrl":"false","id":"false","title":"http:\/\/ui.adsabs.harvard.edu\/abs\/2017arXiv170106981M\/abstract","label_id":"243109","label":0},{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/arxiv.org\/abs\/1701.06981","label_id":"243109","label":0},{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1701.html#ManoelKMZ17","label_id":"243109","label":0},{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/ieeexplore.ieee.org\/document\/8006899\/","label_id":"243109","label":0}],"msr_related_uploader":[],"msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[],"msr-author-ordering":[{"type":"user_nicename","value":"Andre Manoel","user_id":40504,"rest_url":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Andre Manoel"},{"type":"text","value":"Florent Krzakala","user_id":0,"rest_url":false},{"type":"text","value":"Marc Mezard","user_id":0,"rest_url":false},{"type":"text","value":"Lenka Zdeborova","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[],"msr_project":[],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"inproceedings","related_content":[],"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/760723","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":1,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/760723\/revisions"}],"predecessor-version":[{"id":760726,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/760723\/revisions\/760726"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=760723"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=760723"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=760723"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=760723"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=760723"},{"taxonomy":"msr-publication-cta","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-cta?post=760723"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=760723"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=760723"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=760723"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=760723"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=760723"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=760723"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=760723"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=760723"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}