{"id":722389,"date":"2021-02-03T09:04:55","date_gmt":"2021-02-03T17:04:55","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=722389"},"modified":"2021-02-11T12:37:06","modified_gmt":"2021-02-11T20:37:06","slug":"microsoft-vision-model-resnet-50-combines-web-scale-data-and-multi-task-learning-to-achieve-state-of-the-art","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/microsoft-vision-model-resnet-50-combines-web-scale-data-and-multi-task-learning-to-achieve-state-of-the-art\/","title":{"rendered":"Microsoft Vision Model ResNet-50 combines web-scale data and multi-task learning to achieve state of the art"},"content":{"rendered":"\n
\"Two
Microsoft Vision Model ResNet-50 is a state-of-the-art pretrained ResNet-50 model, measured above by the mean average score across seven popular computer vision benchmarks.<\/figcaption><\/figure>\n\n\n\n

Pretrained vision models accelerate deep learning research and bring down the cost of performing computer vision tasks in production. By pretraining one large vision model to learn general visual representation of images, then transferring the learning across multiple downstream tasks, a team achieves competitive performance at a fraction of the cost when compared to collecting new data and training a new model for each task. Further fine-tuning of the pretrained model with task-specific training data often yields even higher performance than training specialized models.<\/p>\n\n\n\n

Microsoft Vision Model ResNet-50 is a large pretrained vision model created by the Multimedia Group at Microsoft Bing. The model is built using the search engine\u2019s web-scale image data in order to power its Image Search (opens in new tab)<\/span><\/a> and Visual Search (opens in new tab)<\/span><\/a>. We are excited to announce that we are making Microsoft Vision Model ResNet-50 publicly available today.<\/p>\n\n\n\n

<\/td>Microsoft
Vision Model<\/strong><\/td>		Google
Big Transfer<\/strong><\/td>		OpenAI
CLIP<\/strong><\/td>		PyTorch
ResNet-50<\/strong><\/td><\/tr>
CIFAR-10<\/strong><\/td>92.64<\/td>92.51<\/td>87.85<\/td>82.23<\/td><\/tr>
CIFAR-100<\/strong><\/td>76.05<\/td>79.84<\/td>67.02<\/td>61.36<\/td><\/tr>
STL-10<\/strong><\/td>98.10<\/td>98.71<\/td>97.20<\/td>96.32<\/td><\/tr>
SVHN<\/strong><\/td>72.64<\/td>64.22<\/td>64.33<\/td>52.05<\/td><\/tr>
CUB<\/strong><\/td>82.20<\/td>82.75<\/td>68.38<\/td>38.79<\/td><\/tr>
Flowers-102<\/strong><\/td>99.28<\/td>99.38<\/td>95.23<\/td>77.62<\/td><\/tr>
ImageNet <\/strong><\/td>73.85<\/td>72.83<\/td>57.00<\/td>75.63<\/td><\/tr>
Average<\/strong><\/td>84.97<\/td>84.32<\/td>76.72<\/td>69.14<\/td><\/tr><\/tbody><\/table>
Evaluation of Microsoft Vision Model ResNet-50 and comparable models on seven popular computer vision benchmarks<\/figcaption><\/figure>\n\n\n\n

We evaluate Microsoft Vision Model ResNet-50 against the state-of-the-art pretrained ResNet-50 models and the baseline PyTorch implementation of ResNet-50, following the experiment setup of OpenAI CLIP (opens in new tab)<\/span><\/a>. Linear probe is a standard evaluation protocol for representation learning in which a linear classifier is trained on frozen embeddings of the pretrained vision model for each benchmark.<\/p>\n\n\n\n

To achieve the state-of-the-art performance in a cost-sensitive production setting, Microsoft Vision Model ResNet-50 leverages multi-task learning and optimizes separately for four datasets, including ImageNet-22k (opens in new tab)<\/span><\/a>, Microsoft COCO<\/a>, and two web-supervised (opens in new tab)<\/span><\/a> datasets containing 40 million image-label pairs collected from image search engines.<\/p>\n\n\n\n

\"\"
Multi-task learning via hard parameter sharing<\/figcaption><\/figure><\/div>\n\n\n\n

We chose to use multi-task learning with hard parameter sharing (see figure above). Single neural networks will optimize each classification problem at the same time. By using tasks of varied sizes\u2014up to 40 million images with 100,000 different labels from web-supervised sources\u2014Microsoft Vision Model ResNet-50 can achieve high robustness and good transferability to different domains.  <\/p>\n\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: AI-POWERED EXPERIENCE<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"\"\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft research copilot experience<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Discover more about research at Microsoft through our AI-powered experience<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tStart now\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n\n

During training, images from each dataset are sampled proportionally to the size of the datasets. This approach promotes larger datasets at first, but once optimization flattens, the optimizer needs to look for improvements to smaller datasets without downgrading the performance of the larger ones. As a result, final accuracy scores for each of the datasets are competitive with specialized models trained on each specific dataset.<\/p>\n\n\n\n

Dive deeper into Microsoft Vision Model ResNet-50<\/h2>\n\n\n\n
\n\t
\n\t\t
\n\t\t\t\t\t\t\t\n\t\t\t\t\t\"Microsoft\t\t\t\t<\/a>\n\t\t\t\t\t\t\tWebinar <\/span>\n\t\t\tMicrosoft Vision Model ResNet-50: Pretrained vision model built with web-scale data<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n
\n\t
\n\t\t
\n\t\t\t\t\t\tCode<\/span>\n\t\t\tMicrosoft Vision Model ResNet-50<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

You can get your hands on Microsoft Vision Model ResNet-50 by visiting https:\/\/aka.ms\/microsoftvision (opens in new tab)<\/span><\/a>. On this webpage, you will find a description of how to install and use the model to encode images into embedding vectors. We are also hosting a public webinar about our model on February 25 at 10 AM PT. Part of the webinar will be a demo of applying the model to example computer vision tasks, and there will be a live Q&A session at the end. You can learn more and register (opens in new tab)<\/span><\/a> for the webinar at its registration page.<\/p>\n\n\n\n

Acknowledgments<\/h3>\n\n\n\n

Microsoft Vision Model ResNet-50 is one in a family of world-class computer vision models we\u2019ve built at Microsoft Bing Multimedia Group. We thank Mark Bolin, Ravi Yada, Kun Wu, Meenaz Merchant, Arun Sacheti, and Jordi Ribas for enabling this work. If the opportunity to build pioneering computer vision models excites you, visit our career page (opens in new tab)<\/span><\/a> to learn about our openings.<\/p>\n","protected":false},"excerpt":{"rendered":"

Pretrained vision models accelerate deep learning research and bring down the cost of performing computer vision tasks in production. By pretraining one large vision model to learn general visual representation of images, then transferring the learning across multiple downstream tasks, a team achieves competitive performance at a fraction of the cost when compared to collecting […]<\/p>\n","protected":false},"author":38838,"featured_media":722935,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13562],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-722389","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-computer-vision","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[{"type":"guest","value":"zygmunt-lenyk","user_id":"722944","display_name":"Zygmunt Lenyk","author_link":"Zygmunt Lenyk<\/a>","is_active":true,"last_first":"Lenyk, Zygmunt","people_section":0,"alias":"zygmunt-lenyk"},{"type":"guest","value":"junwon-park","user_id":"722941","display_name":"Junwon Park","author_link":"Junwon Park<\/a>","is_active":true,"last_first":"Park, Junwon ","people_section":0,"alias":"junwon-park"}],"msr_type":"Post","featured_image_thumbnail":"\"Graphs","byline":"Zygmunt Lenyk<\/a> and Junwon Park<\/a>","formattedDate":"February 3, 2021","formattedExcerpt":"Pretrained vision models accelerate deep learning research and bring down the cost of performing computer vision tasks in production. By pretraining one large vision model to learn general visual representation of images, then transferring the learning across multiple downstream tasks, a team achieves competitive performance…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/722389","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/38838"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=722389"}],"version-history":[{"count":11,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/722389\/revisions"}],"predecessor-version":[{"id":725695,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/722389\/revisions\/725695"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/722935"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=722389"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=722389"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=722389"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=722389"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=722389"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=722389"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=722389"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=722389"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=722389"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=722389"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=722389"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}