{"id":723253,"date":"2021-02-11T11:23:51","date_gmt":"2021-02-11T19:23:51","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=723253"},"modified":"2021-03-24T13:17:46","modified_gmt":"2021-03-24T20:17:46","slug":"denoised-smoothing-provably-defending-pretrained-classifiers-against-adversarial-examples","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/denoised-smoothing-provably-defending-pretrained-classifiers-against-adversarial-examples\/","title":{"rendered":"Denoised smoothing: Provably defending pretrained classifiers against adversarial examples"},"content":{"rendered":"\n
\"An<\/figure>\n\n\n\n

Editor\u2019s note: This post and its research are the result of the collaborative efforts of a team of researchers comprising former Microsoft Research Engineer Hadi Salman (opens in new tab)<\/span><\/a>, CMU PhD student Mingjie Sun (opens in new tab)<\/span><\/a>, Researcher Greg Yang<\/a>, Partner Research Manager Ashish Kapoor<\/a>, and CMU Associate Professor J. Zico Kolter (opens in new tab)<\/span><\/a>.<\/em><\/p>\n\n\n\n

It\u2019s been well-documented that subtle modifications to the inputs of image classification systems can lead to bad predictions. Take, for example, a model trained to classify images of an elephant. The model easily classifies an image of the animal grazing in a grassy field. Now, if just a few pixels in that image are maliciously altered, you can get a very different\u2014and wrong<\/em>\u2014prediction despite the image appearing unchanged to the human eye. Sensitivity to such input perturbations, which are known as adversarial examples, raises security and reliability issues for the vision-based systems that we deploy in the real world. To tackle this challenge, recent research has revolved around building defenses against such adversarial examples. However, most of these adversarial defenses, such as randomized smoothing, require specifically training a classifier with a custom objective, which can be computationally expensive.<\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication <\/span>\n\t\t\tDenoised Smoothing: A Provable Defense for Pretrained Classifiers<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n
\n\t
\n\t\t
\n\t\t\t\t\t\t\t\n\t\t\t\t\t\"\"\t\t\t\t<\/a>\n\t\t\t\t\t\t\tEvent<\/span>\n\t\t\tMicrosoft at NeurIPS 2020<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

We consider a slightly different scenario: can we generate a provably robust classifier from off-the-shelf pretrained classifiers without <\/em>retraining them specifically for robustness? In the paper \u201cDenoised Smoothing: A Provable Defense for Pretrained Classifiers<\/a>,\u201d which we presented at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020)<\/a>, we introduce denoised smoothing. Via the simple addition of a pretrained denoiser, we can apply randomized smoothing to make existing pretrained classifiers provably robust against adversarial examples without custom training. We envision our method being particularly helpful to those who don\u2019t have the ability or access to train a classifier from scratch, such as users of public image classification APIs.<\/p>\n\n\n\n

Read Paper<\/a>                    <\/strong>    Code + Models (opens in new tab)<\/span><\/a> <\/strong><\/p>\n\n\n\n

\"An
Figure 1: Can we get any robustness guarantees when using pretrained classifiers, like those deployed in public image classification APIs, while having only query access to them?<\/figcaption><\/figure><\/div>\n\n\n\n

Randomized smoothing: An effective provable defense for classifiers<\/h2>\n\n\n\n

At the core of our approach is randomized smoothing, a promising and provable adversarial defense that has been shown to scale to large networks and datasets. Essentially, randomized smoothing smooths out <\/em>a classifier: it takes a \u201cbrittle,\u201d or \u201cjittery,\u201d function and makes it more stable, helping to ensure predictions for inputs in the neighborhood of a specific data point are constant.<\/p>\n\n\n\n

Consider, as described in our paper, \u201ca classifier \\(f\\) mapping inputs in \\(R^d\\) to classes in \\(Y\\); the randomized smoothing procedure converts the base classifier \\(f\\) into a new, smoothed classifier \\(g\\). Specifically, for input \\(x\\), \\(g\\) returns the class that is most likely to be returned by the base classifier \\(f\\) under isotropic Gaussian noise perturbations\u201d (which are required for randomized smoothing) of \\(x\\):<\/p>\n\n\n\n

\\( \\begin{align} g(x) = \\arg\\max_{c \\in \\mathcal{Y}} \\; \\mathbb{P}[f(x+\\delta) = c] \\label{eq:smoothed-hard} \\quad \\text{where} \\; \\delta \\sim \\mathcal{N}(0, \\sigma^2 I) \\; \\end{align}\\)<\/p>\n\n\n\n

<\/div>\n\n\n\n

In the above, \u201cthe noise level \\(\\sigma\\) controls the tradeoff between robustness and accuracy: as \\(\\sigma\\) increases, the robustness of the smoothed classifier increases while its standard accuracy decreases.\u201d In other words, as the classifier becomes more robust to perturbations, there could be a loss of accuracy on images that have no perturbations, requiring users to balance the tradeoff depending on the use case.<\/p>\n\n\n\n

A team of researchers from Carnegie Mellon University (CMU) and the Bosch Center for Artificial Intelligence showed that the above procedure leads to a robustness guarantee (opens in new tab)<\/span><\/a> against \\(\\ell_2\\) adversarial attacks, and subsequent works derived similar guarantees for other threat models<\/a>.<\/p>\n\n\n\n

Randomized smoothing in practice<\/h2>\n\n\n\n

In practice, \\(g(x)\\) is hard to calculate, so it\u2019s estimated by querying the classifier multiple times with modified versions of an image. So given a data point \\(x\\) for which we want a prediction using randomized smoothing, we would take the following steps:<\/p>\n\n\n\n

1. Replicate n <\/em>copies of the image, adding random Gaussian noise to each of them.<\/p>\n\n\n\n

2. Query the base classifier \\(f\\) at each of these data points to get a prediction \\(y_i\\). <\/p>\n\n\n\n

3. Take the majority vote of the \\(y_i\\)‘s as the final prediction of the smoothed classifier \\(g\\) at \\(x\\).<\/p>\n\n\n\n

\"A
Figure 2: To obtain a prediction using randomized smoothing, multiple noisy copies of the image of interest are generated. A classifier trained to classify well under noisy inputs is queried with the noisy copies, producing a prediction for each. The majority vote is then taken as the final prediction and accompanied by a robustness guarantee in the form of a certified radius.<\/figcaption><\/figure><\/div>\n\n\n\n

For a base classifier \\(f\\), one can apply the above procedure to get a prediction of any data point \\(x\\) along with a robustness guarantee in the form of a certified radius, the radius around a given input for which the prediction is guaranteed to be fixed. So, for example, if you pass an image of an elephant with a specific certified radius into a classifier and then pass the same image with a small perturbation through the classifier and the new image falls within that radius, the classifier will give the same prediction for both.<\/p>\n\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tvideo series<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"On\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

On Second Thought<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

A video series with Sinead Bovell built around the questions everyone\u2019s asking about AI. With expert voices from across Microsoft, we break down the tension and promise of this rapidly changing technology, exploring what\u2019s evolving and what\u2019s possible.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tExplore the series\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n\n

But the above procedure assumes that the base classifier \\(f\\) classifies well under Gaussian perturbations of its inputs. Specifically, in Step 3, we take the majority votes over noisy perturbation of the input \\(x\\). So the base classifier \\(f\\) must be trained in a way that classifies well under noisy inputs. Several papers have proposed techniques to train the base classifier \\(f\\), including the CMU-Bosch Center for AI paper, which uses Gaussian noise data augmentation, and our 2019 NeurIPS paper, which uses adversarial training<\/a>.<\/p>\n\n\n\n

Applying randomized smoothing to pretrained classifiers<\/h2>\n\n\n\n

Now what if the base classifier \\(f\\) is some off-the-shelf classifier that wasn\u2019t trained specifically for randomized smoothing\u2014that is, it doesn\u2019t classify well under noisy perturbations of its inputs? For example, what if we applied randomized smoothing to a public vision API? Do we get good robustness guarantees? Figure 3 illustrates what happens in this case. Basically, we get poor predictions and poor robustness guarantees since these APIs aren\u2019t trained to classify well under substantial levels of noise added to their inputs (which, to reiterate, is required for randomized smoothing). Therefore, randomized smoothing leads to trivial guarantees when applied as is to these pretrained classifiers, as we demonstrate in our paper, and thus isn\u2019t effective.<\/p>\n\n\n\n

\"A
Figure 3: If we apply randomized smoothing to off-the-shelf classifiers, bad results are obtained, as these classifiers don\u2019t classify well when substantial noise levels are added to their inputs, which is required for randomized smoothing.<\/figcaption><\/figure><\/div>\n\n\n\n

With our work on denoised smoothing, we make randomized smoothing effective for classifiers that aren\u2019t trained specifically for randomized smoothing. Our proposed method is straightforward; as mentioned above, instead of applying randomized smoothing to these classifiers, we prepend a custom-trained <\/em>denoiser in front of these classifiers and then apply randomized smoothing (Figure 4). The denoiser helps by removing noise from the synthetic noisy copies \\(x\\)+\\(\\delta_i\\) of the input \\(x\\), which allows the pretrained classifiers to give better predictions, as they now see denoised images they\u2019re able to classify better.<\/p>\n\n\n\n

\"A
Figure 4: Denoised smoothing, which adds a custom-trained denoiser before the classifier to be defended, makes randomized smoothing effective for classifiers that aren\u2019t trained specifically for it. The denoiser helps by removing noise from the copies of the input, allowing a pretrained classifier to give better predictions, as it now sees denoised images it\u2019s able to classify better.<\/figcaption><\/figure><\/div>\n\n\n\n

How to train the denoiser<\/h2>\n\n\n\n

In our paper, we investigate several ways for training the denoiser \\(\\mathcal{D}_{\\theta}\\).<\/p>\n\n\n\n

Mean square error objective:<\/strong> simply train the denoiser to minimize the mean square error (MSE) between the clean image and the denoised image<\/p>\n\n\n\n

\\(\\begin{equation}L_{\\text{MSE}}= \\mathbb{E}_{\\mathcal{S}, \\delta}||\\mathcal{D}_{\\theta}(x_i + \\delta)-x_i||_{2}^{2}\\end{equation}\\)<\/p>\n\n\n\n

<\/div>\n\n\n\n

Stability objective:<\/strong> given a classifier \\(F\\) attached to the denoiser \\(\\mathcal{D}_{\\theta}\\), minimize the cross-entropy loss between the prediction of \\(F(\\mathcal{D}_{theta})\\) at the noisy input \\(x_i + \\delta\\) and the prediction of the classifier at the clean data point \\(x_i\\)<\/p>\n\n\n\n

\\(\\begin{align}L_{\\text{Stab}}&=\\mathbb{E}_{\\mathcal{S}, \\delta} \\mathcal{\\ell_\\text{CE}}(F(\\mathcal{D}_\\theta(x_i + \\delta)), f(x_i)) \\quad \\text{where} \\; \\delta \\sim \\mathcal{N}(0, \\sigma^2 I) \\; \\end{align}\\)<\/p>\n\n\n\n

<\/div>\n\n\n\n

MSE-stability hybrid: <\/strong>train the denoiser with MSE, then fine-tune it with the stability objective<\/p>\n\n\n\n

Improvements in certified accuracy<\/h2>\n\n\n\n

Our method allows us to boost the certified accuracy of a ResNet-110 pretrained on CIFAR-10 and ResNet-18\/34\/50 classifiers pretrained on ImageNet as shown in the tables below. We test our method in two scenarios. In the white-box scenario, the pretrained classifier is assumed to be known, which allows for the exploration of more ways to train the denoiser. In theory, since we know the specific classifier being used, we can backpropagate gradients through it to update our denoiser. In the black-box scenario, the pretrained classifier is not known; we don\u2019t have any information about its architecture or parameters. We compare our method deployed in these scenarios to a \u201cno denoiser\u201d baseline, which applies randomized smoothing to these pretrained classifiers directly without any denoising step. We observe substantial improvements in the robustness guarantee (certified accuracy) at various \\(\\ell_2\\) radii (the higher the certified accuracy, the more robust the classifier is).<\/p>\n\n\n\n

L2<\/sub> Radius (CIFAR-10) <\/td>.25<\/td>.5<\/td>.75<\/td>1.0<\/td>1.25<\/td>1.5<\/td><\/tr>
No Denoiser (baseline) (%)<\/td>7<\/td>3<\/td>0<\/td>0<\/td>0<\/td>0<\/td><\/tr>
Denoised Smoothing (black box) (%) <\/td>45<\/td>20<\/td>15<\/td>13<\/td>11<\/td>10<\/td><\/tr>
Denoised Smoothing (white box) (%)<\/td>56<\/td>41<\/td>28<\/td>19<\/td>16<\/td>13<\/td><\/tr><\/tbody><\/table>
Certified accuracy of ResNet-110 on CIFAR-10 at various L2<\/sub> radii.<\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

L2<\/sub> Radius (ImageNet)<\/td>.25<\/td>.5<\/td>.75<\/td>1.0<\/td>1.25<\/td>1.5<\/td><\/tr>
No Denoiser (baseline) (%)<\/td>32<\/td>4<\/td>2<\/td>0<\/td>0<\/td>0<\/td><\/tr>
Denoised Smoothing (black box) (%)<\/td>48<\/td>31<\/td>19<\/td>12<\/td>7<\/td>4<\/td><\/tr>
Denoised Smoothing (white box) (%)<\/td>50<\/td>33<\/td>20<\/td>14<\/td>11<\/td>6<\/td><\/tr><\/tbody><\/table>
Certified top-1 accuracy of ResNet-50 on ImageNet at various L2<\/sub> radii.<\/figcaption><\/figure>\n\n\n\n

Finally, we apply denoised smoothing to four public vision APIs and show how we can get certified predictions using these APIs without having access to the underlying models at all (see the paper<\/a> for the results). While our method can help make possible the protection of vision-based systems from adversarial examples without having to custom train the pretrained classifiers powering them, this work is only a starting point in making pretrained classifiers provably robust. Next steps for denoised smoothing can include working to achieve even stronger guarantees, particularly in scenarios in which the classifier being used is not known.<\/p>\n","protected":false},"excerpt":{"rendered":"

Editor\u2019s note: This post and its research are the result of the collaborative efforts of a team of researchers comprising former Microsoft Research Engineer Hadi Salman (opens in new tab), CMU PhD student Mingjie Sun (opens in new tab), Researcher Greg Yang, Partner Research Manager Ashish Kapoor, and CMU Associate Professor J. Zico Kolter (opens […]<\/p>\n","protected":false},"author":38838,"featured_media":725623,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-723253","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[708199],"related-researchers":[{"type":"guest","value":"hadi-salman","user_id":"624675","display_name":"Hadi Salman","author_link":"Hadi Salman","is_active":true,"last_first":"Salman, Hadi","people_section":0,"alias":"hadi-salman"}],"msr_type":"Post","featured_image_thumbnail":"\"An","byline":"Hadi Salman","formattedDate":"February 11, 2021","formattedExcerpt":"Editor\u2019s note: This post and its research are the result of the collaborative efforts of a team of researchers comprising former Microsoft Research Engineer Hadi Salman (opens in new tab), CMU PhD student Mingjie Sun (opens in new tab), Researcher Greg Yang, Partner Research Manager…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/723253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/38838"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=723253"}],"version-history":[{"count":129,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/723253\/revisions"}],"predecessor-version":[{"id":725689,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/723253\/revisions\/725689"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/725623"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=723253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=723253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=723253"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=723253"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=723253"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=723253"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=723253"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=723253"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=723253"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=723253"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=723253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}