SemanticAdv: Generating Adversarial Examples
via Attribute-conditioned Image Editing
通过属性条件图像编辑生成对抗样本

For simplicity, we start with the formulation where the input attribute is represented as a compact vector. This formulation can be directly extended to other input attribute formats including semantic layouts. Let $\mathbf{c}\in\mathbb{R}^{D_{C}}$ be an attribute representation reflecting the semantic factors (e.g., expression or hair color of a portrait image) of image $\mathbf{x}$, where $D_{C}$ indicates the attribute dimension and $c_{i}\in\{0,1\}$ indicates the existence of $i$-th attribute. We are interested in performing semantic image editing using the attribute-conditioned image generator $\mathcal{G}$. For example, given a portrait image of a girl with black hair and the new attribute blonde hair, our generator is supposed to synthesize a new image that turns the girl’s hair color from black to blonde while keeping the rest of appearance unchanged. The synthesized image is denoted as $\mathbf{x}^{\text{new}}=\mathcal{G}(\mathbf{x},\mathbf{c}^{\text{new}})$ where $\mathbf{c}^{\text{new}}\in\mathbb{R}^{D_{C}}$ is the new attribute. In the special case when there is no attribute change ($\mathbf{c}=\mathbf{c}^{\text{new}}$), the generator simply reconstructs the input: $\mathbf{x^{\prime}}=\mathcal{G}(\mathbf{x},\mathbf{c})$ (ideally, we hope $\mathbf{x^{\prime}}$ equals to $\mathbf{x}$). As our attribute representation is disentangled and the change of attribute value is sufficiently small (e.g., we only edit a single semantic attribute), our synthesized image $\mathbf{x}^{\text{new}}$ is expected to be close to the data manifold [4, 57, 55]. In addition, we can generate many similar images by linearly interpolating between the image pair $\mathbf{x}$ and $\mathbf{x}^{\text{new}}$ in the attribute-space or the feature-space of the image-conditioned generator $\mathcal{G}$, which is supported by the previous work [75, 55, 3]
为简便起见，我们从将输入属性表示为紧凑向量的公式开始。这种公式可以直接扩展到其他输入属性格式，包括语义布局。 设 $\mathbf{c}\in\mathbb{R}^{D_{C}}$ 是反映图像 $\mathbf{x}$ 的语义因素（例如肖像图像的表情或头发颜色）的属性表示，其中 $D_{C}$ 表示属性维度， $c_{i}\in\{0,1\}$ 表示属性的存在。我们有兴趣使用属性条件的图像生成器 $\mathcal{G}$ 执行语义图像编辑。例如，给定一幅黑发女孩的肖像图像和新属性金发，我们的生成器应该合成一幅新图像，将女孩的头发颜色从黑色变为金色，同时保持其余外观不变。合成图像表示为 $\mathbf{x}^{\text{new}}=\mathcal{G}(\mathbf{x},\mathbf{c}^{\text{new}})$ ，其中 $\mathbf{c}^{\text{new}}\in\mathbb{R}^{D_{C}}$ 是新属性。在没有属性变化的特殊情况下（ $\mathbf{c}=\mathbf{c}^{\text{new}}$ ），生成器仅重建输入： $\mathbf{x^{\prime}}=\mathcal{G}(\mathbf{x},\mathbf{c})$ （理想情况下，我们希望 $\mathbf{x^{\prime}}$ 等于 $\mathbf{x}$ ）。 由于我们的属性表示是解缰的，属性值的变化足够小（例如，我们只编辑一个语义属性），我们合成的图像 $\mathbf{x}^{\text{new}}$ 预计会接近数据流形。[4, 57, 55]。此外，我们可以在图像条件生成器的属性空间或特征空间中通过线性插值生成许多与图像对 $\mathbf{x}$ 和 $\mathbf{x}^{\text{new}}$ 相似的图像，该方法得到了之前研究的支持[75, 55, 3]

Given a pair of attributes $\mathbf{c}$ and $\mathbf{c}^{\text{new}}$, we introduce an interpolation parameter $\alpha\in(0,1)$ to generate the augmented attribute vector $\mathbf{c}^{*}\in\mathbb{R}^{D_{C}}$ (see Eq. 1). Given augmented attribute $\mathbf{c}^{*}$ and original image $\mathbf{x}$, we produce the image $\mathbf{x}^{\text{*}}$ by the generator $\mathcal{G}$ through attribute-space interpolation.
给定一对属性 $\mathbf{c}$ 和 $\mathbf{c}^{\text{new}}$ ，我们引入一个插值参数 $\alpha\in(0,1)$ 来生成增强的属性向量 $\mathbf{c}^{*}\in\mathbb{R}^{D_{C}}$ （见公式1）。给定增强属性 $\mathbf{c}^{*}$ 和原始图像 $\mathbf{x}$ ，通过属性空间插值，我们通过生成器 $\mathcal{G}$ 生成图像 $\mathbf{x}^{\text{*}}$ 。

Alternatively, we propose to interpolate using the feature map produced by the generator $\mathcal{G}=\mathcal{G}_{\text{dec}}\circ\mathcal{G}_{\text{enc}}$. Here, $\mathcal{G}_{\text{enc}}$ is the encoder module that takes the image as input and outputs the feature map. Similarly, $\mathcal{G}_{\text{dec}}$ is the decoder module that takes the feature map as input and outputs the synthesized image. Let $\mathbf{f^{*}}=\mathcal{G}_{\text{enc}}(\mathbf{x},\mathbf{c})\in\mathbb{R}^{H_{F}\times W_{F}\times C_{F}}$ be the feature map of an intermediate layer in the generator, where $H_{F}$, $W_{F}$ and $C_{F}$ indicate the height, width, and number of channels in the feature map.
另外，我们建议使用生成器生成的特征图进行插值。这里， $\mathcal{G}_{\text{enc}}$ 是将图像作为输入并输出特征图的编码器模块。类似地， $\mathcal{G}_{\text{dec}}$ 是将特征图作为输入并输出合成图像的解码器模块。设 $\mathbf{f^{*}}=\mathcal{G}_{\text{enc}}(\mathbf{x},\mathbf{c})\in\mathbb{R}^{H_{F}\times W_{F}\times C_{F}}$ 为生成器中间层的特征图， $H_{F}$ ， $W_{F}$ 和 $C_{F}$ 表示特征图的高度、宽度和通道数。

Compared to the attribute-space interpolation which is parameterized by a scalar $\alpha$, we parameterize feature-map interpolation by a tensor ${\boldsymbol{\beta}}\in\mathbb{R}^{H_{F}\times W_{F}\times C_{F}}$ ($\beta_{h,w,k}\in[0,1]$, where $1\leq h\leq H_{F}$, $1\leq w\leq W_{F}$, and $1\leq k\leq C_{F}$) with the same shape as the feature map. Compared to linear interpolation over attribute-space, such design introduces more flexibility for adversarial attacks. Empirical results in Section 4.2 show such design is critical to maintain both attack success and good perceptual quality at the same time.
与由标量 $\alpha$ 参数化的属性空间插值相比，我们通过一个与特征图具有相同形状的张量 ${\boldsymbol{\beta}}\in\mathbb{R}^{H_{F}\times W_{F}\times C_{F}}$ （ $\beta_{h,w,k}\in[0,1]$ ，其中 $1\leq h\leq H_{F}$ ， $1\leq w\leq W_{F}$ 和 $1\leq k\leq C_{F}$ ）来参数化特征图插值。与属性空间的线性插值相比，这种设计为对抗攻击引入了更多的灵活性。第4.2节的实证结果表明，这种设计对于同时保持攻击成功和良好的感知质量至关重要。

In the identity verification task, two images are considered to be the same identity if the corresponding identity embeddings from the verification model $\mathcal{M}$ are reasonably close.
在身份验证任务中，如果验证模型中对应的身份嵌入特征相近，那么两个图像被认为是同一个身份。

As we see in Eq. 4, $\Phi_{\mathcal{M}}^{\text{id}}(\cdot,\cdot)$ measures the distance between two identity embeddings from the model $\mathcal{M}$, where the normalized $L_{2}$ distance is used in our setting. In addition, we introduce the parameter $\kappa$ representing the constant related to the false positive rate (FPR) threshold computed from the development set.
如方程式4所示， $\Phi_{\mathcal{M}}^{\text{id}}(\cdot,\cdot)$ 测量了来自模型 $\mathcal{M}$ 的两个身份嵌入之间的距离，我们的设置中采用了归一化的 $L_{2}$ 距离。此外，我们引入了参数 $\kappa$ ，表示与从开发集计算得到的假阳性率（FPR）阈值相关的常数。

For structured prediction tasks such as landmark detection and semantic segmentation, we use Houdini objective proposed in [13] as our adversarial metric and select the target landmark (semantic segmentation) target as $\mathbf{y}^{\text{tgt}}$. As we see in the equation, $\Phi_{\mathcal{M}}(\cdot,\cdot)$ is a scoring function for each image-label pair and $\gamma$ is the threshold. In addition, $l(\mathbf{y}^{*},\mathbf{y}^{\text{tgt}})$ is task loss decided by the specific adversarial target, where $\mathbf{y}^{*}=\mathcal{M}(\mathbf{x}^{*})$.
对于结构化预测任务，如地标检测和语义分割，我们将 [13] 中提出的Houdini目标作为我们的对抗性度量，并选择目标地标（语义分割）目标为 $\mathbf{y}^{\text{tgt}}$

As the tensor to be interpolated in the feature-map space has far more parameters compared to the attribute itself, we propose to enforce a smoothness constraint on the tensor $\alpha$ used in feature-map interpolation. As we see in Eq. 6, the smoothness loss encourages the interpolation tensors to consist of piece-wise constant patches spatially, which has been widely used as a pixel-wise de-noising objective for natural image processing [43, 27].
由于特征图空间中需要插值的张量参数远远多于属性本身，我们建议对特征图插值中使用的张量 $\alpha$ 施加平滑限制。从等式6中可以看到，平滑损失鼓励插值张量在空间上由分段常数块组成，这在自然图像处理中被广泛用作逐像素去噪目标。

We select ResNet-50 and ResNet-101 [23] trained on MS-Celeb-1M [22, 15] as our face verification models. The models are trained using two different objectives, namely, softmax loss [63, 80] and cosine loss [67]. For simplicity, we use the notation “R-N-S” to indicate the model with $N$-layer residual blocks as backbone trained using softmax loss, while “R-N-C” indicates the same backbone trained using cosine loss. The distance between face features is measured by normalized L2 distance. For R-101-S model, we decide the parameter $\kappa$ based on the false positive rate (FPR) for the identity verification task. Four different FPRs have been used: $10^{-3}$ (with $\kappa=1.24$), $3\times 10^{-4}$ (with $\kappa=1.05$), $10^{-4}$ (with $\kappa=0.60$), and $<10^{-4}$ (with $\kappa=0.30$). The distance metrics and selected thresholds are commonly used when evaluating the performance of face recognition model [35, 32]. Supplementary provides more details on the performance of face recognition models and their corresponding $\kappa$. To distinguish between the FPR we used in generating adversarial examples and the other FPR used in evaluation, we introduce two notations “Generation FPR (G-FPR)” and “Test FPR (T-FPR)”. For the experiment with query-free black-box API attacks, we use two online face verification services provided by Face++ [2] and AliYun [1].
我们选择在MS-Celeb-1M上训练的ResNet-50和ResNet-101作为我们的人脸验证模型。这些模型使用两种不同的目标进行训练，即softmax损失和余弦损失。为简单起见，我们使用符号“R-N-S”表示使用softmax损失训练的带有 $N$ -layer残差块作为主干的模型，而“R-N-C”表示使用余弦损失训练的相同主干。人脸特征之间的距离通过归一化的L2距离来衡量。对于R-101-S模型，我们根据身份验证任务的误识率（FPR）来确定参数 $\kappa$ 。使用了四种不同的FPR： $10^{-3}$ （使用括号 $\kappa=1.24$ ）、 $3\times 10^{-4}$ （使用括号 $\kappa=1.05$ ）、 $10^{-4}$ （使用括号 $\kappa=0.60$ ）、和 $<10^{-4}$ （ 使用括号 $\kappa=0.30$ ）。在评估人脸识别模型性能时，距离度量和选定的阈值是常用的。详情请参考附件中关于人脸识别模型性能及其相关的 $\kappa$ 。 为了区分生成对抗样本中使用的FPR和评估中使用的另一个FPR，我们引入了两个符号“生成FPR（G-FPR）”和“测试FPR（T-FPR）”。在无需查询的黑盒API攻击实验中，我们使用由Face++ [2] 和阿里云 [1] 提供的两个在线人脸验证服务。

In our experiments, we randomly sample $1,280$ distinct identities form CelebA [40] and use the StarGAN [12] for attribute-conditional image editing. In particular, we re-train our model on CelebA by aligning the face landmarks and then resizing images to resolution $112\times 112$. We select 17 identity-preserving attributes as our analysis, as such attributes mainly reflect variations in facial expression and hair color.
在我们的实验中，我们从CelebA [40] 中随机抽取 $1,280$ 个不同的身份，并使用StarGAN [12] 进行属性条件的图像编辑。具体来说，我们通过将脸部地标进行对齐，然后将图像调整至分辨率 $112\times 112$ ，重新训练我们的模型。我们选择了17个保持身份的属性进行分析，因为这些属性主要反映了面部表情和发色的变化。

In feature-map interpolation, to reduce the reconstruction error brought by the generator (e.g., $\mathbf{x}\neq\mathcal{G}(\mathbf{x},\mathbf{c})$) in practice, we take one more step to obtain the updated feature map $\mathbf{f}^{\prime}=\mathcal{G}_{\text{enc}}(\mathbf{x}^{\prime},\mathbf{c})$, where $\mathbf{x}^{\prime}=\operatorname*{argmin}_{\mathbf{x}^{\prime}}\|\mathcal{G}(\mathbf{x}^{\prime},\mathbf{c})-\mathbf{x}\|$.
在特征图插值中，为了减少生成器（例如， $\mathbf{x}\neq\mathcal{G}(\mathbf{x},\mathbf{c})$ ）带来的重建误差，在实践中，我们再走一步，得到更新后的特征图 $\mathbf{f}^{\prime}=\mathcal{G}_{\text{enc}}(\mathbf{x}^{\prime},\mathbf{c})$ ，其中 $\mathbf{x}^{\prime}=\operatorname*{argmin}_{\mathbf{x}^{\prime}}\|\mathcal{G}(\mathbf{x}^{\prime},\mathbf{c})-\mathbf{x}\|$ 。

For each distinct identity pair $(\mathbf{x},\mathbf{x}^{\text{tgt}})$, we perform semanticAdv guided by each of the 17 attributes (e.g., we intentionally add or remove one specific attribute while keeping the rest unchanged). In total, for each image $\mathbf{x}$, we generate 17 adversarial images with different augmented attributes. In the experiments, we select a commonly-used pixel-wise adversarial attack method [10] (referred as CW) as our baseline. Compared to our proposed method, CW does not require visual attributes as part of the system, as it only generates one adversarial example for each instance. We refer the corresponding attack success rate as the instance-wise success rate in which the attack success rate is calculated for each instance. For each instance with 17 adversarial images using different augmented attributes, if one of the 17 produced images can attack successfully, we count the attack of this instance as one success, vice verse.
对于每对不同的身份 $(\mathbf{x},\mathbf{x}^{\text{tgt}})$ ，我们进行语义增强，其中依据17个属性之一进行引导（例如，我们有意添加或移除一个特定属性，同时保持其余属性不变）。总共，对于每个图像 $\mathbf{x}$ ，我们生成具有不同增强属性的17个对抗性图像。在实验中，我们选择了一种常用的基于像素的对抗攻击方法[10]（简称为CW）作为基准线。与我们提出的方法相比，CW不需要将视觉属性作为系统的一部分，因为它仅为每个实例生成一个对抗性示例。我们将相应的攻击成功率称为实例级成功率，其中攻击成功率是为每个实例计算的。对于使用不同增强属性产生的17个对抗性图像的每个实例，如果其中一个生成的17个图像可以成功攻击，我们将该实例的攻击计为一个成功，反之亦然。

We select Face Alignment Network (FAN) [9] trained on 300W-LP [82] and fine-tuned on 300-W [58] for 2D landmark detection. The network is constructed by stacking Hour-Glass networks [47] with hierarchical block [8]. Given a face image as input, FAN outputs 2D heatmaps which can be subsequently leveraged to yield $68$ 2D landmarks.
我们选择在300W-LP上训练并在300-W上微调的人脸对齐网络（FAN）进行2D地标检测。该网络通过堆叠带有分层块的Hour-Glass网络构建而成。给定一张人脸图像作为输入，FAN输出2D热图，随后可以利用这些热图得到2D地标。

We select DRN-D-22 [77] as our semantic segmentation model and fine-tune the model on image regions with resolution $256\times 256$. To synthesize semantic adversarial perturbations, we consider semantic label maps as the input attribute and leverage a generative image manipulation model [24] pre-trained on CityScape [14] dataset. Given an input semantic label map at resolution $256\times 256$, we select a target object instance (e.g., a pedestrian) to attack. Then, we create a manipulated semantic label map by inserting another object instance (e.g., a car) in the vicinity of the target object. Similar to the experiments in the face domain, for both semantic label maps, we use the image manipulation encoder to extract features (with $1,024$ channels at spatial resolution $16\times 16$) and conduct feature-space interpolation. We synthesize the final image by feeding the interpolated features to the image manipulation decoder. By searching the interpolation coefficient that maximizes the attack rate, we are able to fool the segmentation model with the synthesized final image.
我们选择DRN-D-22 [77]作为我们的语义分割模型，并在分辨率 $256\times 256$ 的图像区域上微调模型。为了合成语义对抗扰动，我们将语义标签映射视为输入属性，并利用在CityScape [14]数据集上预先训练的生成式图像处理模型[24]。给定分辨率为 $256\times 256$ 的输入语义标签映射，我们选择一个目标对象实例（例如，一个行人）进行攻击。然后，在目标对象的附近插入另一个对象实例（例如，一辆汽车）来创建一个操纵过的语义标签映射。类似于面部领域的实验，对于两个语义标签映射，我们使用图像处理编码器提取特征（在空间分辨率为 $16\times 16$ 的 $1,024$ 通道）并进行特征空间插值。通过寻找最大化攻击率的插值系数，我们能够通过将插值后的特征馈送给图像处理解码器来愚弄合成的最终图像，从而欺骗分割模型。

First, we qualitatively compare the two interpolation methods and found that both attribute-space and feature-space interpolation can generate reasonably looking samples (see Figure 2) through interpolation (these are not adversarial examples). However, we found the two interpolation methods perform differently when we optimize using the adversarial objective (Eq. 3). We measure the attack success rate of attribute-space interpolation (with G-FPR = T-FPR = $10^{-3}$): $0.08\%$ on R-101-S, $0.31\%$ on R-101-C, and $0.16\%$ on both R-50-S and R-50-C, which consistently fails to attack the face verification model. Compared to attribute-space interpolation, generating adversarial examples with feature-space interpolation produces much better quantitative results (see Table 1). We conjecture that this is because the high dimensional feature space can provide more manipulation freedom. This also explains one potential reason of poor samples (e.g., blurry with many noticeable artifacts) generated by the method proposed in [29]. We select $\mathbf{f}_{0}$, the last conv layer before up-sampling layer in the generator for feature-space interpolation due to its good performance.
首先，我们在定性上比较了两种插值方法，并发现属性空间和特征空间插值都可以通过插值生成外观合理的样本（参见图2）（这些并不是对抗样本）。然而，当我们使用对抗目标进行优化时（公式3），我们发现这两种插值方法表现不同。我们测量了属性空间插值的攻击成功率（带有G-FPR = T-FPR = $10^{-3}$ ）： $0.08\%$ 在R-101-S上， $0.31\%$ 在R-101-C上， $0.16\%$ 在R-50-S和R-50-C上，这些插值方法始终无法攻击人脸验证模型。与属性空间插值相比，使用特征空间插值生成对抗样本可以产生更好的定量结果（见表1）。我们推测这是因为高维特征空间可以提供更多的操纵自由度。这也解释了[29]中提出的方法生成的样本质量差（例如，模糊且有许多明显的伪像）的一个潜在原因。我们选择在生成器的上采样层之前最后一个卷积层 $\mathbf{f}_{0}$ 进行特征空间插值，因为它的性能很好。

Figure 3 (top) shows the generated adversarial images and corresponding perturbations against R-101-S of SemanticAdv and CW respectively. The text below each figure is the name of an augmented attribute, the sign before the name represents “adding” (in red) or “removing” (in blue) the corresponding attribute from the original image. Figure 3 (bottom) shows the adversarial examples with 17 augmented semantic attributes, respectively. The attribute names are shown in the bottom. The first row contains images generated by $\mathcal{G}(\mathbf{x},\mathbf{c}^{\text{new}})$ with an augmented attribute $\mathbf{c}^{\text{new}}$ and the second row includes the corresponding adversarial images under feature-space interpolation. It shows that our SemanticAdv can generate examples with reasonably-looking appearance guided by the corresponding attribute. In particular, SemanticAdv is able to generate perturbations on the corresponding regions correlated with the augmented attribute, while the perturbations of CW have no specific pattern and are evenly distributed across the image.
图3（上）显示了SemanticAdv和CW针对R-101-S生成的对抗图像及相应的扰动。每张图像下方的文本是一个扩增属性的名称，名称前的符号代表从原始图像中“添加”（红色）或“移除”（蓝色）相应属性。图3（下）分别展示了带有17个扩增语义属性的对抗示例。属性名称显示在下方。第一排包含由 $\mathcal{G}(\mathbf{x},\mathbf{c}^{\text{new}})$ 生成的带有扩增属性 $\mathbf{c}^{\text{new}}$ 的图像，第二排包含相应的在特征空间插值下的对抗图像。结果显示，我们的SemanticAdv能够根据相应属性生成外观合理的示例。特别是，SemanticAdv能够在与扩增属性相关的区域上生成扰动，而CW的扰动没有特定模式且均匀分布在图像中。

To further measure the perceptual quality of the adversarial images generated by SemanticAdv in the most strict settings (G-FPR $<10^{-4}$), we conduct a user study using Amazon Mechanical Turk (AMT). In total, we collect $2,620$ annotations from $77$ participants. In $39.14\pm 1.96\%$ (close to random guess $50\%$) of trials, the adversarial images generated by our SemanticAdv are selected as reasonably-looking images, while $30.27\pm 1.96\%$ of trials by CW are selected as reasonably-looking. It indicates that SemanticAdv can generate more perceptually plausible adversarial examples compared with CW under the most strict setting (G-FPR $<10^{-4}$). The corresponding images are shown in supplementary materials.
为了进一步衡量SemanticAdv生成的对抗性图像在最严格设置中的感知质量（G-FPR＜b0>），我们使用亚马逊机械土耳其（AMT）进行了用户研究。总共我们收集到 $2,620$ 个参与者的 $77$ 注释。在 $39.14\pm 1.96\%$ （接近随机猜测 $50\%$ ）次试验中，我们SemanticAdv生成的对抗性图像被选为合理的图像，而有 $30.27\pm 1.96\%$ 的试验中由CW选为合理。这表明，相对于CW，在最严格的设置下（G-FPR＜b6>），SemanticAdv可以生成更具感知上可信度的对抗性例子。 相应的图像显示在附加资料中。

One of the key advantages of our SemanticAdv is that we can generate adversarial perturbations in a more controllable fashion guided by the selected semantic attribute. This allows analyzing the robustness of a recognition system against different types of semantic attacks. We group the adversarial examples by augmented attributes in various settings. In Figure 4, we present the attack success rate against two face verification models, namely, R-101-S and R-101-C, using different attributes. We highlight the bar with light blue for G-FPR = $10^{-3}$ and blue for G-FPR = $10^{-4}$, respectively. As shown in Figure 4, with a larger T-FPR = $10^{-3}$, our SemanticAdv can achieve almost 100% attack success rate across different attributes. With a smaller T-FPR = $10^{-4}$, we observe that SemanticAdv guided by some attributes such as Mouth Slightly Open and Arched Eyebrows achieve less than 50% attack success rate, while other attributes such as Pale Skin and Eyeglasses are relatively less affected. In summary, the above experiments indicate that SemanticAdv guided by attributes describing the local shape (e.g., mouth, earrings) achieve a relatively lower attack success rate compared to attributes relevant to the color (e.g., hair color) or entire face region (e.g., skin). This suggests that the face verification models used in our experiments are more robustly trained in terms of detecting local shapes compared to colors. In practice, we have the flexibility to select attributes for attacking an image based on the perceptual quality and attack success rate.
我们的SemanticAdv的一个关键优势是，我们可以更加可控地生成对抗性扰动，通过选择的语义属性进行引导。这样可以分析识别系统对不同类型语义攻击的抗性。我们通过不同设置中的增强属性对对抗性示例进行分组。在图4中，我们展示了针对两个人脸验证模型R-101-S和R-101-C，使用不同属性的攻击成功率。我们分别用浅蓝色代表G-FPR = $10^{-3}$ ，蓝色代表G-FPR = $10^{-4}$ 。如图4所示，当T-FPR = $10^{-3}$ 较大时，我们的SemanticAdv在不同属性上几乎可以达到100%的攻击成功率。当T-FPR = $10^{-4}$ 较小时，我们观察到，由一些属性（如微微张开的嘴和拱形眉毛）引导的SemanticAdv的攻击成功率不到50%，而其他属性（如苍白皮肤和眼镜）受影响相对较小。总之，以上实验表明，由描述局部形状的属性引导的SemanticAdv，例如在攻击成功率方面，与颜色相关的属性（如头发颜色）或整个面部区域（如皮肤）相比，口、耳环等属性实现了相对较低的成功率。这表明我们实验中使用的人脸验证模型在检测局部形状方面的训练比检测颜色更为稳健。在实践中，我们有灵活性根据感知质量和攻击成功率选择攻击图像的属性。

To generate adversarial examples under black-box setting, we analyze the transferability of SemanticAdv in various settings. For each model with different FPRs, we select the successfully attacked adversarial examples from Section 4.1 to construct our evaluation dataset and evaluate these adversarial samples across different models. Table 2(a) illustrates the transferability of SemanticAdv among different models by using the same FPRs (G-FPR = T-FPR = $10^{-3}$). Table 2(b) illustrates the result with different FPRs for generation and evaluation (G-FPR = $10^{-4}$ and T-FPR = $10^{-3}$). As shown in Table 2(a), adversarial examples generated against models trained with softmax loss exhibit certain transferability compared to models trained with cosine loss. We conduct the same experiment by generating adversarial examples with CW and found it has weaker transferability compared to our SemanticAdv (results in brackets of Table 2).
在黑盒设置下生成对抗样本，我们分析了SemanticAdv在不同设置下的可传移性。对于具有不同FPR的每个模型，我们选择从第4.1节成功攻击的对抗样本来构建我们的评估数据集，并在不同模型之间评估这些对抗样本。表2(a)通过使用相同的FPRs（G-FPR = T-FPR = $10^{-3}$ ）展示了SemanticAdv在不同模型之间的可传移性。表2(b)展示了生成和评估时不同FPR的结果（G-FPR = $10^{-4}$ ，T-FPR = $10^{-3}$ ）。如表2(a)所示，针对使用softmax损失训练的模型生成的对抗样本相对于使用余弦损失训练的模型表现出一定的可传移性。我们通过使用CW生成的对抗样本进行相同实验，发现它相对于我们的SemanticAdv具有较弱的可传移性（表2中括号内的结果）。

As Table 2(b) illustrates, the adversarial examples generated against the model with smaller G-FPR $=10^{-4}$ exhibit strong attack success rate when evaluating the model with larger T-FPR $=10^{-3}$. Especially, we found the adversarial examples generated against R-101-S have the best attack performance on other models. These findings motivate the analysis of the query-free black-box API attack detailed in the following paragraph.
如表2(b)所示，针对具有较小G-FPR $=10^{-4}$ 的模型生成的对抗样本，在评估具有更大T-FPR $=10^{-3}$ 的模型时表现出强大的攻击成功率。特别是我们发现针对R-101-S生成的对抗样本在其他模型上具有最佳的攻击性能。这些发现促使我们对下一段详细介绍的无查询黑盒API攻击进行分析。

In this experiment, we generate adversarial examples against R-101-S with G-FPR $=10^{-3}(\kappa=1.24)$, G-FPR $=10^{-4}(\kappa=0.60)$, and G-FPR $<10^{-4}(\kappa=0.30)$, respectively. We evaluate our algorithm on two industry level face verification APIs, namely, Face++ and AliYun. Since attack transferability has never been explored in concurrent work that generates semantic adversarial examples, we use $\mathcal{L}_{p}$ bounded pixel-wise methods (CW [10], MI-FGSM[16], M-DI${}^{2}$-FGSM[73]) as our baselines. We also introduce a much strong baseline by first performing attribute-conditioned image editing and running CW attack on the editted images, which we refer as and StarGAN+CW. Compared to CW, the latter two devise certain techniques to improve their transferability. We adopt the ensemble version of MI-FGSM[16] following the original paper. As shown in Table 3, our proposed SemanticAdv achieves a much higher attack success rate than the baselines in both APIs under all FPR thresholds (e.g., our adversarial examples generated with G-FPR $<10^{-4}$ achieves $67.69\%$ attack success rate on Face++ platform with T-FPR $=10^{-3}$). In addition, we found that lower G-FPR can achieve higher attack success rate on both APIs within the same T-FPR (see our supplementary material for more details).
在这个实验中，我们分别针对R-101-S生成了针对G-FPR $=10^{-3}(\kappa=1.24)$ 、G-FPR $=10^{-4}(\kappa=0.60)$ 和G-FPR $<10^{-4}(\kappa=0.30)$ 的对抗样本。我们在两个行业级人脸验证API，即Face++和AliYun上评估我们的算法。由于攻击可转移性在生成语义对抗样本的同时从未被探讨，我们使用了基于像素级的受限方法（CW[10]、MI-FGSM[16]、M-DI ${}^{2}$ -FGSM[73])作为我们的基线。我们还引入了一个更强大的基线，首先进行属性条件图像编辑，然后对编辑后的图像运行CW攻击，我们称之为StarGAN+CW。与CW相比，后两者设计了一些技术来提高它们的可转移性。我们采用了MI-FGSM[16]的集成版本，遵循原始论文。如表3所示，我们提出的SemanticAdv在两个API中的所有FPR阈值下都比基线实现了更高的攻击成功率（例如，我们生成的带有G-FPR $<10^{-4}$ 的对抗样本在Face++平台上实现了 $67.69\%$ 的攻击成功率，具有T-FPR $=10^{-3}$ ）。 另外，我们发现在相同的T-FPR下，较低的G-FPR可以实现更高的攻击成功率（详细信息请参阅我们的补充材料）。

We evaluate the strength of the proposed attack by testing against five existing defense methods, namely, Feature squeezing [74], Blurring [38], JPEG [17], AMI [66] and adversarial training [42].
我们通过对五种现有防御方法进行测试来评估所提出攻击的强度，即特征挤压[74]、模糊[38]、JPEG[17]、AMI[66]和对抗训练[42]。

Figure 5 illustrates SemanticAdv is more robust against the pixel-wise defense methods comparing with CW. The same G-FPR and T-FPR are used for evaluation. Both SemanticAdv and CW achieve a high attack success rate when T-FPR = $10^{-3}$, while SemanticAdv marginally outperforms CW when T-FPR goes down to $10^{-4}$. While defense methods have proven to be effective against CW attacks on classifiers trained with ImageNet [36], our results indicate that these methods are still vulnerable in the face verification system with small G-FPR.
图5显示SemanticAdv相较于CW在像素级防御方法上更具鲁棒性。评估时使用相同的G-FPR和T-FPR。当T-FPR = $10^{-3}$ 时，SemanticAdv和CW均取得较高的攻击成功率，而当T-FPR降至 $10^{-4}$ 时，SemanticAdv略优于CW。虽然防御方法已被证明对使用ImageNet训练的分类器的CW攻击有效，但我们的结果表明，这些方法在具有小G-FPR的人脸验证系统中仍然容易受到攻击。

We further evaluate SemanticAdv on attribute-based defense method AMI [66] by constructing adversarial examples for the pretrained VGG-Face [53] in a black-box manner. From adversarial examples generated by R-101-S, we use fc7 as the embedding and select the images with normalized L2 distance (to the corresponding benign images) beyond the threshold defined previously. With the benign and adversarial examples, we first extract attribute witnesses with our aligned face images and then leverage them to build a attribute-steered model. When misclassifying $10\%$ benign inputs into adversarial images, it only correctly identifies $8\%$ adversarial images from SemanticAdv and $12\%$ from CW.
我们进一步通过以黑盒方式构建对预训练的VGG-Face的对抗性示例来评估基于属性的防御方法AMI[66]上的SemanticAdv。从R-101-S生成的对抗性示例中，我们使用fc7作为嵌入，并选择与之前定义的阈值之外的带有归一化L2距离的图像。利用良性和对抗性示例，我们首先从我们对齐的人脸图像中提取属性证人，然后利用它们构建一个属性导向的模型。当将 $10\%$ 良性输入误分类为对抗性图像时，它仅能正确识别 $8\%$ 从SemanticAdv和 $12\%$ 从CW中的对抗性图像。

Moreover, we evaluate SemanticAdv on existing adversarial training based defense (the detailed setting is presented in supplementary materials). We find that accuracy of adversarial training based defense method is 10% against the adversarial examples generated by SemanticAdv, while is 46.7% against the adversarial examples generated by PGD [42]. It indicates that existing adversarial training based defense method is less effective against SemanticAdv, which further demonstrates that our SemanticAdv identifies an unexplored research area beyond previous $L_{p}$-based ones.
此外，我们对现有基于对抗训练的防御方法SemanticAdv进行评估（详细设置见补充材料）。我们发现，对抗训练的防御方法在面对由SemanticAdv生成的对抗样本时准确率仅为10%，而在面对由PGD [42]生成的对抗样本时为46.7%。这表明现有基于对抗训练的防御方法对于SemanticAdv的应对效果较差，进一步证明了我们的SemanticAdv在之前基于 $L_{p}$ 的研究领域之外发现了一个未被探索的研究领域。