diff --git a/adversarial_robustness/README.md b/adversarial_robustness/README.md
index 95558c5..e6ba2f1 100644
--- a/adversarial_robustness/README.md
+++ b/adversarial_robustness/README.md
@@ -2,6 +2,9 @@
This repository contains the code needed to evaluate models trained in
[Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://arxiv.org/abs/2010.03593)
+(Gowal et al., 2020) and in
+[Fixing Data Augmentation to Improve Adversarial Robustness](https://arxiv.org/abs/2103.01946)
+(Rebuffi et al., 2021).
## Contents
@@ -18,6 +21,7 @@ Download a model from links listed in the following table.
Clean and robust accuracies are measured on the full test set.
The robust accuracy is measured using
[AutoAttack](https://github.com/fra31/auto-attack).
+The following table contains the models from **Gowal et al., 2020**.
| dataset | norm | radius | architecture | extra data | clean | robust | link |
|---|:---:|:---:|:---:|:---:|---:|---:|:---:|
@@ -31,6 +35,18 @@ The robust accuracy is measured using
| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 60.86% | 30.03% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_without.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_without.pt)
| MNIST | ℓ∞ | 0.3 | WRN-28-10 | ✗ | 99.26% | 96.34% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/mnist_linf_wrn28-10_without.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/mnist_linf_wrn28-10_without.pt)
+The following table contains the models from **Rebuffi et al., 2021**.
+
+| dataset | norm | radius | architecture | extra data | clean | robust | link |
+|---|:---:|:---:|:---:|:---:|---:|---:|:---:|
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-106-16 | ✗ | 88.50% | 64.64% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn106-16_cutmix_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn106-16_cutmix_ddpm_v2.pt)
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 88.54% | 64.25% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_cutmix_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_cutmix_ddpm_v2.pt)
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-28-10 | ✗ | 87.33% | 60.75% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_cutmix_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_cutmix_ddpm_v2.pt)
+| CIFAR-10 | ℓ2 | 128 / 255 | WRN-70-16 | ✗ | 92.41% | 80.42% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_l2_wrn70-16_cutmix_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_l2_wrn70-16_cutmix_ddpm_v2.pt)
+| CIFAR-10 | ℓ2 | 128 / 255 | WRN-28-10 | ✗ | 91.79% | 78.80% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_l2_wrn28-10_cutmix_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_l2_wrn28-10_cutmix_ddpm_v2.pt)
+| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 63.56% | 34.64% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_cutmix_ddpm.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_cutmix_ddpm.pt)
+| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-28-10 | ✗ | 62.41% | 32.06% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn28-10_cutmix_ddpm.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn28-10_cutmix_ddpm.pt)
+
### Using the model
Once downloaded, a model can be evaluated (clean accuracy) by running the
@@ -45,8 +61,8 @@ python3 eval.py \
## Citing this work
-If you use this code or these models in your work, please cite the accompanying
-paper:
+If you use this code or these models in your work, please cite the relevant
+accompanying paper:
```
@article{gowal2020uncovering,
@@ -58,6 +74,18 @@ paper:
}
```
+or
+
+```
+@article{rebuffi2021fixing,
+ title={Fixing Data Augmentation to Improve Adversarial Robustness},
+ author={Rebuffi, Sylvestre-Alvise and Gowal, Sven and Calian, Dan A. and Stimberg, Florian and Wiles, Olivia and Mann, Timothy},
+ journal={arXiv preprint arXiv:2103.01946},
+ year={2021},
+ url={https://arxiv.org/pdf/2103.01946}
+}
+```
+
## Disclaimer
This is not an official Google product.
diff --git a/adversarial_robustness/iclrw2021data/README.md b/adversarial_robustness/iclrw2021data/README.md
new file mode 100644
index 0000000..47653cc
--- /dev/null
+++ b/adversarial_robustness/iclrw2021data/README.md
@@ -0,0 +1,61 @@
+# Adversarial Robustness
+
+This repository contains the code needed to evaluate models trained in
+[Data Augmentation Can Improve Robustness](https://storage.googleapis.com/dm-adversarial-robustness/rebuffi2021data.pdf)
+which has been accepted at
+[ICLR 2021 Security and Safety in Machine Learning Systems Workshop](https://aisecure-workshop.github.io/aml-iclr2021/).
+
+
+## Contents
+
+We have released our top-performing models in two formats compatible with
+[JAX](https://github.com/google/jax) and [PyTorch](https://pytorch.org/).
+This repository also contains our model definitions.
+
+## Running the example code
+
+### Downloading a model
+
+Download a model from links listed in the following table.
+Clean and robust accuracies are measured on the full test set.
+The robust accuracy is measured using
+[AutoAttack](https://github.com/fra31/auto-attack).
+
+| dataset | norm | radius | architecture | extra data | clean | robust | link |
+|---|:---:|:---:|:---:|:---:|---:|---:|:---:|
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-70-16 | ✓ | 92.23% | 66.58% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_cutmix_external.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_cutmix_external.pt)
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 87.25% | 60.07% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_cutmix.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_cutmix.pt)
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-28-10 | ✗ | 86.09% | 57.61% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_cutmix.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_cutmix.pt)
+| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 65.76% | 32.43% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_cutmix.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_cutmix.pt)
+| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-28-10 | ✗ | 62.97% | 29.80% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn28-10_cutmix.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn28-10_cutmix.pt)
+
+### Using the model
+
+Once downloaded, a model can be evaluated (clean accuracy) by running the
+`eval.py` script in either the `jax` or `pytorch` folders. E.g.:
+
+```
+cd jax
+python3 eval.py \
+ --ckpt=${PATH_TO_CHECKPOINT} --depth=70 --width=16 --dataset=cifar10
+```
+
+
+## Citing this work
+
+If you use this code or these models in your work, please cite the complete
+version which combines data augmentation with generated samples:
+
+```
+@article{rebuffi2021fixing,
+ title={Fixing Data Augmentation to Improve Adversarial Robustness},
+ author={Rebuffi, Sylvestre-Alvise and Gowal, Sven and Calian, Dan A. and Stimberg, Florian and Wiles, Olivia and Mann, Timothy},
+ journal={arXiv preprint arXiv:2103.01946},
+ year={2021},
+ url={https://arxiv.org/pdf/2103.01946}
+}
+```
+
+## Disclaimer
+
+This is not an official Google product.
diff --git a/adversarial_robustness/iclrw2021doing/README.md b/adversarial_robustness/iclrw2021doing/README.md
new file mode 100644
index 0000000..d6cf694
--- /dev/null
+++ b/adversarial_robustness/iclrw2021doing/README.md
@@ -0,0 +1,62 @@
+# Adversarial Robustness
+
+This repository contains the code needed to evaluate models trained in
+[Doing More with Less: Improving Robustness using Generated Data](https://storage.googleapis.com/dm-adversarial-robustness/gowal2021doing.pdf)
+which has been accepted at
+[ICLR 2021 Security and Safety in Machine Learning Systems Workshop](https://aisecure-workshop.github.io/aml-iclr2021/).
+
+
+## Contents
+
+We have released our top-performing models in two formats compatible with
+[JAX](https://github.com/google/jax) and [PyTorch](https://pytorch.org/).
+This repository also contains our model definitions.
+
+## Running the example code
+
+### Downloading a model
+
+Download a model from links listed in the following table.
+Clean and robust accuracies are measured on the full test set.
+The robust accuracy is measured using
+[AutoAttack](https://github.com/fra31/auto-attack).
+
+| dataset | norm | radius | architecture | extra data | clean | robust | link |
+|---|:---:|:---:|:---:|:---:|---:|---:|:---:|
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 86.94% | 63.62% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_ddpm_v2.pt)
+| CIFAR-10 | ℓ∞ | 8 / 255 | WRN-28-10 | ✗ | 85.97% | 60.73% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_ddpm_v2.pt)
+| CIFAR-10 | ℓ2 | 8 / 255 | WRN-70-16 | ✗ | 90.83% | 78.39% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn70-16_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_l2_wrn70-16_ddpm_v2.pt)
+| CIFAR-10 | ℓ2 | 8 / 255 | WRN-28-10 | ✗ | 90.24% | 77.44% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_linf_wrn28-10_ddpm_v2.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar10_l2_wrn28-10_ddpm_v2.pt)
+| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-70-16 | ✗ | 60.46% | 33.49% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_ddpm.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn70-16_ddpm.pt)
+| CIFAR-100 | ℓ∞ | 8 / 255 | WRN-28-10 | ✗ | 59.18% | 30.81% | [jax](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn28-10_ddpm.npy), [pt](https://storage.googleapis.com/dm-adversarial-robustness/cifar100_linf_wrn28-10_ddpm.pt)
+
+### Using the model
+
+Once downloaded, a model can be evaluated (clean accuracy) by running the
+`eval.py` script in either the `jax` or `pytorch` folders. E.g.:
+
+```
+cd jax
+python3 eval.py \
+ --ckpt=${PATH_TO_CHECKPOINT} --depth=70 --width=16 --dataset=cifar10
+```
+
+
+## Citing this work
+
+If you use this code or these models in your work, please cite the complete
+version which combines data augmentation with generated samples:
+
+```
+@article{rebuffi2021fixing,
+ title={Fixing Data Augmentation to Improve Adversarial Robustness},
+ author={Rebuffi, Sylvestre-Alvise and Gowal, Sven and Calian, Dan A. and Stimberg, Florian and Wiles, Olivia and Mann, Timothy},
+ journal={arXiv preprint arXiv:2103.01946},
+ year={2021},
+ url={https://arxiv.org/pdf/2103.01946}
+}
+```
+
+## Disclaimer
+
+This is not an official Google product.