玩转pytorch中的torchvision.transforms

文章作者:Tyan
博客:noahsnail.com  |  CSDN  |  简书

0. 运行环境

python 3.6.8, pytorch 1.5.0

1. torchvision.transforms

在深度学习中,计算机视觉(CV)是其中的一大方向,而在CV任务中,图像变换(Image Transform)通常是必不可少的一环,其可以用来对图像进行预处理,数据增强等。本文主要整理PyTorch中torchvision.transforms提供的一些功能(代码加示例)。具体定义及参数可参考PyTorch文档

1.1 torchvision.transforms.Compose

Compose的主要作用是将多个变换组合在一起,具体用法可参考2.5。下面的示例结果左边为原图,右边为保存的结果。

2. Transforms on PIL Image

这部分主要是对Python最常用的图像处理库Pillow中Image的处理。基本环境及图像如下:

1
2
3
4
5
6
7
8
9
10
import torchvision.transforms as transforms
from PIL import Image
img = Image.open('tina.jpg')
...
# Save image
img.save('image.jpg')

Demo

2.1 torchvision.transforms.CenterCrop(size)

CenterCrop的作用是从图像的中心位置裁剪指定大小的图像。例如一些神经网络的输入图像大小为224*224,而训练图像的大小为256*256,此时就需要对训练图像进行裁剪。示例代码及结果如下:

1
2
3
4
size = (224, 224)
transform = transforms.CenterCrop(size)
center_crop = transform(img)
center_crop.save('center_crop.jpg')

CenterCrop

2.2 torchvision.transforms.ColorJitter(brightness=0, contrast=0, saturation=0, hue=0)

ColorJitter的作用是随机修改图片的亮度、对比度和饱和度,常用来进行数据增强,尤其是训练图像类别不均衡或图像数量较少时。示例代码及结果如下:

1
2
3
4
5
6
7
brightness = (1, 10)
contrast = (1, 10)
saturation = (1, 10)
hue = (0.2, 0.4)
transform = transforms.ColorJitter(brightness, contrast, saturation, hue)
color_jitter = transform(img)
color_jitter.save('color_jitter.jpg')

ColorJitter

2.3 torchvision.transforms.FiveCrop(size)

FiveCrop的作用是分别从图像的四个角以及中心进行五次裁剪,图像分类评估时分为Singl Crop Evaluation/TestMulti Crop Evaluation/TestFiveCrop可以用在Multi Crop Evaluation/Test中。示例代码及结果如下:

1
2
3
size = (224, 224)
transform = transforms.FiveCrop(size)
five_crop = transform(img)

FiveCrop

2.4 torchvision.transforms.Grayscale(num_output_channels=1)

Grayscale的作用是将图像转换为灰度图像,默认通道数为1,通道数为3时,RGB三个通道的值相等。示例代码及结果如下:

1
2
3
transform = transforms.Grayscale()
grayscale = transform(img)
grayscale.save('grayscale.jpg')

Grayscale

2.5 torchvision.transforms.Pad(padding, fill=0, padding_mode=’constant’)

Pad的作用是对图像进行填充,可以设置要填充的值及填充的大小,默认是图像四边都填充。示例代码及结果如下:

1
2
3
4
5
6
7
8
9
size = (224, 224)
padding = 16
fill = (0, 0, 255)
transform = transforms.Compose([
transforms.CenterCrop(size),
transforms.Pad(padding, fill)
])
pad = transform(img)
pad.save('pad.jpg')

Pad

2.6 torchvision.transforms.RandomAffine(degrees, translate=None, scale=None, shear=None, resample=False, fillcolor=0)

RandomAffine的作用是保持图像中心不变的情况下对图像进行随机的仿射变换。示例代码及结果如下:

1
2
3
4
5
6
7
degrees = (15, 30)
translate=(0, 0.2)
scale=(0.8, 1)
fillcolor = (0, 0, 255)
transform = transforms.RandomAffine(degrees=degrees, translate=translate, scale=scale, fillcolor=fillcolor)
random_affine = transform(img)
random_affine.save('random_affine.jpg')

RandomAffine

2.7 torchvision.transforms.RandomApply(transforms, p=0.5)

RandomApply的作用是以一定的概率执行提供的transforms操作,即可能执行,也可能不执行。transforms可以是一个,也可以是一系列。示例代码及结果如下:

1
2
3
4
5
6
size = (224, 224)
padding = 16
fill = (0, 0, 255)
transform = transforms.RandomApply([transforms.CenterCrop(size), transforms.Pad(padding, fill)])
for i in range(3):
random_apply = transform(img)

RandomApply

2.8 torchvision.transforms.RandomChoice(transforms)

RandomChoice的作用是从提供的transforms操作中随机选择一个执行。示例代码及结果如下:

1
2
3
4
5
6
7
size = (224, 224)
padding = 16
fill = (0, 0, 255)
degrees = (15, 30)
transform = transforms.RandomChoice([transforms.RandomAffine(degrees), transforms.CenterCrop(size), transforms.Pad(padding, fill)])
for i in range(3):
random_choice = transform(img)

RandomChoice

2.9 torchvision.transforms.RandomCrop(size, padding=None, pad_if_needed=False, fill=0, padding_mode=’constant’)

RandomCrop的作用是在一个随机位置上对图像进行裁剪。示例代码及结果如下:

1
2
3
size = (224, 224)
transform = transforms.RandomCrop(size)
random_crop = transform(img)

RandomCrop

2.10 torchvision.transforms.RandomGrayscale(p=0.1)

RandomGrayscale的作用是以一定的概率将图像变为灰度图像。示例代码及结果如下:

1
2
3
4
p = 0.5
transform = transforms.RandomGrayscale(p)
for i in range(3):
random_grayscale = transform(img)

RandomGrayscale

2.11 torchvision.transforms.RandomHorizontalFlip(p=0.5)

RandomHorizontalFlip的作用是以一定的概率对图像进行水平翻转。示例代码及结果如下:

1
2
3
4
p = 0.5
transform = transforms.RandomHorizontalFlip(p)
for i in range(3):
random_horizontal_filp = transform(img)

RandomHorizontalFlip

2.12 torchvision.transforms.RandomOrder(transforms)

RandomOrder的作用是以随机顺序执行提供的transforms操作。示例代码及结果如下:

1
2
3
4
5
6
7
size = (224, 224)
padding = 16
fill = (0, 0, 255)
degrees = (15, 30)
transform = transforms.RandomOrder([transforms.RandomAffine(degrees), transforms.CenterCrop(size), transforms.Pad(padding, fill)])
for i in range(3):
random_order = transform(img)

RandomOrder

2.13 torchvision.transforms.RandomPerspective(distortion_scale=0.5, p=0.5, interpolation=3, fill=0)

RandomPerspective的作用是以一定的概率对图像进行随机的透视变换。示例代码及结果如下:

1
2
3
4
5
6
distortion_scale = 0.5
p = 1
fill = (0, 0, 255)
transform = transforms.RandomPerspective(distortion_scale=distortion_scale, p=p, fill=fill)
random_perspective = transform(img)
random_perspective.save('random_perspective.jpg')

RandomPerspective

2.14 torchvision.transforms.RandomResizedCrop(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=2)

RandomResizedCrop的作用是以随机大小和随机长宽比裁剪图像并缩放到指定的大小。示例代码及结果如下:

1
2
3
4
5
6
size = (256, 256)
scale=(0.8, 1.0)
ratio=(0.75, 1.0)
transform = transforms.RandomResizedCrop(size=size, scale=scale, ratio=ratio)
random_resized_crop = transform(img)
random_resized_crop.save('random_resized_crop.jpg')

RandomResizedCrop

2.15 torchvision.transforms.RandomRotation(degrees, resample=False, expand=False, center=None, fill=None)

RandomRotation的作用是对图像进行随机旋转。示例代码及结果如下:

1
2
3
4
5
degrees = (15, 30)
fill = (0, 0, 255)
transform = transforms.RandomRotation(degrees=degrees, fill=fill)
random_rotation = transform(img)
random_rotation.save('random_rotation.jpg')

RandomRotation

2.16 torchvision.transforms.RandomSizedCrop(args, *kwargs)

已废弃,参见RandomResizedCrop

2.17 torchvision.transforms.RandomVerticalFlip(p=0.5)

RandomVerticalFlip的作用是以一定的概率对图像进行垂直翻转。示例代码及结果如下:

1
2
3
4
p = 1
transform = transforms.RandomVerticalFlip(p)
random_vertical_filp = transform(img)
random_vertical_filp.save('random_vertical_filp.jpg')

RandomVerticalFlip

2.18 torchvision.transforms.Resize(size, interpolation=2)

Resize的作用是对图像进行缩放。示例代码及结果如下:

1
2
3
4
size = (224, 224)
transform = transforms.Resize(size)
resize_img = transform(img)
resize_img.save('resize_img.jpg')

Resize

2.19 torchvision.transforms.Scale(args, *kwargs)

已废弃,参加Resize

2.20 torchvision.transforms.TenCrop(size, vertical_flip=False)

TenCrop与2.3类似,除了对原图裁剪5个图像之外,还对其翻转图像裁剪了5个图像。

3. Transforms on torch.*Tensor

3.1 torchvision.transforms.LinearTransformation(transformation_matrix, mean_vector)

LinearTransformation的作用是使用变换矩阵和离线计算的均值向量对图像张量进行变换,可以用在白化变换中,白化变换用来去除输入数据的冗余信息。常用在数据预处理中。

3.2 torchvision.transforms.Normalize(mean, std, inplace=False)

Normalize的作用是用均值和标准差对Tensor进行归一化处理。常用在对输入图像的预处理中,例如Imagenet竞赛的许多分类网络都对输入图像进行了归一化操作。

3.3 torchvision.transforms.RandomErasing(p=0.5, scale=(0.02, 0.33), ratio=(0.3, 3.3), value=0, inplace=False)

RandomErasing的作用是随机选择图像中的一块区域,擦除其像素,主要用来进行数据增强。示例代码及结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
p = 1.0
scale = (0.2, 0.3)
ratio = (0.5, 1.0)
value = (0, 0, 255)
transform = transforms.Compose([
transforms.ToTensor(),
transforms.RandomErasing(p=p, scale=scale, ratio=ratio, value=value),
transforms.ToPILImage()
])
random_erasing = transform(img)
random_erasing.save('random_erasing.jpg')

RandomErasing

4 Conversion Transforms

4.1 torchvision.transforms.ToPILImage(mode=None)

ToPILImage的作用是将pytorch的Tensornumpy.ndarray转为PIL的Image。示例代码及结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
img = Image.open('tina.jpg')
transform = transforms.ToTensor()
img = transform(img)
print(img.size())
img_r = img[0, :, :]
img_g = img[1, :, :]
img_b = img[2, :, :]
print(type(img_r))
print(img_r.size())
transform = transforms.ToPILImage()
img_r = transform(img_r)
img_g = transform(img_g)
img_b = transform(img_b)
print(type(img_r))
img_r.save('img_r.jpg')
img_g.save('img_g.jpg')
img_b.save('img_b.jpg')
# output
torch.Size([3, 256, 256])
<class 'torch.Tensor'>
torch.Size([256, 256])
<class 'PIL.Image.Image'>

ToPILImage

4.2 torchvision.transforms.ToTensor

ToTensor的作用是将PIL Imagenumpy.ndarray转为pytorch的Tensor,并会将像素值由[0, 255]变为[0, 1]之间。通常是在神经网络训练中读取输入图像之后使用。示例代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
img = Image.open('tina.jpg')
print(type(img))
print(img.size)
transform = transforms.ToTensor()
img = transform(img)
print(type(img))
print(img.size())
# output
<class 'PIL.JpegImagePlugin.JpegImageFile'>
(256, 256)
<class 'torch.Tensor'>
torch.Size([3, 256, 256])

5. Code

代码参见https://github.com/SnailTyan/deep-learning-tools/blob/master/transforms.py

References

  1. https://pytorch.org/docs/stable/torchvision/transforms.html
如果有收获,可以请我喝杯咖啡!