关于深度学习系列笔记十四（中间过程可视化）

必须得实话实说，现在有点懒惰的心理了，随着深度学习的加深，对相关概念的理解和知识也越来越吃力，此外工作上的一些其他事情也牵扯了不少精力；心迟迟无法安定下来，学习和心情是密切相关的，不以物喜不以己悲，估计除了圣人很难做到，很快就到了要决定的时候了，不管怎样，都逃脱不了IT的命啊。

可视化中间激活，是指对于给定输入，展示网络中各个卷积层和池化层输出的特征图（层的输出通常被称为该层的激活，即激活函数的输出）。这让我们可以看到输入如何被分解为网络学到的不同过滤器。我们希望在三个维度对特征图进行可视化：宽度、高度和深度（通道）。每个通道都对应相对独立的特征，所以将这些特征图可视化的正确方法是将每个通道的内容分别绘制成二维图像。

随着层数的加深，层所提取的特征变得越来越抽象。更高的层激活包含关于特定输入的信息越来越少，而关于目标的信息越来越多。

第一层是各种边缘探测器的集合。在这一阶段，激活几乎保留了原始图像中的所有信息。

随着层数的加深，激活变得越来越抽象，并且越来越难以直观地理解。它们开始表示更高层次的概念，比如“猫耳朵”和“猫眼睛”。层数越深，其表示中关于图像视觉内容的信息就越少，而关于类别的信息就越多。

激活的稀疏度（sparsity）随着层数的加深而增大。在第一层里，所有过滤器都被输入图像激活，但在后面的层里，越来越多的过滤器是空白的。也就是说，输入图像中找不到这些过滤器所编码的模式。

代码示例

代码语言：javascript复制

from keras.models import load_model
model = load_model('cats_and_dogs_small_1_withdropout.h5')
#print(model.summary())
#  _________________________________________________________________
#  Layer (type)                 Output Shape              Param #
#  =================================================================
#  conv2d_5 (Conv2D)            (None, 148, 148, 32)      896
#  _________________________________________________________________
#  max_pooling2d_5 (MaxPooling2 (None, 74, 74, 32)        0
#  _________________________________________________________________
#  conv2d_6 (Conv2D)            (None, 72, 72, 64)        18496
#  _________________________________________________________________
#  max_pooling2d_6 (MaxPooling2 (None, 36, 36, 64)        0
#  _________________________________________________________________
#  conv2d_7 (Conv2D)            (None, 34, 34, 128)       73856
#  _________________________________________________________________
#  max_pooling2d_7 (MaxPooling2 (None, 17, 17, 128)       0
#  _________________________________________________________________
#  conv2d_8 (Conv2D)            (None, 15, 15, 128)       147584
#  _________________________________________________________________
#  max_pooling2d_8 (MaxPooling2 (None, 7, 7, 128)         0
#  _________________________________________________________________
#  flatten_2 (Flatten)          (None, 6272)              0
#  _________________________________________________________________
#  dropout_1 (Dropout)          (None, 6272)              0
#  _________________________________________________________________
#  dense_3 (Dense)              (None, 512)               3211776
#  _________________________________________________________________
#  dense_4 (Dense)              (None, 1)                 513
#  =================================================================
#  Total params: 3,453,121
#  Trainable params: 3,453,121
#  Non-trainable params: 0
#  _________________________________________________________________

img_path = 'D:/Python36/Coding/PycharmProjects/ttt/dataset/small/test/cats/cat.1700.jpg'

from keras.preprocessing import image
import numpy as np
import matplotlib.pyplot as plt
img = plt.imread(img_path)
#[[163 173 185]
#  [162 172 184]
#  [160 170 182]
plt.imshow(img)
plt.show()

img = image.load_img(img_path, target_size=(150, 150))
#<PIL.Image.Image image mode=RGB size=150x150 at 0x169210F3EF0>
plt.imshow(img)
plt.show()
img_tensor = image.img_to_array(img)
#(150, 150, 3)
#[[[164. 174. 186.]
#  [161. 171. 183.]
#  [159. 171. 185.]
img_tensor = np.expand_dims(img_tensor, axis=0)
##(1, 150, 150, 3)
##[[[164. 174. 186.]
#  [161. 171. 183.]
#  [159. 171. 185.]
img_tensor /= 255.
#(1, 150, 150, 3)
#[[[[0.6431373  0.68235296 0.7294118 ]
#   [0.6313726  0.67058825 0.7176471 ]
#   [0.62352943 0.67058825 0.7254902 ]
plt.imshow(img_tensor[0])
plt.show()

from keras import models
layer_outputs = [layer.output for layer in model.layers[:8]]
#model.layers=
#[<keras.layers.convolutional.Conv2D object at 0x000001CC9751FEF0>,
#<keras.layers.pooling.MaxPooling2D object at 0x000001CC975315F8>,
#<keras.layers.convolutional.Conv2D object at 0x000001CC9755AE10>,
#<keras.layers.pooling.MaxPooling2D object at 0x000001CC9757F5C0>,
#<keras.layers.convolutional.Conv2D object at 0x000001CC97531438>,
#<keras.layers.pooling.MaxPooling2D object at 0x000001CC9758CF28>,
#<keras.layers.convolutional.Conv2D object at 0x000001CC9757F208>,
#<keras.layers.pooling.MaxPooling2D object at 0x000001CC975B0B70>,
#<keras.layers.core.Flatten object at 0x000001CC9758CE48>,
#<keras.layers.core.Dropout object at 0x000001CC975BDF60>,
#keras.layers.core.Dense object at 0x000001CC975BDEF0>,
#<keras.layers.core.Dense object at 0x000001CC975D3E48>]
#layer_outputs=
#[<tf.Tensor 'conv2d_5/Relu:0' shape=(?, 148, 148, 32) dtype=float32>,
# <tf.Tensor 'max_pooling2d_5/MaxPool:0' shape=(?, 74, 74, 32) dtype=float32>,
# <tf.Tensor 'conv2d_6/Relu:0' shape=(?, 72, 72, 64) dtype=float32>,
# <tf.Tensor 'max_pooling2d_6/MaxPool:0' shape=(?, 36, 36, 64) dtype=float32>,
# <tf.Tensor 'conv2d_7/Relu:0' shape=(?, 34, 34, 128) dtype=float32>,
# <tf.Tensor 'max_pooling2d_7/MaxPool:0' shape=(?, 17, 17, 128) dtype=float32>,
# <tf.Tensor 'conv2d_8/Relu:0' shape=(?, 15, 15, 128) dtype=float32>,
# <tf.Tensor 'max_pooling2d_8/MaxPool:0' shape=(?, 7, 7, 128) dtype=float32>]
#返回8个Numpy数组组成的列表， 每个层激活对应一个 Numpy 数组
activation_model = models.Model(inputs=model.input, outputs=layer_outputs)
#以预测模式运行模型
activations = activation_model.predict(img_tensor)
#len(activations)=8
#activations=[array([[[[3.99328321e-01, 4.74339366e-01, 4.44672465e-01, ...,
#                       2.51882430e-03, 1.12424940e-02, 1.63998783e-01],
#例如，对于输入的猫图像，第一个卷积层的激活如下所示。
first_layer_activation = activations[0]
#它是大小为148×148 的特征图，有32 个通道。
#(1, 148, 148, 32)

import matplotlib.pyplot as plt

for i in range(32):
    plt.matshow(first_layer_activation[0, :, :, i], cmap='viridis')
    plt.show()


#将每个中间激活的所有通道可视化
layer_names = []
for layer in model.layers[:8]:
    layer_names.append(layer.name)
#layer_names=['conv2d_5', 'max_pooling2d_5', 'conv2d_6', 'max_pooling2d_6', 'conv2d_7', 'max_pooling2d_7', 'conv2d_8', 'max_pooling2d_8']

images_per_row = 16
import numpy as np

#np.seterr(divide='ignore',invalid='ignore')

for layer_name, layer_activation in zip(layer_names, activations):
    n_features = layer_activation.shape[-1]
    #32
    size = layer_activation.shape[1]
    #148
    n_cols = n_features // images_per_row
    #2
    display_grid = np.zeros((size * n_cols, images_per_row * size))
    #shape=(296.2368)
    for col in range(n_cols):
       for row in range(images_per_row):
            channel_image = layer_activation[0, :, :, col * images_per_row   row]
            channel_image -= channel_image.mean()
            #channel_image /= channel_image.std()
            channel_image *= 64
            channel_image  = 128
            channel_image = np.clip(channel_image, 0, 255).astype('uint8')
            display_grid[col * size: (col   1) * size, row * size: (row   1) * size] = channel_image
    scale = 1. / size
    plt.figure(figsize=(scale * display_grid.shape[1], scale * display_grid.shape[0]))
    plt.title(layer_name)
    plt.grid(False)
    plt.imshow(display_grid, aspect='auto', cmap='viridis')
plt.show()

原始图片