原文链接：https://www.kaggle.com/gzuidhof/full-preprocessing-tutorial/notebook

Introduction

Working with these files can be a challenge, especially given their heterogeneous nature. Some preprocessing is required before they are ready for consumption by your CNN.

Fortunately, I participated in the LUNA16 competition as part of a university course on computer aided diagnosis, so I have some experience working with these files. At this moment we top the leaderboard there :)

This tutorial aims to provide a comprehensive overview of useful steps to take before the data hits your ConvNet/other ML method.

What we will cover:

• Loading the DICOM files, and adding missing metadata
• Converting the pixel values to Hounsfield Units (HU), and what tissue these unit values correspond to
• Resampling to an isomorphic resolution to remove variance in scanner resolution.
• 3D plotting, visualization is very useful to see what we are doing.
• Lung segmentation
• Normalization that makes sense.
• Zero centering the scans.

---

Before we start, let's import some packages and determine the available patients.

英文不咋好，只能意译，没有办法一字不漏的翻译。见谅。如有错误，请大家指正。

介绍

处理这些文件是一种挑战，特别是考虑到他们成分的复杂性。在将他们送进CNN网络之前，一些预处理是必不可少的。

幸运的是，我曾经参加过LNUA16比赛，所以我对处理这些文件有点经验。

这个教程目的在于提供有用的步骤的概览，以便于将这些数据应用与你们的网络。

主要内容：

• 加载DICOM文件，添加缺失的metadata
• 将像素值转换为亨氏单位（肺部组织对应的单位）
• 重采样到同构（同样）分辨率，以消除扫描分辨率的差异。
• 3D绘点（绘图），可视化能让我们明白究竟在做什么。
• 肺部（图片）切割。
• 归一化。
• 零中心化。

Before we start, let's import some packages and determine the available patients.

%matplotlib inlineimport numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import dicom
import os
import scipy.ndimage
import matplotlib.pyplot as pltfrom skimage import measure, morphology
from mpl_toolkits.mplot3d.art3d import Poly3DCollection# Some constants 
INPUT_FOLDER = '../input/sample_images/'
patients = os.listdir(INPUT_FOLDER)
patients.sort()

Loading the files

Dicom is the de-facto file standard in medical imaging. This is my first time working with it, but it seems to be fairly straight-forward. These files contain a lot of metadata (such as the pixel size, so how long one pixel is in every dimension in the real world).

This pixel size/coarseness of the scan differs from scan to scan (e.g. the distance between slices may differ), which can hurt performance of CNN approaches. We can deal with this by isomorphic resampling, which we will do later.

Below is code to load a scan, which consists of multiple slices, which we simply save in a Python list. Every folder in the dataset is one scan (so one patient). One metadata field is missing, the pixel size in the Z direction, which is the slice thickness. Fortunately we can infer this, and we add this to the metadata.

加载文件

DICOM是医疗图像的文件标准。这是我第一次处理DICOM，但事实上处理起来还是相当顺利的。这些文件包含很多metadata(中文翻译：元数据)（例如像素尺寸，所以在现实世界中在每个维度中一个像素有多长呢）。

这些每个扫描文件的像素尺寸各不相同，会影响CNN的表现。后面我们会使用同构重采样解决这个问题。

下面的代码是加载一份扫描文件。扫描文件包含多个切片，我们将其简单地存储在python列表中。数据集中的每个文件夹

都是一份扫描文件（对应一名病人）。缺少一个元数据字段，即Z方向 的像素大小，即片厚。幸运的是我们能推断这一点，并将其添加到元数据中。

# Load the scans in given folder path
def load_scan(path):slices = [dicom.read_file(path + '/' + s) for s in os.listdir(path)]slices.sort(key = lambda x: float(x.ImagePositionPatient[2]))try:slice_thickness = np.abs(slices[0].ImagePositionPatient[2] - slices[1].ImagePositionPatient[2])except:slice_thickness = np.abs(slices[0].SliceLocation - slices[1].SliceLocation)for s in slices:s.SliceThickness = slice_thicknessreturn slices

对于某些边缘情况，这个分割可能会失败。原因在于病人外的空气与肺中的空气不相连。如果病人有气管造口术，情况就不是这样了，我不知道这是否存在于数据集中。另外，对于特别嘈杂的图像（例如下图有起搏器的例子）这种图像也可能失败。相反，身体中的第二大气囊将被分割。你可以通过检查mask对应的图像分数来识别这一点，在这种情况下，将非常小。然后，你可以先用一个尺寸为几毫米的内核进行形态学闭合操作，然后关闭这些孔，然后再工作（或者更简单地说，不要使用这个图像的mask）。

Normalization

Our values currently range from -1024 to around 2000. Anything above 400 is not interesting to us, as these are simply bones with different radiodensity. A commonly used set of thresholds in the LUNA16 competition to normalize between are -1000 and 400. Here's some code you can use:

归一化

值从-1024到接近2000。任何在400以上的都没啥意义，因为那些都是在不同放射线下的骨骼。在LUNA16比赛中一个普遍的用于归一化的阈值集合是-1000到400。下面是代码：

MIN_BOUND = -1000.0
MAX_BOUND = 400.0def normalize(image):image = (image - MIN_BOUND) / (MAX_BOUND - MIN_BOUND)image[image>1] = 1.image[image<0] = 0.return image

Zero centering

As a final preprocessing step, it is advisory to zero center your data so that your mean value is 0. To do this you simply subtract the mean pixel value from all pixels.

To determine this mean you simply average all images in the whole dataset. If that sounds like a lot of work, we found this to be around 0.25 in the LUNA16 competition.

Warning: Do not zero center with the mean per image (like is done in some kernels on here). The CT scanners are calibrated to return accurate HU measurements. There is no such thing as an image with lower contrast or brightness like in normal pictures.

零中心化

作为最后的预处理步骤，一个忠告——零中心化你的数据，这样数据的均值为0。要做这个，只需要减去所有像素的平均像素值即可。

确定平均像素值，你可以对整个数据集求一个平均。如果这个听起来是一个大工程量，那么可以使用我们在LUNA16比赛中发现的差不多的均值0.25。

警告：

不要以每个图像的平均值为中心。CT扫描校准，以返回准确的亨氏测量值。在正常图像中，没有像对比度或亮度一样低的图像。（然而其实不知道说的啥）

PIXEL_MEAN = 0.25def zero_center(image):image = image - PIXEL_MEANreturn image

What's next?

With these steps your images are ready for consumption by your CNN or other ML method :). You can do all these steps offline (one time and save the result), and I would advise you to do so and let it run overnight as it may take a long time.

Tip: To save storage space, don't do normalization and zero centering beforehand, but do this online (during training, just after loading). If you don't do this yet, your image are int16's, which are smaller than float32s and easier to compress as well.

If this tutorial helped you at all, please upvote it and leave a comment :)

下一步做什么？

经过这些步骤，你的图像已经能够送进你的卷积神经网络或其他的机器学习方法了。你能离线（也可能是能够在不进行训练的时候提前做）做这些步骤，我推荐你这样做，并且让他跑一整晚，因为它会执行很长一段时间。

建议：

为了节省存储空间，不要预先做归一化和零中心化，要在加载之后、训练的过程中做。如果你不这样做（指的就是不做预先的归一化和零中心化），你的图像是16位的int,均小于float32，易于压缩。

如果这个教程对你有帮助，请帮我点个赞并且留下个评论吧！