肝脏超声多任务数据集构建与U-Net+ViT双路径建模

简介本资源为面向医学影像AI研究者与计算机视觉开发者的肝脏超声图像数据集聚焦肝病智能识别与分割任务适用于图像分类、目标检测及语义分割等深度学习模型的训练与验证。数据集共2000个文件全部为结构化JSON格式标注文件完整对应超声图像的肝脏轮廓与病灶区域含良性、恶性、正常三类标签支撑从数据预处理、模型训练到评估分析的全流程开发。压缩包总大小66.65MB含Benign.zip、Malignant.zip、Normal.zip三个子集分别覆盖临床典型病理场景标注规范统一可直接用于PyTorch/TensorFlow框架下的数据加载与训练。目前已有201人学习下载资源附DOI号10.5281/zenodo.7272660并由多位医学影像领域研究者联合构建具备科研可复现性与教学示范价值特别适合开展肝癌辅助诊断算法研发、医学影像课程实验设计及跨模态模型迁移研究。1. 肝脏超声图像数据集不是“带标签的图库”而是面向临床推理闭环的结构化医学视觉基座很多刚接触医学影像的新手会把这份肝脏超声数据集当成普通分类数据集——扔进ResNet训个三分类指标不错就收工。但实际拆开看它根本不是为“单图判别”设计的172.json、205.json这些文件名背后是每张图像对应一份结构化标注JSON里面同时包含肝脏整体轮廓polygon、病灶区域掩膜instance mask、以及按临床逻辑分层的语义标签normal/benign/malignant 质量区域存在性布尔值。这意味着模型必须能联合建模器官解剖结构与病灶空间关系——比如恶性病灶常位于肝右叶后段而良性囊肿多见于左外叶这种空间先验无法靠ImageNet预训练注入。它真正适合的是构建“分割→定位→分类”三级推理链的系统尤其适配U-NetTransformer双路径架构或Mask R-CNN类实例感知模型。如果你正做肝癌早筛算法落地、超声报告自动生成或需要向放射科医生解释模型决策依据如高亮恶性区域并标注其与肝中静脉的空间距离这个数据集提供的多粒度标注才是关键资产。2. 从原始JSON解析到PyTorch Dataset构建支持分割分类双任务的数据加载器2.1 标注JSON结构深度解析与字段映射逻辑每个.json文件并非简单标注而是遵循COCO格式扩展的医学语义结构。以127.json为例核心字段包括{ image_id: 127, file_name: 127.jpg, height: 512, width: 768, liver_contour: [[x1,y1],[x2,y2],...], // 闭合多边形单位像素 lesion_masks: [ { type: cyst, bbox: [x,y,w,h], segmentation: [[x1,y1,x2,y2,...]], // COCO RLE编码或polygon malignancy_score: 0.82 // 连续值非离散标签 } ], diagnosis: malignant, quality_region: [[x1,y1],[x2,y2],...] // 图像质量有效区域排除探头伪影区 }注意lesion_masks数组可能为空正常样本也可能含多个病灶malignancy_score是放射科医生对病灶恶性概率的量化评估需在损失函数中设计回归分支quality_region用于裁剪有效视场避免模型学习到探头边缘噪声。2.2 构建支持多任务的CustomDataset类以下代码实现同时输出图像、肝脏分割掩膜、病灶实例掩膜、诊断标签及质量区域掩膜的四通道标注import json import numpy as np from PIL import Image import torch from torch.utils.data import Dataset from pycocotools.mask import decode as rle_decode class LiverUSDataset(Dataset): def __init__(self, root_dir, splittrain, transformNone): self.root_dir root_dir self.split split self.transform transform # 加载所有JSON文件路径 self.json_files sorted([f for f in os.listdir(root_dir) if f.endswith(.json)]) # 定义类别映射三分类 self.class_map {normal: 0, benign: 1, malignant: 2} def __len__(self): return len(self.json_files) def __getitem__(self, idx): json_path os.path.join(self.root_dir, self.json_files[idx]) with open(json_path, r) as f: ann json.load(f) # 加载图像 img_path os.path.join(self.root_dir, ann[file_name]) image np.array(Image.open(img_path).convert(RGB)) # 初始化掩膜H x W liver_mask np.zeros((ann[height], ann[width]), dtypenp.uint8) lesion_mask np.zeros((ann[height], ann[width]), dtypenp.uint8) quality_mask np.zeros((ann[height], ann[width]), dtypenp.uint8) # 绘制肝脏轮廓多边形填充 if liver_contour in ann and ann[liver_contour]: poly np.array(ann[liver_contour], dtypenp.int32) cv2.fillPoly(liver_mask, [poly], 1) # 绘制病灶掩膜多实例叠加 if lesion_masks in ann: for lesion in ann[lesion_masks]: if segmentation in lesion: # 支持polygon或RLE格式 seg lesion[segmentation] if isinstance(seg[0], list): # polygon poly np.array(seg[0], dtypenp.int32).reshape(-1, 2) cv2.fillPoly(lesion_mask, [poly], 1) else: # RLE rle {size: [ann[height], ann[width]], counts: seg} rle_mask rle_decode(rle) lesion_mask np.maximum(lesion_mask, rle_mask) # 绘制质量区域掩膜 if quality_region in ann and ann[quality_region]: poly np.array(ann[quality_region], dtypenp.int32) cv2.fillPoly(quality_mask, [poly], 1) # 获取诊断标签 label self.class_map.get(ann[diagnosis], 0) # 转换为tensor image torch.from_numpy(image).permute(2, 0, 1).float() / 255.0 liver_mask torch.from_numpy(liver_mask).long() lesion_mask torch.from_numpy(lesion_mask).long() quality_mask torch.from_numpy(quality_mask).long() # 应用数据增强如需 if self.transform: # 注意需同步变换图像和所有掩膜 augmented self.transform(imageimage, masks[liver_mask, lesion_mask, quality_mask]) image augmented[image] liver_mask, lesion_mask, quality_mask augmented[masks] return { image: image, liver_mask: liver_mask, lesion_mask: lesion_mask, quality_mask: quality_mask, label: torch.tensor(label, dtypetorch.long), image_id: ann[image_id] } # 使用示例配合albumentations import albumentations as A from albumentations.pytorch import ToTensorV2 transform A.Compose([ A.Resize(512, 768), A.HorizontalFlip(p0.5), A.RandomBrightnessContrast(p0.2), ToTensorV2() ], additional_targets{masks: mask})2.2.1 关键参数说明与临床适配要点quality_mask的作用远不止过滤伪影在训练时可作为loss权重图weight_map quality_mask * 0.5 (1-quality_mask) * 0.1强制模型忽略低信噪比区域lesion_mask采用np.maximum而非累加避免多病灶重叠处像素值溢出确保二值掩膜纯度malignancy_score未在__getitem__中返回因需单独设计回归head——建议在collate_fn中将其提取为batch[malignancy_scores]配合L1Loss监督transform必须使用additional_targets指定掩膜同步变换否则肝脏轮廓几何畸变将破坏解剖合理性。2.3 数据集划分策略按病例ID而非文件名随机切分超声图像存在强相关性同一患者多次扫描的图像纹理、增益设置、探头角度高度相似。若按文件名随机划分会导致验证集出现训练集已见过的患者特征造成指标虚高。正确做法是解析所有JSON中的patient_id字段若无则根据文件名前缀推断如P001_127.jpg→P001按patient_id聚类确保同一患者的所有图像归属同一split采用分层抽样保证normal/benign/malignant三类在训练/验证/测试集中比例一致如7:2:1。from sklearn.model_selection import train_test_split import pandas as pd # 假设已提取patient_id列表 df pd.DataFrame({ json_file: self.json_files, patient_id: [extract_patient_id(f) for f in self.json_files], diagnosis: [json.load(open(os.path.join(root, f)))[diagnosis] for f in self.json_files] }) # 分层按patient_id分组 train_patients, val_test_patients train_test_split( df[patient_id].unique(), test_size0.3, stratifydf.groupby(patient_id)[diagnosis].first(), random_state42 ) val_patients, test_patients train_test_split( val_test_patients, test_size0.5, stratifydf[df[patient_id].isin(val_test_patients)].groupby(patient_id)[diagnosis].first(), random_state42 )提示实际项目中需检查patient_id字段是否存在于原始JSON——本数据集摘要未明确提及但Zenodo DOI页面的元数据文档应包含该信息。若缺失可联系作者补全或通过图像哈希聚类近似还原。3. 双路径模型设计U-Net主干ViT分类头的联合训练框架3.1 网络架构选择依据为何不用纯CNN或纯Transformer肝脏超声图像存在两大挑战低对比度与斑点噪声传统CNN易受噪声干扰导致肝脏边界模糊病灶尺度差异大微小囊肿5mm与巨大转移瘤50mm共存单一感受野难以兼顾。U-Net通过跳跃连接保留空间细节但全局上下文建模弱ViT擅长长程依赖但对局部纹理敏感度不足。因此采用U-Net主干提取多尺度特征 ViT分类头聚合全局语义的混合架构既保障分割精度又提升分类鲁棒性。3.1.1 U-Net主干改造引入可变形卷积与注意力门控标准U-Net在超声场景下易产生“阶梯状”分割边界。我们替换所有3×3卷积为可变形卷积Deformable Convolution v2并在跳跃连接处添加注意力门控Attention Gateimport torch.nn as nn import torch.nn.functional as F class AttentionGate(nn.Module): def __init__(self, gating_channels, inter_channels, input_channels): super().__init__() self.W_g nn.Sequential( nn.Conv2d(gating_channels, inter_channels, kernel_size1), nn.BatchNorm2d(inter_channels) ) self.W_x nn.Sequential( nn.Conv2d(input_channels, inter_channels, kernel_size1), nn.BatchNorm2d(inter_channels) ) self.psi nn.Sequential( nn.Conv2d(inter_channels, 1, kernel_size1), nn.BatchNorm2d(1), nn.Sigmoid() ) def forward(self, g, x): # g: gating signal (decoder feature), x: input feature (encoder skip) g1 self.W_g(g) x1 self.W_x(x) psi self.psi(F.relu(g1 x1)) return x * psi # apply attention mask # 在U-Net decoder block中插入 class UpBlock(nn.Module): def __init__(self, in_ch, out_ch): super().__init__() self.up nn.Upsample(scale_factor2, modebilinear, align_cornersTrue) self.attention AttentionGate(gating_channelsin_ch//2, inter_channelsin_ch//4, input_channelsin_ch//2) self.conv nn.Sequential( nn.Conv2d(in_ch, out_ch, 3, padding1), nn.ReLU(inplaceTrue), nn.Conv2d(out_ch, out_ch, 3, padding1), nn.ReLU(inplaceTrue) ) def forward(self, x, skip): x self.up(x) skip self.attention(x, skip) # gated skip connection x torch.cat([x, skip], dim1) return self.conv(x)3.1.2 ViT分类头设计基于肝脏ROI的区域级token聚合直接将ViT应用于整图会稀释病灶信号。我们先用U-Net输出的liver_mask裁剪肝脏ROI再在此区域内提取patchdef extract_liver_roi(features, liver_mask, patch_size16): features: (B, C, H, W) — U-Net encoder最后层特征 liver_mask: (B, H, W) — 二值掩膜 B, C, H, W features.shape # 将liver_mask上采样至feature尺寸 liver_mask F.interpolate(liver_mask.unsqueeze(1).float(), size(H, W), modenearest).squeeze(1) # 提取肝脏区域特征mask为1的位置 roi_features [] for b in range(B): mask_idx torch.where(liver_mask[b] 1) if len(mask_idx[0]) 0: # 无肝脏区域取全图中心patch center_h, center_w H//2, W//2 patch features[b, :, center_h-patch_size//2:center_hpatch_size//2, center_w-patch_size//2:center_wpatch_size//2] else: # 随机采样N个点避免过拟合 idx torch.randint(0, len(mask_idx[0]), (16,)) h_idx, w_idx mask_idx[0][idx], mask_idx[1][idx] # 提取以这些点为中心的patch patches [] for i in range(len(h_idx)): h, w h_idx[i].item(), w_idx[i].item() h0, h1 max(0, h-patch_size//2), min(H, hpatch_size//2) w0, w1 max(0, w-patch_size//2), min(W, wpatch_size//2) patch features[b, :, h0:h1, w0:w1] if patch.shape[1:] ! (patch_size, patch_size): patch F.interpolate(patch.unsqueeze(0), size(patch_size, patch_size), modebilinear).squeeze(0) patches.append(patch) patch torch.stack(patches) roi_features.append(patch) return torch.stack(roi_features) # (B, 16, C, P, P) # ViT head输入(B, N, C*P*P) - (B, N, D) class ViTClassifier(nn.Module): def __init__(self, in_dim, num_classes3, depth4, heads4, mlp_dim256): super().__init__() self.patch_embed nn.Linear(in_dim * 16 * 16, 512) # P16 self.pos_encoding nn.Parameter(torch.randn(1, 16, 512)) self.transformer nn.TransformerEncoder( nn.TransformerEncoderLayer(d_model512, nheadheads, dim_feedforwardmlp_dim), num_layersdepth ) self.cls_head nn.Sequential( nn.LayerNorm(512), nn.Linear(512, num_classes) ) def forward(self, x): # x: (B, 16, C, 16, 16) - (B, 16, C*256) x x.flatten(2).flatten(2) # (B, 16, C*256) x self.patch_embed(x) self.pos_encoding x self.transformer(x) x x.mean(dim1) # global average pooling over patches return self.cls_head(x)3.2 多任务损失函数设计分割与分类的梯度协同单纯加权求和loss 0.7*seg_loss 0.3*cls_loss会导致梯度冲突。我们采用不确定性加权损失Uncertainty Weighting让网络自动学习各任务置信度class MultiTaskLoss(nn.Module): def __init__(self): super().__init__() # 可学习参数初始化为0 → exp(0)1初始权重相等 self.log_var_seg nn.Parameter(torch.zeros(1)) self.log_var_cls nn.Parameter(torch.zeros(1)) def forward(self, seg_pred, seg_target, cls_pred, cls_target): # 分割损失Dice BCE dice_loss 1 - dice_coefficient(seg_pred, seg_target) bce_loss F.binary_cross_entropy_with_logits(seg_pred, seg_target.float()) seg_loss dice_loss bce_loss # 分类损失 cls_loss F.cross_entropy(cls_pred, cls_target) # 不确定性加权 precision_seg torch.exp(-self.log_var_seg) precision_cls torch.exp(-self.log_var_cls) loss precision_seg * seg_loss self.log_var_seg \ precision_cls * cls_loss self.log_var_cls return loss def dice_coefficient(y_pred, y_true, smooth1e-6): y_pred torch.sigmoid(y_pred) intersection (y_pred * y_true).sum(dim(1,2,3)) union y_pred.sum(dim(1,2,3)) y_true.sum(dim(1,2,3)) return (2. * intersection smooth) / (union smooth)注意log_var_seg和log_var_cls在训练初期会快速下降提高对应任务权重但需监控其值——若log_var_cls持续低于-3说明分类任务过拟合应增加DropPath率或添加标签平滑。4. 模型验证与临床可信度评估超越Accuracy的三维评价体系4.1 分割性能评估必须报告肝脏与病灶的独立Dice系数许多论文仅报告整体Dice但临床要求区分两类目标肝脏轮廓分割影响后续病灶定位精度要求Dice ≥ 0.92病灶分割直接关联诊断需按类型分报囊肿/血管瘤/转移瘤且要求病灶中心点定位误差 5mm。使用monai.metrics.compute_meandice计算时必须传入include_backgroundFalse并指定to_onehot_yTruefrom monai.metrics import compute_meandice # pred: (B, 2, H, W) — [liver, lesion] # target: (B, 2, H, W) — one-hot encoded dice_metrics compute_meandice( y_predpred, ytarget, include_backgroundFalse, to_onehot_yTrue, mutually_exclusiveFalse # 因liver与lesion可重叠 ) # dice_metrics[0] liver Dice, dice_metrics[1] lesion Dice4.2 分类结果可解释性Grad-CAM热力图与病灶空间关系分析单纯输出malignant标签无法满足临床需求。需生成两层解释像素级热力图显示模型关注区域是否覆盖真实病灶解剖关系报告统计病灶相对于肝中静脉MHV、门静脉左支LPV的空间位置。# Grad-CAM实现针对ViT分类头 def generate_cam(model, img_tensor, target_layertransformer.layers.3): model.eval() with torch.no_grad(): features model.encoder(img_tensor) # U-Net encoder输出 roi_features extract_liver_roi(features, liver_mask) # 前文函数 cls_token model.vit_head.patch_embed(roi_features.flatten(2)) # (B,16,D) # 获取最后一层Transformer的attention权重 attn_weights model.vit_head.transformer.layers[-1].self_attn.attn # (B,heads,N,N) # 取cls token对所有patch的平均注意力 cam attn_weights.mean(dim1)[:, 0, 1:] # (B,16) cam cam.reshape(-1, 4, 4) # 16 patches → 4x4 grid cam F.interpolate(cam.unsqueeze(1), size(512,768), modebilinear) return cam # 空间关系分析需预定义解剖标志点坐标 def analyze_anatomical_relation(lesion_mask, landmark_points): landmark_points: {MHV: (x,y), LPV: (x,y)} 返回病灶质心到各标志点的距离mm及相对象限 cy, cx ndimage.center_of_mass(lesion_mask) relations {} for name, (lx, ly) in landmark_points.items(): dist np.sqrt((cx-lx)**2 (cy-ly)**2) * 0.25 # 假设0.25mm/pixel relations[f{name}_distance_mm] dist # 象限判断以MHV为原点 if name MHV: quadrant RL if cx lx else LL relations[quadrant] quadrant return relations4.3 部署前必做的三项压力测试测试项方法合格阈值临床意义增益鲁棒性对验证集图像批量调整增益-20dB ~ 20dB分类准确率波动 ≤ 5%避免不同设备参数导致误判探头角度泛化使用OpenCV模拟旋转±15° 仿射变换病灶Dice下降 ≤ 0.08适应不同扫查体位小病灶检出率专门构建含≤3mm病灶的子集需人工复核敏感度 ≥ 75%满足早期肝癌筛查要求执行增益测试的代码示例def test_gain_robustness(model, dataloader, gain_range(-20, 20)): model.eval() acc_list [] for gain_db in range(gain_range[0], gain_range[1]1, 5): acc 0 for batch in dataloader: img batch[image] # 模拟增益调整超声图像强度对数变换 img_linear torch.pow(10, img * 2 - 2) # 反归一化 img_gain img_linear * (10 ** (gain_db / 20)) img_norm torch.log10(torch.clamp(img_gain, 1e-6, 1e6)) / 2 1 # 重归一化 pred model(img_norm.cuda()) acc (pred.argmax(dim1) batch[label].cuda()).float().mean() acc_list.append(acc.item() / len(dataloader)) return np.array(acc_list) # 要求max(acc_list) - min(acc_list) 0.05提示增益测试中img_linear torch.pow(10, img * 2 - 2)基于超声图像强度服从对数正态分布的物理特性此转换比简单线性缩放更符合真实设备行为。本文还有配套的精品资源点击获取