博客
关于我
一种通用的载入本地数据集的方法
阅读量:252 次
发布时间:2019-03-01

本文共 3385 字,大约阅读时间需要 11 分钟。

??????????

1. ?????????

???????????????????????????????????????????

????????????????????????????????????????2????????????????????????????

2. ??????

???????????????????????????

import syssys.path.append(r'E:\Pycharm\project\yeah&ok\load_data')  from load_data import load_data_func, test_image, augment

??????load_data_func?test_image???

3. ?????????

?????????????????????????????

data_dir = 'E:\Pycharm\project\yeah&ok\dataset'Batch_size = 32  # ?????train_dataset, test_dataset = load_data_func(data_dir, batchsize=Batch_size)test_image(train_dataset)  # ??9???

????????????????????????

4. ?????

????????

import tensorflow as tfimport matplotlib.pyplot as pltimport numpy as npimport pathlibimport randomimport tensorflow_datasets as tfds

5. ?????

??????????????????????????????

def load_data_func(data_dir, batch_size):    data_root = pathlib.Path(data_dir)    all_image_path = list(data_root.glob('*/*'))    all_image_path = [str(path) for path in all_image_path]    random.shuffle(all_image_path)        label_names = sorted(item.name for item in data_root.glob('*/') if item.is_dir())    label_to_index = {name: idx for idx, name in enumerate(label_names)}    print(label_to_index)        all_image_label = [label_to_index[pathlib.Path(p).parent.name] for p in all_image_path]    print(len(all_image_label))        image_path = all_image_path[5]    image_show = (1 + load_preprocess_image(image_path)) / 2.    plt.imshow(image_show)    plt.show()        path_ds = tf.data.Dataset.from_tensor_slices(all_image_path)    image_dataset = path_ds.map(load_preprocess_image)    label_dataset = tf.data.Dataset.from_tensor_slices(all_image_label)    dataset = tf.data.Dataset.zip((image_dataset, label_dataset))        image_count = len(all_image_path)    test_count = int(image_count * 0.2)    train_count = image_count - test_count    print(test_count, train_count)        train_dataset = dataset.skip(test_count).shuffle(buffer_size=150).repeat(3).batch(batch_size)    test_dataset = dataset.take(test_count).batch(batch_size)        train_dataset = train_dataset.map(augment)    return train_dataset, test_dataset

6. ???????

???????????????

def load_preprocess_image(img_path):    img_raw = tf.io.read_file(img_path)    img_tensor = tf.image.decode_jpeg(img_raw, channels=3)    img_tensor = tf.image.resize(img_tensor, [160, 160])    img_tensor = tf.cast(img_tensor, tf.float32)    img = img_tensor / 127.5 - 1    return img

7. ??????

???????????

def augment(image, label):    image = tf.image.random_flip_left_right(image)    image = tf.image.random_contrast(image, lower=0.0, upper=1.0)    image = tf.image.random_flip_up_down(image)    image = tf.image.random_brightness(image, max_delta=0.5)    image = tf.image.random_hue(image, max_delta=0.3)    image = tf.image.random_saturation(image, lower=0.3, upper=0.5)    return image, label

8. ??????

???????????

def test_image(train_dataset):    plt.figure(figsize=(12, 12))    for batch in tfds.as_numpy(train_dataset):        for i in range(9):            image, label = (1 + batch[0][i]) / 2., batch[1][i]            plt.subplot(3, 3, i + 1)            plt.imshow(image)            plt.grid(False)        break    plt.show()

9. ?????

  • ???????????????????????????
  • ???????????????sys.path????????
  • ?????????load_data_func??????????????
  • ???????????????????????????????
  • ???????????????????????????????????
  • ?????????test_image????????????

转载地址:http://whcv.baihongyu.com/

你可能感兴趣的文章
power English (3)原文
查看>>
POWER ENGLISH(7)- repetition
查看>>
SpringBoot中集成SpringBatch详细解析与实战示例(CSV文件读取十万条数据进行业务处理后写入Mysql数据库)
查看>>
powerbi 一张表在另外一张表中出现的数量_PowerBi之初步学习笔记
查看>>
QGIS怎样设置简体中文以及新建可编辑的多边形的图层
查看>>
PowerBuilder 使用自定义事件触发键盘Enter事件
查看>>
PowerCreatorCMS UploadResourcePic 任意文件上传漏洞复现
查看>>
PowerDesigner 使用的一些技巧(转)
查看>>
QGIS在Windows上下载安装与建立空间数据库连接
查看>>
PowerDesigner165安装婆姐汉花教程
查看>>
PowerDesigner使用教程:设置注释、默认值属性
查看>>
PowerDesigner使用教程:不显示背景网格
查看>>
PowerDesigner使用教程:创建数据模型以及导出
查看>>
PowerDesigner使用教程:右侧工具栏显示/隐藏
查看>>
PowerDesigner使用教程:导出sql文件以及解决中文乱码问题
查看>>
PowerDesigner使用教程:时间字段设置
查看>>
PowerDesigner使用教程:给字段添加唯一约束
查看>>
QGIS中怎样设置图层样式并导出地图样式
查看>>
PowerDesigner使用笔记
查看>>
QGIS中怎样实现数据坐标系转换
查看>>