这里面写的非常详细
http://www.itdadao.com/articles/c15a1401577p0.html
看了网上N多的教程,发现mnist的教程的数据都是官网已经制作好的,那么如果我们自己有数字图片,我们该怎么利用tensoeflow制作数据呢?
现在我有6万张训练集,1万张测试集,下载地址在这mnist图片数据下载:http://pan.baidu.com/s/1pLMV4Kz
首先我们需要有图片数据的txt表,以及对应的标签,如下所示,制作txt表在caffe中已经提到,传送门
mnist/train/5/00000.png 5
mnist/train/0/00001.png 0
mnist/train/4/00002.png 4
mnist/train/1/00003.png 1 下面这串代码就可以在原路径得到a.tfrecords文件
import numpy as np
import cv2
import tensorflow as tf
resize_height=28 #存储图片高度
resize_width=28 #存储图片宽度
train_file_root = '/home/hjxu/PycharmProjects/tf_examples/hjxu_mnist/mnist_img_data'
train_file = train_file_root+'/train.txt' #trainfile是txt文件存放的目录
def _int64_feature(value):#将value转化成int64字节属性,
return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))
def _bytes_feature(value):#将value转化成bytes属性
return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
def load_file(examples_list_file):
# type: (object) -> object
lines = np.genfromtxt(examples_list_file, delimiter=" ", dtype=[('col1', 'S120'), ('col2', 'i8')])
examples = []
labels = []
for example, label in lines:
examples.append(example)
labels.append(label)
return np.asarray(examples), np.asarray(labels), len(lines)
##load_file函数返回的examples,labels,lines,比如examples[0]指的是mnist/train/4/00002.png,也就是 txt的路径 labels[0]返回的是对应的label值是4
def extract_image(filename, resize_height, resize_width): #这边调用cv2.imread()来读取图像,由于cv2读取是BGR格式,需要转换成RGB格式
image = cv2.imread(filename)
image = cv2.resize(image, (resize_height, resize_width))
b,g,r = cv2.split(image)
rgb_image = cv2.merge([r,g,b]) # this is suitable
rgb_image = rgb_image / 255.
rgb_image = rgb_image.astype(np.float32)
return rgb_image
examples, labels, examples_num = load_file(train_file)
writer = tf.python_io.TFRecordWriter('/home/hjxu/PycharmProjects/tf_examples/hjxu_mnist/a.tfrecords')
# root = train_file_root + '/' + examples[0]
for i, [example, label] in enumerate(zip(examples, labels)):
print('No.%d' % (i))
root = train_file_root + '/' + examples[i]
image = extract_image(root, resize_height, resize_width)
a = image.shape
print(root)
print('shape: %d, %d, %d, label: %d' % (image.shape[0], image.shape[1], image.shape[2], label))
image_raw = image.tostring() #将Image转化成字符
example = tf.train.Example(features=tf.train.Features(feature={
'image_raw': _bytes_feature(image_raw),
'height': _int64_feature(image.shape[0]),
'width': _int64_feature(image.shape[1]),
'depth': _int64_feature(image.shape[2]),
'label': _int64_feature(label)
}))
writer.write(example.SerializeToString())
writer.close()
上面代码最重要的是
example = tf.train.Example(features=tf.train.Features(feature={
'image_raw': _bytes_feature(image_raw),
'height': _int64_feature(image.shape[0]),
'width': _int64_feature(image.shape[1]),
'depth': _int64_feature(image.shape[2]),
'label': _int64_feature(label)
}))
这一段,我们可以看出,a.tfrecords里面其实对应的是一些字典,比如Image_raw对应的是图像矩阵本身保存的字节文件,height则是则是对应的高,其实height什么的不写进去也没事,但label一定要写。
现在我们可以得到a.tfrecords这个文件,我们该怎么解析里面的内容呢?或者我们该怎么将tfrecords里面的二进制文件转换成我们可以可视化的数字图片呢
下面这串代码可以得出
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
tfrecord_list_file = '/home/hjxu/PycharmProjects/tf_examples/hjxu_mnist/a.tfrecords'
def read_and_decode(filename_queue,shuffle_batch=True):
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(serialized_example, features={
'image_raw': tf.FixedLenFeature([], tf.string),
'label': tf.FixedLenFeature([], tf.int64)
})
image = tf.decode_raw(features['image_raw'], tf.float32)
image = tf.reshape(image, [28, 28, 3])
image = image * 255.0
labels = features['label']
if shuffle_batch:
images, labels = tf.train.shuffle_batch(
[image,labels],
batch_size=4,
capacity=8000,
num_threads=4,
min_after_dequeue=2000)
else:
images,labels = tf.train.batch([image,labels],
batch_size=4,
capacity=8000,
num_threads=4)
return images,labels
def test_run(tfrecord_filename):
filename_queue = tf.train.string_input_producer([tfrecord_filename],
num_epochs=3)
images,labs = read_and_decode(filename_queue)
init_op = tf.group(tf.global_variables_initializer(),
tf.local_variables_initializer())
# meanfile = sio.loadmat(root_path + 'mats/mean300.mat')
# meanvalue = meanfile['mean'] #如果在制作数据时减去的均值,则需要加上来
with tf.Session() as sess:
sess.run(init_op)
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
for i in range(1):
imgs,labs = sess.run([images,labs])
print 'batch' + str(i) + ': '
#print type(imgs[0])
for j in range(4):
print str(labs[j])
img = np.uint8(imgs[j] )
plt.subplot(4, 2, j * 2 + 1)
plt.imshow(img)
plt.show()
coord.request_stop()
coord.join(threads) #注意,要关闭文件
test_run('/home/hjxu/PycharmProjects/tf_examples/hjxu_mnist/a.tfrecords')
print ("has done")
主要用到tf.decode_raw,这个内置函数的意思是解析 tfrecords文件里的二进制数据,我的read_and_decode只返回图像和label,所以只需要用到tfrecords里面的image_raw和label
image = tf.decode_raw(features['image_raw'], tf.float32) #解析image_raw数据,注意,tf.float32是数据类型,一定要和制作数据时用的类型一样
image = tf.reshape(image, [28, 28, 3])
image = image * 255.0 #我在制作数据时除了255,这里可以补回来或者不补
labels = features['label'] #label则是对应的标签
目前了解的也就这么多,基本都是从其他博客整理得到的,下面是参考博客