我以列表的形式标记了数据形状不均 arrays:
array([array([1179, 6, 208, 2, 1625, 92, 9, 3870, 3, 2136, 435,
5, 2453, 2180, 44, 1, 226, 166, 3, 4409, 49, 6728,
...
10, 17, 1396, 106, 8002, 7968, 111, 33, 1130, 60, 181,
7988, 7974, 7970])], dtype=object)
以及各自的目标:
Out[74]: array([0, 0, 0, ..., 0, 0, 1], dtype=object)
我正在尝试将它们改造成有衬垫的tf.data.Dataset()
,但它不允许我将不等的形状转换为张量。我会得到这个错误:
ValueError: Can't convert non-rectangular Python sequence to Tensor.
完整的代码在这里。假设我的起点是之后y = ...
:
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np
(train_data, test_data) = tfds.load('imdb_reviews/subwords8k',
split=(tfds.Split.TRAIN, tfds.Split.TEST),
as_supervised=True)
x = np.array(list(train_data.as_numpy_iterator()))[:, 0]
y = np.array(list(train_data.as_numpy_iterator()))[:, 1]
train_tensor = tf.data.Dataset.from_tensor_slices((x.tolist(), y))\
.padded_batch(batch_size=8, padded_shapes=([None], ()))
我有哪些选择可以将其变成填充批次 tensor?