TensorFlow构建LSTM来实现时间序列预测（负载预测）

I。前言

我之前写过很多时间序列预测代码，但都是基于PyTorch的。由于目前很多人都在使用TensorFlow，所以上面文章中的代码将在接下来的一段时间内逐步使用TensorFlow实现。

二．数据处理

数据集表示特定区域在特定时间段内的表现。除了负载之外，还包含温度、湿度等信息。

本文尚未考虑其他变量，仅考虑利用历史负荷来预测未来负荷。在本文中，我们将根据前24个时刻的负载计算下一时刻的负载。

代码风格与之前的PyTorch类似：

def nn_seq_us(seq_len, B):
    print('data processing...')
    dataset = load_data()
    # split
    train = dataset[:int(len(dataset) * 0.6)]
    val = dataset[int(len(dataset) * 0.6):int(len(dataset) * 0.8)]
    test = dataset[int(len(dataset) * 0.8):len(dataset)]
    m, n = np.max(train[train.columns[1]]), np.min(train[train.columns[1]])

    def process(data, batch_size, shuffle):
        load = data[data.columns[1]]
        data = data.values.tolist()
        load = (load - n) / (m - n)
        load = load.tolist()
        X, Y = [], []
        for i in range(len(data) - seq_len):
            train_seq = []
            train_label = []
            for j in range(i, i + seq_len):
                x = [load[j]]
                # for c in range(2, 8):
                #     x.append(data[i + 24][c])
                train_seq.append(x)
            train_label.append(load[i + seq_len])
            X.append(train_seq)
            Y.append(train_label)

        X = tf.data.Dataset.from_tensor_slices(X)
        Y = tf.data.Dataset.from_tensor_slices(Y)
        seq = tf.data.Dataset.zip((X, Y))
        if shuffle:
            seq = seq.batch(batch_size, drop_remainder=False).shuffle(batch_size).prefetch(batch_size)
        else:
            seq = seq.batch(batch_size, drop_remainder=False).prefetch(batch_size)

        return seq

    Dtr = process(train, B, shuffle=True)
    Val = process(val, B, shuffle=True)
    Dte = process(test, B, shuffle=False)

之前PyTorch中的批量数据处理：

seq = MyDataset(seq)
seq = DataLoader(dataset=seq, batch_size=batch_size, shuffle=shuffle, num_workers=0, drop_last=False)

TensorFlow中的批量数据处理：

X = tf.data.Dataset.from_tensor_slices(X)
Y = tf.data.Dataset.from_tensor_slices(Y)
seq = tf.data.Dataset.zip((X, Y))
seq = seq.batch(batch_size, drop_remainder=False).prefetch(batch_size)

III。使用TensorFlow创建的模型

LSTM模型如下 表示形式：

class LSTM(keras.Model):
    def __init__(self, args):
        super(LSTM, self).__init__()
        self.lstm = Sequential()
        for i in range(args.num_layers):
            self.lstm.add(layers.LSTM(units=args.hidden_size, input_shape=(args.seq_len, args.input_size),
                                      activation='tanh', return_sequences=True))
        self.fc1 = layers.Dense(64, activation='relu')
        self.fc2 = layers.Dense(args.output_size)

    def call(self, data, training=None, mask=None):
        x = self.lstm(data)
        x = self.fc1(x)
        x = self.fc2(x)

        return x[:, -1:, :]

参数与PyTorch中类似：units❙❙❀代表_ input_shape=(seq_长度，输入大小） , return_sequences=True 表示返回所有时间步的输出。我们只需要输出最后一个时间步。由于keras中的LSTM没有与PyTorch中的LSTM类似的参数num_layers，因此我们需要手动添加。

为了对比，这里是之前用PyTorch定义的LSTM模型：

class LSTM(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, output_size, batch_size):
        super().__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.output_size = output_size
        self.num_directions = 1 # 单向LSTM
        self.batch_size = batch_size
        self.lstm = nn.LSTM(self.input_size, self.hidden_size, self.num_layers, batch_first=True)
        self.linear = nn.Linear(self.hidden_size, self.output_size)

    def forward(self, input_seq):
        batch_size, seq_len = input_seq.shape[0], input_seq.shape[1]
        h_0 = torch.randn(self.num_directions * self.num_layers, self.batch_size, self.hidden_size).to(device)
        c_0 = torch.randn(self.num_directions * self.num_layers, self.batch_size, self.hidden_size).to(device)
        # output(batch_size, seq_len, num_directions * hidden_size)
        output, _ = self.lstm(input_seq, (h_0, c_0)) # output(5, 30, 64)
        pred = self.linear(output)  # (5, 30, 1)
        pred = pred[:, -1, :]  # (5, 1)
        return pred

可以看出两者基本一致。

IV。训练/测试

def train(args, Dtr, Val, Dte, M, path):
    model = LSTM(args)
    if args.optimizer == 'adam':
        optimizer = tf.keras.optimizers.Adam(learning_rate=args.lr)
    else:
        optimizer = tf.keras.optimizers.SGD(learning_rate=args.lr,
                                            momentum=0.9)
    loss_function = tf.keras.losses.MeanSquaredError()
    min_val_loss = 5
    best_model = None
    best_test_mape = 0
    best_test_res = None
    min_epochs = 5
    for epoch in tqdm(range(args.epochs)):
        train_loss = []
        for batch_idx, (seq, label) in enumerate(Dtr):
            with tf.GradientTape() as tape:
                pred = model(seq)
                loss = loss_function(pred, label)
                train_loss.append(loss)
            # 计算梯度
            grads = tape.gradient(loss, model.trainable_variables)
            # 根据梯度更新权重
            optimizer.apply_gradients(zip(grads, model.trainable_variables))

        val_loss, test_mape, res = test(model, Val, Dte, M)
        if epoch + 1 > min_epochs and val_loss < min_val_loss:
            min_val_loss = val_loss
            best_test_mape = test_mape
            best_model = copy.deepcopy(model)
            best_test_res = copy.deepcopy(res)

        print('epoch {:03d} train_loss {:.8f} val_loss {:.8f} test_mape {:.5f}'
              .format(epoch, np.mean(train_loss), val_loss, test_mape))
    best_model.save_weights(path)

    return best_test_mape, best_test_res

训练还会返回在验证集上具有最佳性能的模型。

需要注意的是，TensorFlow中更新模型的过程是：

for batch_idx, (seq, label) in enumerate(Dtr):
    with tf.GradientTape() as tape:
        pred = model(seq)
        loss = loss_function(pred, label)
        train_loss.append(loss)
    # 计算梯度
    grads = tape.gradient(loss, model.trainable_variables)
    # 根据梯度更新权重
    optimizer.apply_gradients(zip(grads, model.trainable_variables))

相反，在PyTorch中是：

for (seq, label) in Dtr:
    seq = seq.to(device)
    label = label.to(device)
    y_pred = model(seq)
    loss = loss_function(y_pred, label)
    train_loss.append(loss.item())
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

保存模型：

best_model.save_weights('models/model')

测试和测试模型：❝0现场MAPE是：

best_test_mape: 0.052237886800780085

画图：