Hands-on Practical: Time Series Forecasting

Let's apply the concepts we've discussed to a practical time series forecasting problem. We'll use an LSTM network to predict future values of a synthetic sine wave. This exercise demonstrates the core workflow: preparing sequential data, building a recurrent model, training it, and evaluating its predictions. While we use a synthetic dataset for clarity and reproducibility, the principles apply directly to real-world time series like weather patterns, stock prices, or sensor readings.

We'll use TensorFlow with the Keras API for this example. Make sure you have TensorFlow and NumPy installed.

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import matplotlib.pyplot as plt # Using Matplotlib temporarily for internal visualization if needed, will use Plotly for output

# Set random seed for reproducibility
tf.random.set_seed(42)
np.random.seed(42)

1. Generating and Preparing the Data

First, we create our time series data, a sine wave with some added noise to make it slightly more challenging.

# Generate sine wave data
time = np.arange(0, 100, 0.1)
amplitude = np.sin(time) + np.random.normal(scale=0.15, size=len(time))

# Visualize the generated data

Generated sine wave with added random noise. We will use this time series for our forecasting task.

Before feeding data into a neural network, it's standard practice to scale it, typically to a range between 0 and 1 or -1 and 1. This helps the optimization algorithm converge more effectively. We'll use MinMaxScaler.

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler(feature_range=(0, 1))
# Reshape data for scaler: (n_samples, n_features)
amplitude_scaled = scaler.fit_transform(amplitude.reshape(-1, 1))

Now, we need to transform this single time series into input/output pairs suitable for supervised learning. We'll use a sliding window approach: a sequence of past values (window_size) will be the input ($X$), and the single next value will be the output ($y$).

def create_sequences(data, window_size):
    X, y = [], []
    for i in range(len(data) - window_size):
        X.append(data[i:(i + window_size), 0])
        y.append(data[i + window_size, 0])
    return np.array(X), np.array(y)

window_size = 50 # Number of past time steps to use as input features
X, y = create_sequences(amplitude_scaled, window_size)

# Split data into training and testing sets (e.g., 80% train, 20% test)
train_size = int(len(X) * 0.8)
X_train, X_test = X[:train_size], X[train_size:]
y_train, y_test = y[:train_size], y[train_size:]

print(f"Original data length: {len(amplitude)}")
print(f"Scaled data length: {len(amplitude_scaled)}")
print(f"Number of sequences (X): {len(X)}")
print(f"Training sequences: {len(X_train)}, Test sequences: {len(X_test)}")

LSTMs (and other RNN layers in TensorFlow/Keras) expect input data in a specific 3D shape: (batch_size, timesteps, features).

• batch_size: Number of sequences processed at once during training.
• timesteps: Length of each input sequence (our window_size).
• features: Number of features observed at each timestep. Since we have a univariate time series (only amplitude), this is 1.

# Reshape X for LSTM input: [samples, time steps, features]
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

print(f"X_train shape: {X_train.shape}") # Should be (num_train_samples, window_size, 1)
print(f"X_test shape: {X_test.shape}")   # Should be (num_test_samples, window_size, 1)
print(f"y_train shape: {y_train.shape}") # Should be (num_train_samples,)
print(f"y_test shape: {y_test.shape}")   # Should be (num_test_samples,)

2. Building the LSTM Model

We'll construct a simple LSTM model. It will consist of one LSTM layer followed by a Dense output layer.

• LSTM Layer: We use layers.LSTM. The first argument is the number of units (neurons) in the LSTM layer. This determines the dimensionality of the hidden state and cell state. input_shape=(window_size, 1) tells the layer what shape of input sequence to expect (we don't need to specify the batch size here).
• Dense Layer: A standard fully connected layer with one neuron, as we are predicting a single output value (the next point in the time series). The activation function is typically linear for regression tasks like this.

model = keras.Sequential(
    [
        layers.LSTM(units=64, input_shape=(window_size, 1)),
        layers.Dropout(0.2), # Adding dropout for regularization
        layers.Dense(units=1) # Output layer: predicts the next single value
    ]
)

model.summary()

We need to compile the model, specifying the optimizer and the loss function. Adam is a common and effective optimizer. Since this is a regression problem (predicting a continuous value), Mean Squared Error (MSE) is a suitable loss function.

$$\text{MSE} = \frac{1}{N} \sum_{i=1}^{N} (y_{\text{true}, i} - y_{\text{pred}, i})^2$$

model.compile(optimizer='adam', loss='mean_squared_error')

3. Training the Model

Now we train the model using the prepared training data. We specify the number of epochs (passes through the entire dataset) and the batch size. We also use a portion of the training data as a validation set to monitor performance on unseen data during training.

history = model.fit(
    X_train,
    y_train,
    epochs=25,
    batch_size=32,
    validation_split=0.1, # Use 10% of training data for validation
    verbose=1 # Show progress
)

We can visualize the training and validation loss to check for convergence and overfitting.

Training and validation loss curves over epochs. Ideally, both should decrease and converge. A large gap between them might indicate overfitting.

4. Making Predictions and Evaluating

With the model trained, let's evaluate its performance on the held-out test set.

# Evaluate the model on test data
test_loss = model.evaluate(X_test, y_test, verbose=0)
print(f"Test Loss (MSE): {test_loss:.6f}")

# Make predictions on the test data
predictions_scaled = model.predict(X_test)

# Inverse transform predictions and actual values to original scale
predictions = scaler.inverse_transform(predictions_scaled)
y_test_original = scaler.inverse_transform(y_test.reshape(-1, 1))

Finally, let's visualize how well the model's predictions match the actual values in the test set.

Comparison of the actual time series values (test set) and the values predicted by the LSTM model.

Discussion

This hands-on example demonstrated forecasting a sine wave using a simple LSTM model. We covered data generation, scaling, sequence creation, model building (LSTM + Dense), training, evaluation (MSE), and prediction visualization.

The results show the LSTM capturing the underlying pattern reasonably well. Potential next steps for improving performance could include:

• Tuning hyperparameters (number of LSTM units, window size, learning rate, batch size).
• Trying a deeper model (stacking LSTM layers).
• Using GRUs instead of LSTMs to compare performance and efficiency.
• Experimenting with different optimizers or adding more regularization.
• Applying this workflow to more complex, real-world time series data.

This exercise provides a foundation for tackling various time series forecasting problems using recurrent neural networks.