Hands-on Practical: Setup and Tensor Basics

Having covered the essential concepts of PyTorch installation and the fundamental Tensor object, it's time to put this knowledge into practice. These exercises reinforce your understanding of setting up your environment and performing basic Tensor manipulations, which are prerequisites for building any deep learning model in PyTorch.

Verifying Your PyTorch Installation

First, let's ensure PyTorch is correctly installed and accessible in your Python environment. Open your Python interpreter or a Jupyter Notebook and run the following commands:

import torch
import numpy as np # We'll use NumPy for comparison later

# Print PyTorch version
print(f"PyTorch Version: {torch.__version__}")

# Check if CUDA (GPU support) is available
if torch.cuda.is_available():
    print(f"CUDA is available. Device: {torch.cuda.get_device_name(0)}")
    # Get the default device PyTorch will use
    device = torch.device("cuda")
else:
    print("CUDA not available. Using CPU.")
    device = torch.device("cpu")

print(f"Default device: {device}")

Executing this code confirms that the torch library can be imported and provides its version. It also checks for GPU availability, which is significant for accelerating computations in later chapters. If you have a compatible NVIDIA GPU and installed the correct PyTorch version, you should see CUDA reported as available. For now, we'll primarily use the CPU, but knowing how to check for the GPU is important.

Creating Tensors

Let's practice creating Tensors using various methods introduced earlier.

1. From Python Lists: Create a 2x3 tensor from a nested Python list.

# Create a tensor from a Python list
data = [[1, 2, 3], [4, 5, 6]]
tensor_from_list = torch.tensor(data)

print("Tensor from list:")
print(tensor_from_list)
print(f"Shape: {tensor_from_list.shape}")
print(f"Data type: {tensor_from_list.dtype}") # Defaults usually to int64

2. Specifying Data Type: Create the same tensor but explicitly set the data type to 32-bit floating point.

# Create tensor with specific dtype
tensor_float32 = torch.tensor(data, dtype=torch.float32)

print("\nTensor with float32 dtype:")
print(tensor_float32)
print(f"Shape: {tensor_float32.shape}")
print(f"Data type: {tensor_float32.dtype}")

Notice the change in dtype and the representation of numbers (e.g., 1. instead of 1).

3. Using Factory Functions: Create tensors with specific shapes and initial values.

# Create a 3x4 tensor of zeros
zeros_tensor = torch.zeros(3, 4)
print("\nZeros Tensor (3x4):")
print(zeros_tensor)

# Create a 2x2 tensor of ones with integer type
ones_tensor_int = torch.ones(2, 2, dtype=torch.int32)
print("\nOnes Tensor (2x2, int32):")
print(ones_tensor_int)

# Create a 1D tensor representing a range of numbers
range_tensor = torch.arange(start=0, end=5, step=1) # Similar to Python's range
print("\nRange Tensor (0 to 4):")
print(range_tensor)

# Create a 2x3 tensor with random values (uniform distribution 0 to 1)
rand_tensor = torch.rand(2, 3)
print("\nRandom Tensor (2x3):")
print(rand_tensor)

These factory functions are convenient for initializing tensors without needing pre-existing data structures like lists.

Basic Tensor Operations

Now, let's perform some fundamental operations on the tensors we've created.

# Use the float32 tensor created earlier
a = torch.tensor([[1, 2], [3, 4]], dtype=torch.float32)
b = torch.ones(2, 2) # Defaults to float32

print("Tensor 'a':")
print(a)
print("Tensor 'b':")
print(b)

# Element-wise addition
sum_tensor = a + b
# Alternative syntax: sum_tensor = torch.add(a, b)
print("\nElement-wise Sum (a + b):")
print(sum_tensor)

# Element-wise multiplication
prod_tensor = a * b
# Alternative syntax: prod_tensor = torch.mul(a, b)
print("\nElement-wise Product (a * b):")
print(prod_tensor)

# Scalar multiplication
scalar_mult = a * 3
print("\nScalar Multiplication (a * 3):")
print(scalar_mult)

# In-place addition (modifies 'a')
print(f"\n'a' before in-place add: ID {id(a)}")
a.add_(b) # Note the trailing underscore for in-place operations
print("'a' after in-place add (a.add_(b)):")
print(a)
print(f"'a' after in-place add: ID {id(a)}") # ID remains the same

# Matrix multiplication
# Ensure dimensions are compatible for matrix multiplication
# Let's create compatible tensors: x (2x3), y (3x2)
x = torch.rand(2, 3)
y = torch.rand(3, 2)
matmul_result = torch.matmul(x, y)
# Alternative syntax: matmul_result = x @ y
print("\nMatrix Multiplication (x @ y):")
print(f"Shape of x: {x.shape}, Shape of y: {y.shape}")
print(f"Result shape: {matmul_result.shape}")
print(matmul_result)

Pay attention to the difference between element-wise operations (like +, *) and matrix multiplication (torch.matmul or @). Also, note how in-place operations (like add_) modify the tensor directly without creating a new object.

Interacting with NumPy

PyTorch integrates well with NumPy. Let's practice converting between NumPy arrays and PyTorch Tensors.

# 1. NumPy array to PyTorch Tensor
numpy_array = np.array([[1.0, 2.0], [3.0, 4.0]])
print("\nNumPy Array:")
print(numpy_array)
print(f"Type: {type(numpy_array)}")

# Convert to PyTorch Tensor
tensor_from_numpy = torch.from_numpy(numpy_array)
print("\nTensor from NumPy array:")
print(tensor_from_numpy)
print(f"Type: {type(tensor_from_numpy)}")

# Important: On CPU, torch.from_numpy shares memory with the NumPy array
# Modifying one affects the other
numpy_array[0, 0] = 99.0
print("\nNumPy array after modification:")
print(numpy_array)
print("Tensor after modifying NumPy array (shared memory):")
print(tensor_from_numpy)

# 2. PyTorch Tensor to NumPy array
# Let's use a different tensor to avoid the previous modification
another_tensor = torch.tensor([[5, 6], [7, 8]], dtype=torch.float64)
print("\nAnother PyTorch Tensor:")
print(another_tensor)

# Convert to NumPy array
numpy_from_tensor = another_tensor.numpy()
print("\nNumPy array from Tensor:")
print(numpy_from_tensor)
print(f"Type: {type(numpy_from_tensor)}")

# Again, memory is shared on CPU
another_tensor[1, 1] = 100.0
print("\nTensor after modification:")
print(another_tensor)
print("NumPy array after modifying Tensor (shared memory):")
print(numpy_from_tensor)

This memory sharing behavior between NumPy arrays and CPU Tensors is efficient but requires careful handling, as unintended modifications can occur. If you need a distinct copy, you can use the .clone() method on the tensor before converting or standard Python/NumPy copying mechanisms.

This practical session covered verifying your setup, creating tensors in multiple ways, performing basic arithmetic and matrix operations, and converting between PyTorch Tensors and NumPy arrays. Mastering these fundamentals is essential as we move on to more complex topics like automatic differentiation and building neural network modules in the subsequent chapters.