Mastering Linear Algebra for Data Science: Concepts Explained
Written on
Chapter 1: Introduction to Linear Algebra
By now, you should feel more comfortable with the initial parts of our linear algebra series. In the first part, we covered elementary operations and learned how to reduce a matrix to its row echelon form and row-reduced echelon form, as well as how to solve systems of linear equations. We also discussed the identity matrix and its role in these operations.
In part two, we delved into fundamental matrix operations, including matrix addition, subtraction, and multiplication, along with operations involving scalars.
This article will build upon those concepts, focusing on various calculations performed both manually and using Python's Numpy library, along with the introduction of the Sympy library for symbolic mathematics.
Section 1.1: Transpose of a Matrix
The transpose of a matrix is formed by switching its rows into columns. In essence, each row from the original matrix becomes a column in the transposed matrix, and vice versa. The formal notation for this operation can be represented as:
Here's a practical example:
Although transposing a matrix is straightforward, it can become cumbersome with larger matrices. In Python, you can easily compute the transpose as follows:
import numpy as np
import sympy as sym
# Using numpy:
A = np.array([[1, 5, -7, 10],
[3, 3, 2, 8],
[7, -5, 2, 9],
[13, 5, -3, 6]])
At = A.transpose()
print('A=', A)
print('At=', At)
# Using Sympy:
matrix_A = sym.Matrix([[1, 5, -7, 10],
[3, 3, 2, 8],
[7, -5, 2, 9],
[13, 5, -3, 6]])
matrix_At = matrix_A.transpose()
matrix_A
matrix_At
Section 1.2: Trace of a Matrix
The trace of a matrix, applicable only to square matrices, is defined as the sum of the elements along its main diagonal. It is crucial to note that the trace cannot be calculated for non-square matrices.
To compute the trace using Python:
# With Numpy:
trace_A_method_1 = A.trace()
trace_A_method_2 = np.trace(A)
print(trace_A_method_1)
print(trace_A_method_2)
# With Sympy:
from sympy import Trace
matrix_A_trace = Trace(matrix_A).simplify()
print('Trace of A with sympy is: ', matrix_A_trace)
Chapter 2: Inverse and Determinant of a Matrix
In this section, we will explore the concept of the inverse of a matrix. The inverse of a square matrix A is denoted as matrix B, such that:
A * B = Identity Matrix
B * A = Identity Matrix
If no such matrix B exists, A is termed singular, meaning it lacks an inverse. Each matrix has a unique inverse, and notably, the inverse of the identity matrix is itself.
To find the inverse, we use elementary row operations on both matrix A and the identity matrix simultaneously. Let's illustrate this with an example:
The following demonstrates how to compute the inverse of a matrix using Python:
# With Numpy
from numpy.linalg import inv
A = np.array([[1, 2, 3],
[0, 1, 4],
[0, 0, 1]])
inverse_A = inv(A)
print(inverse_A)
To verify that A multiplied by its inverse yields the identity matrix:
identity = np.matmul(A, inverse_A)
print(identity)
We can also achieve this using Sympy:
# With Sympy:
matrix_A = sym.Matrix([[1, 2, 3],
[0, 1, 4],
[0, 0, 1]])
inv_A = matrix_A.inv()
inv_A
# Verification with identity matrix:
identity_sym = matrix_A * inv_A
identity_sym
The determinant of a matrix is defined only for square matrices. The complexity of calculating the determinant increases with the size of the matrix. For a 1x1 matrix, the determinant is simply the value of its single element.
For a 2x2 matrix, the determinant is calculated by finding the difference between the product of the main diagonal and the product of the secondary diagonal.
For a 3x3 matrix, the Sarrus rule provides an efficient method for calculating the determinant.
For larger matrices (4x4 and above), we apply Laplace's rule, although we will not delve into its details here. It is essential to recognize that the larger the matrix, the more challenging it becomes to compute the determinant.
To illustrate how to compute the determinant of a 4th-order matrix (or any size) using Python:
# Using Numpy:
A = np.array([[1, 5, -7, 10],
[3, 3, 2, 8],
[7, -5, 2, 9],
[13, 5, -3, 6]])
print('A=', A)
det_A = round(np.linalg.det(A))
print('Det(A)=', det_A)
# Using Sympy:
matrix_A = sym.Matrix([[1, 5, -7, 10],
[3, 3, 2, 8],
[7, -5, 2, 9],
[13, 5, -3, 6]])
determinant_A = matrix_A.det()
print('Det(A)=', determinant_A)
References and Further Reading
- Elementary Linear Algebra, Applications Version, 9th Edition, Howard Anton and Chris Rorres.
- Maths for Computing and Information Technology, Frank Giannasi and Robert Low.
Thank you for taking the time to read this article! If you found it helpful, consider subscribing for updates on future publications. If you're interested in further reading, check out my book "Data-Driven Decisions: A Practical Introduction to Machine Learning," which provides valuable insights into machine learning concepts and applications.