x̄ - > Worked example of a matrix-related concept: Eigenvalues and Eigenvectors.

Mathematics of Transformers

Advanced Worked Example: Eigenvalues and Eigenvectors

This topic is crucial in various fields, including data science, physics, and engineering.

Problem: Finding Eigenvalues and Eigenvectors

Given the matrix:

\[ A = \begin{bmatrix} 4 & 1 \\ 2 & 3 \end{bmatrix} \]

Find the eigenvalues and eigenvectors of \( A \).

Solution:

Step 1: Eigenvalue Equation

The eigenvalue equation is:

\[ A\mathbf{v} = \lambda\mathbf{v} \]

Rewriting, we get:

\[ (A - \lambda I)\mathbf{v} = 0 \]

Here:

• \( \lambda \) is the eigenvalue.
• \( I \) is the identity matrix.

For non-trivial solutions (\( \mathbf{v} \neq 0 \)), the determinant of \( A - \lambda I \) must be zero:

\[ \det(A - \lambda I) = 0 \]

Step 2: Characteristic Polynomial

Substitute \( A \) and \( I \) into \( \det(A - \lambda I) \):

\[ A - \lambda I = \begin{bmatrix} 4 & 1 \\ 2 & 3 \end{bmatrix} - \lambda \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{bmatrix} \]

The determinant is:

\[ \det(A - \lambda I) = \begin{vmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{vmatrix} \]

Use the determinant formula for a 2x2 matrix:

\[ \det(A - \lambda I) = (4 - \lambda)(3 - \lambda) - (2 \cdot 1) \]

Simplify:

\[ \det(A - \lambda I) = (4 - \lambda)(3 - \lambda) - 2 \]

\[ \det(A - \lambda I) = 12 - 4\lambda - 3\lambda + \lambda^2 - 2 \]

\[ \det(A - \lambda I) = \lambda^2 - 7\lambda + 10 \]

Step 3: Solve for Eigenvalues

Set \( \det(A - \lambda I) = 0 \):

\[ \lambda^2 - 7\lambda + 10 = 0 \]

Factorize:

\[ \lambda^2 - 7\lambda + 10 = (\lambda - 5)(\lambda - 2) \]

Thus, the eigenvalues are:

\[ \lambda_1 = 5, \quad \lambda_2 = 2 \]

Step 4: Find Eigenvectors

For each eigenvalue, solve \( (A - \lambda I)\mathbf{v} = 0 \).

For \( \lambda_1 = 5 \):

\[ A - \lambda_1 I = \begin{bmatrix} 4 - 5 & 1 \\ 2 & 3 - 5 \end{bmatrix} = \begin{bmatrix} -1 & 1 \\ 2 & -2 \end{bmatrix} \]

Solve:

\[ \begin{bmatrix} -1 & 1 \\ 2 & -2 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \]

From the first row:

\[ -v_1 + v_2 = 0 \quad \Rightarrow \quad v_2 = v_1 \]

The eigenvector is:

\[ \mathbf{v}_1 = k \begin{bmatrix} 1 \\ 1 \end{bmatrix} \quad (k \text{ is any scalar}) \]

For \( \lambda_2 = 2 \):

\[ A - \lambda_2 I = \begin{bmatrix} 4 - 2 & 1 \\ 2 & 3 - 2 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 2 & 1 \end{bmatrix} \]

Solve:

\[ \begin{bmatrix} 2 & 1 \\ 2 & 1 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \]

From the first row:

\[ 2v_1 + v_2 = 0 \quad \Rightarrow \quad v_2 = -2v_1 \]

The eigenvector is:

\[ \mathbf{v}_2 = k \begin{bmatrix} 1 \\ -2 \end{bmatrix} \quad (k \text{ is any scalar}) \]

Final Answer

Eigenvalues: \( \lambda_1 = 5, \lambda_2 = 2 \)

Eigenvectors: \( \mathbf{v}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \mathbf{v}_2 = \begin{bmatrix} 1 \\ -2 \end{bmatrix} \)

This work is licensed under a Creative Commons Attribution 4.0 International License.