User:IssaRice/Linear algebra/Outline of linear algebra

From Machinelearning
Jump to: navigation, search

Two approaches to linear algebra

  • Coordinate-based approach: looks at concrete matrices, more emphasis on computation, works a lot in the standard basis. If linear algebra was analysis, this would be called "calculus"
  • Coordinate-free approach: Abstract vector spaces, more emphasis on linear maps. If linear algebra was analysis, this would be called "real analysis".

First half of linear algebra

The point of the first half is to consider general linear transformations (i.e. does not restrict to operators) and classify them into injective/surjective/bijective. See this table.

Topics include:

  • Elementary row operations, elementary matrices
  • Row equivalence (several equivalent formulations)
  • Echelon form, reduced row echelon form, pivots
  • Linear independence, span, basis (there seems to be a choice of doing this as sets vs lists, although I think even books that use sets are sometimes forced to then define an "ordered basis")
  • Column space
  • Row space
  • Rank, column rank, row rank
  • Linear systems of equations
  • Matrix multiplication
  • Null space = solution set of homogeneous linear system
  • Finding a basis for range, null space, range of transpose, null space of transpose
  • Fundamental theorem of linear maps: rank + nullity = dimension of domain
  • Equivalent properties of injective, surjective, bijective
  • Change of basis
  • Matrix similarity
  • Going back and forth between coordinate-based and coordinate-free approach, e.g. diagonalization via multiplying on both sides vs diagonalization via picking some basis

Second half of linear algebra

The second half focuses on operators (linear maps that map from a vector space to the same vector space) and does inner product stuff. Maybe this is called "spectral theory".


  • Inner product
  • Norm
  • Eigen stuff
  • Determinants? Trace?
  • Spectral theorem
  • Singular value decomposition
  • Diagonalization
  • Orthogonality
  • Orthonormal bases
  • Orthogonal projection


  • Can the second half be done first?