## Linear Algebra

A lot of the Data Science methods we will see in this tutorial require some understanding of linear algebra, and in this notebook we will focus on how Julia handles matrices, the types that exist, and how to call basic linear algebra tasks.

All rights reserved with Julia Academy

The code is avaiable at Github by **Dr Nassar**.

## Youtube Tutorial ๐

We will get started with creating a random matrix.

A few things that are noteworthy:

`A`

is a`Matrix`

type, and`b`

is a`Vector`

type.- The transpose function creates a matrix of type
`Adjoint`

. `\`

is always the recommended way to solve a linear system. You almost never want to call the`inv`

function

`adjoint`

in julia is a lazy adjoint -- often, we can easily perform Linear Algebra operations such as `A*A'`

without actually transposing the matrix.

That's because it's an array of Float64's, each is of size 8 bytes, and there are 10*10 numbers.

The `\`

operator allows you to solve a system of linear equations, and often uses a suitable matrix factorization to solve the problem. We will cover factorizations next.

## Factorizations

A common tool used in Linear Algebra is matrix factorizations. These factorizations are often used to solve linear systems like `Ax=b`

, and as we will see later in this tutorial... `Ax=b`

comes up in a lot of Data Science problems

### LU factorization

L*U = P*A

### QR factorization

Q*R = A

### Cholesky factorization, note that A needs to be symmetric positive definite

L*L' = A

`I`

is a function

## Sparse Linear Algebra

Sparse matrices are stored in Compressed Sparse Column (CSC) form

## Images as matrices

Let's get to the more "data science-y" side. We will do so by working with images (which can be viewed as matrices), and we will use the `SVD`

decomposition.

First let's load an image. I chose this image as it has a lot of details.

We can easily convert this image to gray scale.

We can easily extract the RGB layers from the image. We will make use of the `reshape`

function below to reshape a vector to a matrix.

We can easily obtain the `Float64`

values of the grayscale image.

Next, we will downsample this image using the SVD. First, let's obtain the SVD decomposition.

As you can see, it's still far away from the original image. Let's try using 100 singular vectors/values.

This looks almost identical to the original image, even though it's not identical to the original image (and we can see that from the norm difference).

Our next problem will still be related to images, but this time we will solve a simple form of the face recognition problem. Let's get the data first.

Each vector in `M["V2"]`

is a fase image. Let's reshape the first one and take a look.

Now we will go back to the vectorized version of this image, and try to select the images that are most similar to it from the "dictionary" matrix. Let's use `b = q[:]`

to be the query image. Note that the notation `[:]`

vectorizes a matrix column wise.

We will remove the first image from the dictionary. The goal is to find the solution of the linear system `Ax=b`

where `A`

is the dictionary of all images. In face recognition problem we really want to minimize the norm differece `norm(Ax-b)`

but the `\`

actually solves a least squares problem when the matrix at hand is not invertible.

This was an easy problem. Let's try to make the picture harder to recover. We will add some random error.

The error is so much bigger this time.

# Finally...

After finishing this notebook, you should be able to:

- reshape and vectorize a matrix
- apply basic linear algebra operations such as transpose, matrix-matrix product, and solve a linear systerm
- call a linear algebra factorization on your matrix
- use SVD to created a compressed version of an image
- solve the face recognition problem via a least square approach
- create a sparse matrix, and call the components of the Compressed Sparse Column storage
- list a few types of matrices Julia uses (diagonal, upper triangular,...)
- (unrelated to linear algebra): load an image, convert it to grayscale, and extract the RGB layers

# ๐ฅณ One cool finding

We can solve a simple form of the face recognition problem even when a face image has been distorted with wrong pixels. Example, one of our inputs was this image:

And we were able to detect this face to be closest to the input image:

**Thank you!! for taking you time in learning Julia and Leaer Algebra, do like โฅ or comment ๐จ my work so that I can keep posting these amaing work.**