Python Basics for Data Science

Author: Ventsislav Yordanov
Original article: towardsdatascience.com

Python Data Types

In Python, we have many data types. The most common ones are float (floating point), int (integer), str (string), bool (Boolean), list, and dict (dictionary).

float - used for real numbers.
int - used for integers.
str - used for texts. We can define strings using single quotes 'value' , double quotes "value" , or triple quotes """value""" . The triple quoted strings can be on multiple lines, the new lines will be included in the value of the variable. They’re also used for writing function documentation.
bool - used for truthy values. Useful to perform a filtering operation on a data.
list - used to store a collection of values.
dict - used to store a key-values pairs.

We can use the type(variable_name) function to check the type of a specific variable. Operators in Python behave differently depending on the variable’s type and there are different built-in methods for each one.

Here we can look at some examples with creating a floating points, intergers, strings and booleans in Python.

year_of_birth = 1994 height_cm = 170.50 subject = "Data Science" is_success = True print(type(year_of_birth), type(height_cm), type(subject), type(is_success))

Python Lists

Python list is a basic sequence type. We can use this type to store a collection of values. One list can contain values of any type . It is possible that one list contains another nested lists for its values. It’s not commonly used, but you can have a list with a mix of Python types. You can create a new one using square brackets like this:

fruits = ["pineapple", "apple", "lemon", "strawberry", "orange", "kiwi"]

Subsetting Lists

You can use indexes to get element or elements from the list. In Python, the indexes start from 0. Therefore, the first element in the list will have an index 0 . We can also use negative indexes to access elements. The last element in the list will have an index -1 , the one before the last one will have an index -2 and so on. We have also something called list slicing in Python which can be used to get multiple elements from a list. We can use it like this: sliceable[start_index:end_index:step].

The start_index is the beginning index of the slice, the element at this index will be included to the result, the default value is 0 .
The end_index is the end index of the slice, the element at this index will not be included to the result, the default value will be the length of the list . Also, the default value can be - length of the list -1 if the step is negative. If you skip this, you will get all the elements from the start index to the end.
The step is the amount by which the index increases,
the default value is 1 . If we set a negative value for the step, we’ll move backward.

fruits = ["pineapple", "apple", "lemon", "strawberry", "orange", "kiwi"] fruits[1] # apple fruits[0] # "pineapple" fruits[-1] # "kiwi" fruits[5] # "kiwi" fruits[-3] # "strawberry" # List slicing fruits[::] # ["pineapple", "apple", "lemon", "strawberry", "orange", "kiwi"] fruits[0:2] # ["pineapple", "apple"] fruits[-2:-1] # ["orange"] fruits[3:] # ["strawberry", "orange", "kiwi"] fruits[:4] # ["pineapple", "apple", "lemon", "strawberry"] fruits[:] # ["pineapple", "apple", "lemon", "strawberry", "orange", "kiwi"] fruits[::-1] # ["kiwi", "orange", "strawberry", "lemon", "apple", "pineapple"] fruits[::-2] # ["kiwi", "strawberry", "apple"] fruits[::2] # ["pineapple", "lemon", "orange"]

List Manipulation

We can add element or elements to a list using append method or by using the plus operator . If you’re using the plus operator on two lists, Python will give a new list of the contents of the two lists.
We can change element or elements to list using the same square brackets that we already used for indexing and list slicing.
We can delete an element from a list with the remove(value) method. This method will delete the first element of the list with the passed value.

# Add values to a list fruits.append("peach") fruits # ["pineapple", "apple", "lemon", "strawberry", "orange", "kiwi", "peach"] fruits = fruits + ["fig", "melon"] fruits # ["pineapple", "apple", "lemon", "strawberry", "orange", "kiwi", "peach", "fig", "melon"] # Change values from a list fruits[0:2] = ["grape", "mango"] fruits # ["grape", "mango", "lemon", "strawberry", "orange", "kiwi", "peach", "fig", "melon"] # Delete values from a list fruits.remove("mango") fruits # ["grape", "lemon", "strawberry", "orange", "kiwi", "peach", "fig", "melon"]

It’s important to understand how lists work behind the scenes in Python. When you create a new list my_list , you’re storing the list in your computer memory, and the address of that list is stored in the my_list variable. The variable my_list doesn’t contain the elements of the list. It contains a reference to the list. If we copy a list with the equal sign only like this my_list_copy = my_list , you’ll have the reference copied in the my_list_copy variable instead of the list values. So, if you want to copy the actual values, you can use the list(my_list) function or slicing [:] .

numbers = [10, 42, 28, 420] numbers_copy = numbers numbers_copy[2] = 100 numbers # [10, 42, 100, 420] numbers_copy # [10, 42, 100, 420] ratings = [4.5, 5.0, 3.5, 4.75, 4.00] ratings_copy = ratings[:] ratings_copy[0] = 2.0 ratings # [4.5, 5.0, 3.5, 4.75, 4.0] ratings_copy # [2.0, 5.0, 3.5, 4.75, 4.0] characters = ["A", "B", "C"] characters_copy = list(characters) characters_copy[-1] = "D" characters # ["A", "B", "C"] characters_copy # ["A", "B", "D"]

Python Dictionaries

The dictionaries are used to store key-value pairs . They are helpful when you want your values to be indexed by unique keys . In Python, you can create a dictionary using curly braces . Also, a key and a value are separated by a colon . If we want to get the value for a given key, we can do it like that: our_dict[key] .

Dictionaries vs Lists

Let’s see an example and compare the lists versus dictionaries. Imagine that we have some movies and you want to store the ratings for them. Also, we want to access the rating for a movie very fast by having the movie name. We can do this by using two lists or one dictionary. In examples the movies.index(“Ex Machina”) code returns the index for the “Ex Machina” movie.

movies = ["Ex Machina", "Mad Max: Fury Road", "1408"] ratings = [7.7, 8.1, 6.8] movie_choice_index = movies.index("Ex Machina") print(ratings[movie_choice_index]) # 7.7

ratings = { "Ex Machina": 7.7, "Mad Max: Fury Road": 8.1, "1408" : 6.8 } print(ratings["Ex Machina"]) # 7.7

In this case, the usage of a dictionary is a more intuitive and convenient way to represent the ratings.

Dictionaries Operations

We can add , update , and delete data from our dictionaries. When we want to add or update the data we can simply use this code our_dict[key] = value . When we want to delete a key-value pair we do this like that del(our_dict[key]) .

ratings["Deadpool"] = 8.0 print(ratings) # {'Ex Machina': 7.7, 'Mad Max: Fury Road': 8.1, '1408': 6.8, 'Deadpool': 8.0} ratings["Ex Machina"] = 7.8 print(ratings) # {'Ex Machina': 7.8, 'Mad Max: Fury Road': 8.1, '1408': 6.8, 'Deadpool': 8.0} del(ratings["1408"]) print(ratings) # {'Ex Machina': 7.8, 'Mad Max: Fury Road': 8.1, 'Deadpool': 8.0}

We can also check if a given key is in our dictionary like that: key in our_dict .

print("Ex Machina" in ratings) # True

Functions

A function is a piece of reusable code solving a specific task. We can write our functions using the def keyword like that:

def is_prime(n): if n <= 1: return False elif n <= 3: return True elif n % 2 == 0 or n % 3 == 0: return False current_number = 5 while current_number * current_number <= n: if n % current_number == 0 or n % (current_number + 2) == 0: return False current_number = current_number + 6 return True

However, there are many built-in function in Python like max(iterable [, key]), , min(iterable [, key]) , type(object) , round(number [, ndigits]) , etc. So, in many cases when we need a function that solves a given task, we can research for a built-in function that solves this task or a Python package for that. We don’t have to “ reinventing the wheel ”.

Most of the functions take some input and return some output. These functions have arguments, and Python matches the passed inputs in a function call to the arguments. If square brackets surround an argument, it’s optional.

We can use the function help([object]) or ?function_name to see the documentation of any function. If we’re using Jupyter Notebook, the help function will show us the documentation in the current cell, while the second option will show us the documentation in the pager.

Methods

We’ve seen that we have strings, floats, integers, booleans, etc. in Python. Each one of these data structures is an object. A method is a function that is available for a given object depending on the object’s type. So, each object has a specific type and a set of methods depending on this type.

# String methods text = "Data Science" text.upper() # "DATA SCIENCE" text.lower() # "data science" text.capitalize() # "Data science" # Lists methods numbers = [1, 4, 0, 2, 9, 9, 10] numbers.reverse() print(numbers) # [10, 9, 9, 2, 0, 4, 1] numbers.sort() print(numbers) # [0, 1, 2, 4, 9, 9, 10] # Dictionaris methods ratings = { "Ex Machina": 7.7, "Mad Max: Fury Road": 8.1, "1408" : 6.8 } print(ratings.keys()) # dict_keys(['Ex Machina', 'Mad Max: Fury Road', '1408']) print(ratings.values()) # dict_values([7.7, 8.1, 6.8]) print(ratings.items()) # dict_items([('Ex Machina', 7.7), ('Mad Max: Fury Road', 8.1), ('1408', 6.8)])

Objects with different type can have methods with the same name. Depending on the object’s type, methods have different behavior.

numbers = [10, 30, 55, 40, 8, 30] text = "Data Science" numbers.index(8) # 4 text.index("a") # 1 numbers.count(30) # 2 text.count("i") # 1dd

Watch out! Some methods can change the objects they are called on. For example, the append() method called on list type.

Packages

A module is a file containing Python definitions and statements. Modules specify functions, methods and new Python types which solved particular problems.

A package is a collection of modules in directories. There are many available packages for Python covering different problems. For example, “NumPy”, “matplotlib”, “seaborn”, and “scikit-learn” are very famous data science packages.

“NumPy” is used for efficiently working with arrays
“matplotlib” and “seaborn” are popular libraries used for data visualization
“scikit-learn” is a powerful library for machine learning

There are some packages available in Python by default, but there are also so many packages that we need and that we don’t have by default. If we want to use some package, we have to have it installed already or just install it using pip (package maintenance system for Python).

However, there is also something called “Anaconda”.

Anaconda Distribution is a free, easy-to-install package manager, environment manager and Python distribution with a collection of 1,000+ open source packages with free community support.

So, if you don’t want to install many packages, I’ll recommend you to use the “Anaconda”. There are so many useful packages in this distribution.

Import Statements

Once you have installed the needed packages, you can import them into your Python files. We can import an entire package, submodules or specific functions from it. Also, we can add an alias for a package. We can see the different ways of import statements from the examples below.

import numpy numbers = numpy.array([3, 4, 20, 15, 7, 19, 0])

import numpy as np # np is an alias for the numpy package numbers = np.array([3, 4, 20, 15, 7, 19, 0]) # works fine numbers = numpy.array([3, 4, 20, 15, 7, 19, 0]) # NameError: name 'numpy' is not defined

# import the "pyplot" submodule from the "matplotlib" package with alias "plt" import matplotlib.pyplot as plt

from numpy import array numbers = array([3, 4, 20, 15, 7, 19, 0]) # works fine numbers = numpy.array([3, 4, 20, 15, 7, 19, 0]) # NameError: name 'numpy' is not defined type(numbers) # numpy.ndarray

We can also do something like this from numpy import * . The asterisk symbol here means to import everything from that module. This import statement creates references in the current namespace to all public objects defined by the numpy module. In other words, we can just use all available functions from numpy only with their names without prefix. For example, now we can use the NumPy’s absolute function like that absolute() instead of numpy.absolute() .
However, I’m not recommending you to use that because:

If we import all functions from some modules like that, the current namespace will be filled with so many functions and if someone looks our code, he or she can get confused from which package is a specific function.
If two modules have a function with the same name, the second import will override the function of the first.

NumPy

NumPy is a fundamental package for scientific computing with Python. It’s very fast and easy to use. This package helps us to make calculations element-wise (element by element).

The regular Python list doesn’t know how to do operations element-wise. Of course, we can use Python lists, but they’re slow, and we need more code to achieve a wanted result. A better decision in most cases is to use NumPy .

Unlike the regular Python list, the NumPy array always has one single type. If we pass an array with different types to the np.array() , we can choose the wanted type using the parameter dtype . If this parameter is not given, then the type will be determined as the minimum type required to hold the objects.

np.array([False, 42, "Data Science"]) # array(["False", "42", "Data Science"], dtype="<U12") np.array([False, 42], dtype = int) # array([ 0, 42]) np.array([False, 42, 53.99], dtype = float) # array([ 0. , 42. , 53.99]) # Invalid converting np.array([False, 42, "Data Science"], dtype = float) # could not convert string to float: 'Data Science'

NumPy array comes with his own attributes and methods. Remember that the operators in Python behave differently on the different data types? Well, in NumPy the operators behave element-wise.

np.array([37, 48, 50]) + 1 # array([38, 49, 51]) np.array([20, 30, 40]) * 2 # array([40, 60, 80]) np.array([42, 10, 60]) / 2 # array([ 21., 5., 30.]) np.array([1, 2, 3]) * np.array([10, 20, 30]) # array([10, 40, 90]) np.array([1, 2, 3]) - np.array([10, 20, 30]) # array([ -9, -18, -27])

If we check the type of a NumPy array the result will be numpy.ndarray . Ndarray means n-dimensional array. In the examples above we used 1-dimensional arrays, but nothing can stop us to make 2, 3, 4 or more dimensional array. We can do subsetting on an array independently of that how much dimensions this array has. I’ll show you some examples with a 2-dimensional array.

numbers = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12] ]) numbers[2, 1] # 8 numbers[-1, 0] # 10 numbers[0] # array([1, 2, 3]) numbers[:, 0] # array([ 1, 4, 7, 10]) numbers[0:3, 2] # array([3, 6, 9]) numbers[1:3, 1:3] # array([[5, 6],[8, 9]])

If we want to see how many dimensional is our array and how much elements have each dimension, we can use the shape attribute. For 2-dimensional arrays, the first element of the tuple will be the number of rows and the second the number of the columns.

numbers = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15] ]) numbers.shape # (5, 3)

Basic Statistics

The first step of analyzing data is to get familiar with the data. NumPy has a lot of methods which help us to do that. We’ll see some basic methods to make statistics on our data.

np.mean() - returns the arithmetic mean (the sum of the elements divided by the length of the elements).
np.median() - returns the median (the middle value of a sorted copy of the passed array, if the length of array is even - the average of the two middle values will be computed)
np.corrcoef() - returns a correlation matrix. This function is useful when we want to see if there is a correlation between two variables in our dataset or with other words, between two arrays with the same length.
np.std() - returns a standard deviation

learning_hours = [1, 2, 6, 4, 10] grades = [3, 4, 6, 5, 6] np.mean(learning_hours) # 4.6 np.median(learning_hours) # 4.0 np.std(learning_hours) # 3.2 np.corrcoef(learning_hours, grades) # [[ 1. 0.88964891][ 0.88964891 1. ]]

From the example above, we can see that there is a high correlation between the hours of learning and the grade.
Also, we can see that:

the mean for the learning hours is 4.6
the median for the learning hours is 4.0
the standard deviation for the learning hours is 3.2

NumPy also has some basic functions like np.sort() and np.sum() which exists in the basic Python lists, too. An important note here is that NumPy enforces a single type in an array and this speeds up the calculations.

Exercises

I have prepared some exercises including subsetting, element-wise operations, and basic statistics. If you want, you can try to solve them.

Final Words

Thank you for the read. If you like this post, please hold the clap button and share it with your friends. Also, I’ll be happy to hear your feedback. If you want to be notified when I create a new blog post, you can subscribe to my newsletter .