The DataFrame: loading and exploring

In the previous module, Awa wrote her sales in a list of dictionaries, a bit like lines in a ledger. But her shop is growing, and she now has thousands of lines. Will she have to write a loop for every calculation? No, because there is a library made for this, and it is called pandas. Pandas works on tables, like those in a ledger, and in a single line it does what would take a long loop.

To use it, we write import pandas, then as pd to give it a shorter nickname. It is the same idea as with NumPy. Remember pd well, because we will write it in front of every pandas function.

Awa then gives her list of dictionaries to the DataFrame function. Look on the right: she gets a real table. Each dictionary becomes a row, and each key becomes a column, with its name at the top. You already know the square brackets and the curly braces: they are the list and the dictionaries from the previous module. So this table has rows and columns, like a page in a ledger, and pandas calls it a DataFrame.

When she displays it with print, pandas adds a number on the left of each row, starting from zero, like the boxes of a list. This number is called the index. Careful, this is not one of Awa's data: it is only the number that lets us find a row.

But surely she won't copy thousands of lines into her code? No, her sales are already in a file, called a CSV file. The read_csv function reads it and puts it straight into a table. CSV is a very simple format: a text file where each line is a sale, and where commas separate the columns. Between the parentheses, she writes the name of the file, in quotation marks. Look: the seven sales are now in the table.

Before any calculation, we look at what we received. The shape attribute gives the size of the table: first the number of rows, then the number of columns. Here, 7 rows and 3 columns. Notice that we write no parentheses after shape, because it is not an action to start: it is simply a piece of information about the table.

With thousands of rows, we cannot read everything. So the head method shows only the beginning of the table, in other words the first five rows unless we ask for something else. This time there are parentheses, because head is a method, in other words an action. Look at the rows lit up on the right: those are the ones head displays.

Between square brackets, the name of a column lets us take it on its own. Then the value_counts method counts how many times each value appears: 3 sales in Yaoundé, 3 in Douala and 1 in Garoua. In the lesson, you will also see info and describe, which complete this first look.

And if she only wants to see the sales above 2000 francs? She writes the condition, the amount greater than 2000, then she places it between the square brackets of the table. Pandas then checks the condition on each row, and it keeps only those where it is true. Read this line from the inside out: first the question asked of each row, then the table that keeps the true answers. The result is a new table, which she calls big.

The len function counts the rows kept: there are 3. The original table has not changed.

And for the big sales in Douala only? Then both conditions must be true at the same time. We link them with the ampersand sign, and we put each one in parentheses. 2 sales are left. Why not the word and? Because and cannot compare a whole column, row by row, while this sign can.

Finally, to add a column, we simply write its new name between square brackets, then a calculation. Here, the discount is one tenth of the amount, and pandas does the calculation on every row in one go. Look: the sale of 800 francs gets a discount of 80 francs, without any loop.

Now it is your turn: in the lesson, load a real dataset and explore it, without watching the video again. It is by searching for yourself that these commands will stay with you.