Section 2.4: pandas, Series and DataFrame

pandas puts labels on arrays of values. A Series is one column with an index of labels, and a DataFrame is a table of such columns sharing one index. This section covers installing pandas, the two objects, selecting from a table by label and by position, writing into it under pandas 3's Copy-on-Write, and moving a table in and out of CSV, JSON, Excel and a SQL database.

Sections 2.2 and 2.3 computed over arrays whose rows and columns had no names, so the meaning of column 2 lived in a comment. A labeled table keeps the meaning with the data, and it brings silent failures of its own: arithmetic that pairs labels instead of positions, a selection by label that returns other rows than the same numbers by position, and a write that never reaches the table. This section is also where the module's dataset, housing.csv, first opens, and every later section works on it.

Lessons

  1. Installing pandas, the Series and the DataFrame - the two objects, label alignment, and a first look at housing.csv.
  2. Selection, Assignment and Copy-on-Write - loc, iloc, boolean filters, and why neither an assignment to a selection nor a chained one reaches the table.
  3. Reading and Writing Data - Including From a Query - CSV, JSON and Excel in and out, a query's result read into a table, and a table written into a STRICT one.

By the end you will be able to:

  • Build a Series and a DataFrame, and predict where label alignment produces missing values.
  • Select rows by label and by position, and account for every row a missing value drops from a filter.
  • Write into a table with one .loc assignment, and explain why chained assignment changes nothing in pandas 3.
  • Read and write CSV, JSON, Excel and SQL, and read the refusal a STRICT table gives a value it cannot store.