The Coder's Apprentice (2020–2021)

Pieter Spronck

the coder's apprentice

In this interactive tutorial you’ll learn how to code in Python 3. We assume you do not have any prior experience with computer programming. The learning material is based on the book The Coder’s Apprentice by Pieter Spronck (Tilburg University).

As a teacher you may go through the documentation before diving into the course with a group of students.

Read book as PDF
NEDERLANDSTALIGE VERSIE

You are not a member of this course.

Register

Exercise series

Computers are wonderful machines. While most machines (cars, television sets, microwaves) have a specific purpose which they excel at accomplishing, computers are purposeless machines that can be taught to accomplish anything. The power to make a computer do your bidding is called “programming.”

Nowadays, in any scientific and professional endeavor, people have to deal with large volumes of data. Those who are able to leverage the power of computers to make use of such data, i.e., those who can program, are far better able to do their jobs than those who are not. In fact, it can be argued that in the very near future, those who do not possess programming skills will become unemployable. Therefore, I feel that it is necessary for anyone to acquire basic skills in this area during their education.

Being able to write computer programs not only entails knowing what specific code statements mean and do; it also entails having the ability think like a programmer, and to analyze problems from the perspective of solving them with a computer program. Such skills cannot be learned from a book, they can only be learned by actually creating programs.

This book has been designed to teach the basics of the Python 3 computer language. Students will not only learn to use the language, but also do their first practical exercises with it.

The book is not only catering towards people who are naturally inclined towards programming. It is meant to also be used by those who have no particular aptitude for programming. This is exemplified by texts which try to be extensive and foresee problems that might arise when trying to understand certain concepts.

As explained in the introduction, you will need to write and run Python code to learn anything from this book. That means that you need a computer on which Python is installed, and you need to know how to write and run Python programs. This chapter will explain to you how to get “up and running” with Python.

Title Class progress Status
Introduction
Getting Python
Creating Python programs
Running Python programs
Reference material
Hello, world!
Evaluate an expression

Welcome to the first real programming chapter. In this chapter I discuss “expressions,” which are straightforward calculations which you can also do with any simple calculator. It is a small start, but you are going to need such expressions for every chapter after this one.

Title Class progress Status
Introduction
Displaying results
Display text
Data types
Expressions
Style
What you learned
Ordering books
Syntax errors
ZeroDivisionError
Runtime errors
Set an alarm

When working with program code, very often you are designing a procedure (or “algorithm”) that solves a problem in a general way. For instance, in the previous chapter one of the exercises had you calculate the wholesale price for a stack of books, for a given book price and a given number of books. The code you wrote did not solve this problem for a general case, but only for the specific case of 60 books costing 24.95 per book. If you want to write code that solves problems in a more general way, you need to use variables that store values.

Title Class progress Status
Introduction
Variables and values
Variable names
Debugging variables
Soft typing
Shorthand operators
Comments
What you learned
Average of three values
Area of a circle
Number of coins
Swap variables

Up to this point, I have already introduced some basic “functions,” such as print() and int(). In this chapter these functions will be discussed a bit more in-depth, and a few other functions will be introduced, which will be helpful in the coming chapters. In Chapter 8, I will discuss how you can create your own functions.

Title Class progress Status
Introduction
Elements of a function
Some basic functions
Modules
What you learned
String length
Pythagorean theorem
Minimum, maximum and average
Powers of e
Random integers

In program code, there are often statements that you only want to execute when certain conditions hold. Every programming language therefore supports conditional statements. In this chapter I will explain how to use conditions in Python.

Title Class progress Status
Introduction
Boolean expressions
Conditional statements
Early exits
What you learned
Letter grades
Letter grades
Different vowels
Quadratic equations

Computers do not get bored. If you want the computer to repeat a certain task hundreds of thousands of times, it does not protest. Humans hate too much repetition. Therefore, repetitious tasks should be performed by computers. All programming languages support repetitions. The general class of programming constructs that allow the definition of repetitions are called “iterations.” A term which is even more common for such tasks is “loops.”

This chapter explains all you need to know about loops in Python. Students who are completely new to programming often find loops the first really hard topic in programming that they encounter. If that is the case for you, then make sure you take your time for this chapter, and work on it until you understand it completely. Loops are such a basic concept in programming that you need to understand them in all their details. Each and every chapter after this one needs loops.

Title Class progress Status
Introduction
While loop
For loops
Loop control statements
Nested loops
The loop-and-a-half
Being smart about loops
On designing algorithms
What you learned
Exercises
Multiplication tables
Multiplication tables
Largest, smallest and triplicates
99 bottles of beer
Fibonacci sequence
Common characters
Approximation of pi
Number guessing game (TODO)
Number guessing game (TODO)
Prime numbers
Multiplication tables
Sum of two squares
Five dice
DCBA = 4 × ABCD
Monkeys and coconuts
Triangle Crawlers

In Chapter 5 I described how to use simple functions, and how to import functions from modules. This chapter is about how to write your own functions and modules. If you do not remember what Chapter 5 said about functions, re-read that chapter before continuing with this one.

Title Class progress Status
Introduction
Why create functions?
Creating functions
Scope and lifetime
Managing program complexity
Modules
Anonymous functions
What you learned
Multiplication tables
Common characters
Grerory-Leibnitz series
Quadratic equations
Loop-and-a-half (TODO)
Binomial coefficient
What is wrong?

Recursion is a special technique that can be used now you are able to create and use functions. Recursion can be very elegant and powerful, but students often find it hard to employ. That is why I decided to spend a separate chapter on it. If, while studying this chapter, you feel that it is getting too complex for you, feel free to skip it for the time being. The following chapters are a lot easier again.

Title Class progress Status
Introduction
What is recursion?
Recursive definitions
What you learned
Fibonacci numbers
Fibonacci numbers
Greatest common divider
Lowercase letters
Towers of Hanoi

Until now, most examples and exercises have been using numbers. You might have been wondering by this time if programming is just for number manipulation. In daily life, it is far more commonplace to deal with textual information.

The reason that dealing with texts was postponed until this point, is that dealing with numbers is simply easier than dealing with texts. But in the present chapter, the first steps are taken to learn to manipulate textual information.

Texts, in programming languages, are dealt with in the form of strings. This chapter is on the details of strings, and on readily-available functions to juggle them.

Title Class progress Status
Introduction
What you already know about strings
Multi-line strings
Escape sequences
Accessing characters of a string
Strings are immutable
String methods
Character encoding
What you learned
Counting vowels
Between brackets
Rotate a circulair alphabet
Counting words
ASCII order
Autocorrect

A tuple is a group of one or more values that are treated as a whole. This chapter explains how to recognize and use tuples.

Title Class progress Status
Introduction
Using tuples
Tuples are immutable
Applications of tuples
What you learned
Add complex numbers
Multiply complex numbers
Inttuple

Lists are ordered collections of data items, just like tuples. The major difference with tuples is that lists are mutable. This makes them a highly flexible data structure, that you will find many uses for.

Title Class progress Status
Introduction
List basics
Lists are mutable
Lists and operators
List methods
Aliasing
Nested lists
List casting
List comprehensions
What you learned
Shuffled card deck
FIFO (TODO)
Counting letters (TODO)
Sieve of Eratosthenes (TODO)
Tic-Tac-Toe (TODO)
Battleship (TODO)
Subset sum (TODO)

Strings, tuples and lists are ordered data structures, which entails that they can be indexed. Not all data is naturally ordered, which is why Python offers dictionaries as a way to structure unordered data.

Title Class progress Status
Introduction
Basics of dictionaries
Dictionary methods
Keys
Storing complicated values
Lookup speed
What you learned
Counting words
Movie ratings
Library (TODO)

Sets are unordered data structures, which contain only unique elements. Few programming languages support sets natively, but Python is one of them. Sets are not used often, but occasionally they provide a nice solution to a problem that you are trying to solve, when you need to ensure that you only have unique solution items.

Title Class progress Status
Introduction
Basics of sets
Set methods
Frozensets
What you learned
Socrates
Divisible

Until now, we considered Python programs as self-contained functionalities. Python programs, however, run on a computer, and occasionally the program must deal with computer intricacies. This will start to play a major role from Chapter 16 onward, when I will discuss file handling. To deal with the computer, Python offers a series of standard functionalities in the os module, whereby os is a common abbreviation for “operating system.” This chapter explains the most important functions from the os module.

Title Class progress Status
Introduction
Basics of operating systems
Command prompt
File system
OS functions
What you learned
Directory listing

One of the most important uses of Python for data processing is the reading, changing, and writing of text files. Data is often stored in text files, because text files can be easily transferred between different programs. There are multiple standardized formats for text files, such as “comma-separated values” (CSV) files. Python supports particular text file formats through modules, some of which will be discussed later. This chapter focusses on opening, reading, writing, and closing of any text file, regardless of format.

Title Class progress Status
Introduction
Flat text files
Reading text files
Writing text files
Appending to text files
os.path methods
File encoding
What you learned
Counting words
Counting words
Vowel removal
Common words
Letter frequencies

Sometimes runtime errors occur not because you made a programming mistake, but because your program encountered a situation that you could not foresee when you wrote it. This is particularly relevant when you deal with files: for instance, when you access a file on a USB-stick, and the user pulls out the USB-stick during the file processing, obviously an error occurs that you cannot really account for in your code. Every runtime error raises a so-called “exception,” and you can “capture” such exceptions if you want to deal with them in your program, rather than make the program end abruptly.

Title Class progress Status
Introduction
Errors and exceptions
Exception handling
File handling exceptions
Raising exceptions
What you learned
Check

“Binary files” is the term used to refer to all files that are not text files. Executable programs are binary files, as are image files, movies, word processor documents, and many other file types. It is not common to use Python to process binary files (usually binary files are not handled by general-purpose programming languages, but by special-purpose programs), but it is possible. This chapter will explain how to deal with binary files.

Title Class progress Status
Introduction
Opening and closing binary files
Reading a binary file
Writing a binary file
Positioning the file pointer
What you learned
Encryption (TODO)
Compression (TODO)
Decompression (TODO)
File compression (TODO)

Chapter 18 discussed dealing with binary files. When binary files are used, you are no longer working with characters and numbers; rather, you are working with bytes. To manipulate information on the level of bytes, Python offers a number of so-called “bitwise operators.” You will not need these often, but when you delve into binary file manipulation, they might come in handy.

Title Class progress Status
Introduction
Bits and bytes
Manipulating bits
Usefulness of bitwise operations
What you learned
Bitmask
Bitstorage

The chapters until this point covered an approach to programming that is often referred to as “structured programming” or “imperative programming,” wherein a program is considered a sequence of statements, decisions, and loops. You can solve any programming problem with a structured programming approach. However, in the last decades several other programming “paradigms” have been coined up, which help designing and implementing large-scale programs. One of the most successful paradigms is “object orientation,” and most modern programming languages support the object oriented paradigm. Python is, in fact, an object oriented language.

While object orientation tends to provide a natural way to look at problems and solutions, designing an object oriented program can be quite hard. The reason that it is hard, is that you have to really think about your approach to a problem in all of its aspects, before you start coding. For bigger problems, this can be daunting, especially when you lack experience with programming. However, for bigger problems you have to spend a lot of time designing your solution anyway, and an object oriented approach may be quite helpful in creating it. Moreover, you will find that most modules provide object oriented implementations, and that object orientation can be helpful for many smaller problems too.

Since object orientation is a broad topic, several chapters will be spent on it, of which this one is the first. It discusses the basics of object orientation, leaving the more specialized (and powereful!) aspects of object orientation for later chapters.

Title Class progress Status
Introduction
The object oriented world
Classes and objects
Methods
Nesting objects
What you learned
Rectangle
Enrollments

Operator overloading is a powerful feature of object orientation that allows you to integrate your new classes into programs in a natural way. Operator overloading is always based on the definition of some special methods, that have the typical __<name>__() structure.

Title Class progress Status
Introduction
The idea behind operator overloading
Comparisons
Calculations
Unary operators
Sequences
What you learned
Playing cards (TODO)
Pile of cards (TODO)
War (TODO)
Fruit basket (TODO)

Inheritance allows you to create new classes based on existing ones, just by indicating the difference. It is an extremely powerful concept that allows for the creation of highly flexible, easily maintainable programs.

Title Class progress Status
Introduction
Class inheritance
Interfaces
What you learned
Rectangle (TODO)
Shape (TODO)
Prisoner (TODO)

23. Iterators and generators

Iterators allow your classes to be used in for … in … statements. Generators are an easy way to create iterators.

Title Class progress Status
Introduction
Iterators
Generators
The itertools module
What you learned
Integers (TODO)
Factorials (TODO)
Anagrams (TODO)
Anagrams (TODO)
Subset sum (TODO)
Subdirectories (TODO)
Eight queens (TODO)

In Chapter 15 I mentioned that when handling large volumes of data spread over multiple files, you occasionally might want to write a Python program that supports command-line processing. This chapter tells you how to do that.

Title Class progress Status
Introduction
The command line
Flexible command line processing
What you learned
Sum (TODO)

When you are facing a problem with text mining, data processing, finding patterns in large collections of data, or scraping data from web pages, and you explore the Internet to find a solution to your problem, you will find that often the very first answer given to questions in this respect is “Why don’t you use regular expressions?” or even “Just use regex,” without further explanations. Rather smug answers, as many people have never heard of regular expressions, and if they have, might find them scary and incomprehensible. In fact, at first glance they come over as so arcane and confusing that most people rather shy away than delve into them. Which is a pity, as regular expressions are a powerful tool that should not be missing in the toolbox of anyone who deals with unstructured data on a regular basis.

In this chapter I will explain how to write and use basic regular expressions with Python. You will find them indeed a powerful way to quickly express and discover complex and diverse patterns in data, providing access to functionalities that would be very hard to implement in vanilla Python. While this chapter does not contain a complete overview of regular expressions, after studying it you will be able to understand and use regular expressions for most, if not all, pattern-matching problems that you encounter in practice, and be confident in telling the uninitiated: “You should use regular expressions to solve your problems.” Now you can feel smug too!

Title Class progress Status
Introduction
Regular expressions with Python
Writing regular expressions
Grouping
Replacing
What you learned
Words
The
Twofold names
Multifold names
Quotes
Web scraper

Data is usually stored in files, which are constructed according to a specific file format. Standardized file formats often are supported by Python using a particular module. In this chapter, I discuss some of more common file formats and the modules that support them.

Title Class progress Status
Introduction
Comma-Separated Values (CSV)
Pickling
JavaScript Object Notation (JSON)
HTML and XML
What you learned
Copy CSV file
Convert from CSV to JSON

At this point in the book, everything that you really need to know to be an accomplished Python programmer – or maybe even an accomplished programmer in general – has been covered. I want to quickly highlight a few useful modules that do not need a chapter of their own. I will not give many details; once you know what the purpose of a module is, you can look up more information on it in the Python reference.

Title Class progress Status
Introduction
Datetime
Collections
Urllib
Glob
What you learned
Counter (TODO)
Mode (TODO)
Title Class progress Status
Epilogue
Troubleshooting
Differences with Python 2
pcinput.py
pcmaze.py
Test Text Files
Answers to Exercises