Computers are wonderful machines. While most machines (cars, television sets, microwaves) have a specific purpose which they excel at accomplishing, computers are purposeless machines that can be taught to accomplish anything. The power to make a computer do your bidding is called “programming.”

Nowadays, in any scientific and professional endeavor, people have to deal with large volumes of data. Those who are able to leverage the power of computers to make use of such data, i.e., those who can program, are far better able to do their jobs than those who are not. In fact, it can be argued that in the very near future, those who do not possess programming skills will become unemployable. Therefore, I feel that it is necessary for anyone to acquire basic skills in this area during their education.

Being able to write computer programs not only entails knowing what specific code statements mean and do; it also entails having the ability think like a programmer, and to analyze problems from the perspective of solving them with a computer program. Such skills cannot be learned from a book, they can only be learned by actually creating programs.

This book has been designed to teach the basics of the Python 3 computer language. Students will not only learn to use the language, but also do their first practical exercises with it.

The book is not only catering towards people who are naturally inclined towards programming. It is meant to also be used by those who have no particular aptitude for programming. This is exemplified by texts which try to be extensive and foresee problems that might arise when trying to understand certain concepts.

As explained in the introduction, you will need to write and run Python code to learn anything from this book. That means that you need a computer on which Python is installed, and you need to know how to write and run Python programs. This chapter will explain to you how to get “up and running” with Python.

Welcome to the first real programming chapter. In this chapter I discuss “expressions,” which are straightforward calculations which you can also do with any simple calculator. It is a small start, but you are going to need such expressions for every chapter after this one.

When working with program code, very often you are designing a procedure (or “algorithm”) that solves a problem in a general way. For instance, in the previous chapter one of the exercises had you calculate the wholesale price for a stack of books, for a given book price and a given number of books. The code you wrote did not solve this problem for a general case, but only for the specific case of 60 books costing 24.95 per book. If you want to write code that solves problems in a more general way, you need to use variables that store values.

Up to this point, I have already introduced some basic “functions,” such as print() and int(). In this chapter these functions will be discussed a bit more in-depth, and a few other functions will be introduced, which will be helpful in the coming chapters. In Chapter 8, I will discuss how you can create your own functions.

In program code, there are often statements that you only want to execute when certain conditions hold. Every programming language therefore supports conditional statements. In this chapter I will explain how to use conditions in Python.

Computers do not get bored. If you want the computer to repeat a certain task hundreds of thousands of times, it does not protest. Humans hate too much repetition. Therefore, repetitious tasks should be performed by computers. All programming languages support repetitions. The general class of programming constructs that allow the definition of repetitions are called “iterations.” A term which is even more common for such tasks is “loops.”

This chapter explains all you need to know about loops in Python. Students who are completely new to programming often find loops the first really hard topic in programming that they encounter. If that is the case for you, then make sure you take your time for this chapter, and work on it until you understand it completely. Loops are such a basic concept in programming that you need to understand them in all their details. Each and every chapter after this one needs loops.

In Chapter 5 I described how to use simple functions, and how to import functions from modules. This chapter is about how to write your own functions and modules. If you do not remember what Chapter 5 said about functions, re-read that chapter before continuing with this one.

Recursion is a special technique that can be used now you are able to create and use functions. Recursion can be very elegant and powerful, but students often find it hard to employ. That is why I decided to spend a separate chapter on it. If, while studying this chapter, you feel that it is getting too complex for you, feel free to skip it for the time being. The following chapters are a lot easier again.

Until now, most examples and exercises have been using numbers. You might have been wondering by this time if programming is just for number manipulation. In daily life, it is far more commonplace to deal with textual information.

The reason that dealing with texts was postponed until this point, is that dealing with numbers is simply easier than dealing with texts. But in the present chapter, the first steps are taken to learn to manipulate textual information.

Texts, in programming languages, are dealt with in the form of strings. This chapter is on the details of strings, and on readily-available functions to juggle them.

A tuple is a group of one or more values that are treated as a whole. This chapter explains how to recognize and use tuples.

Lists are ordered collections of data items, just like tuples. The major difference with tuples is that lists are mutable. This makes them a highly flexible data structure, that you will find many uses for.

Strings, tuples and lists are ordered data structures, which entails that they can be indexed. Not all data is naturally ordered, which is why Python offers dictionaries as a way to structure unordered data.

Sets are unordered data structures, which contain only unique elements. Few programming languages support sets natively, but Python is one of them. Sets are not used often, but occasionally they provide a nice solution to a problem that you are trying to solve, when you need to ensure that you only have unique solution items.

Until now, we considered Python programs as self-contained functionalities. Python programs, however, run on a computer, and occasionally the program must deal with computer intricacies. This will start to play a major role from Chapter 16 onward, when I will discuss file handling. To deal with the computer, Python offers a series of standard functionalities in the os module, whereby os is a common abbreviation for “operating system.” This chapter explains the most important functions from the os module.

One of the most important uses of Python for data processing is the reading, changing, and writing of text files. Data is often stored in text files, because text files can be easily transferred between different programs. There are multiple standardized formats for text files, such as “comma-separated values” (CSV) files. Python supports particular text file formats through modules, some of which will be discussed later. This chapter focusses on opening, reading, writing, and closing of any text file, regardless of format.

Sometimes runtime errors occur not because you made a programming mistake, but because your program encountered a situation that you could not foresee when you wrote it. This is particularly relevant when you deal with files: for instance, when you access a file on a USB-stick, and the user pulls out the USB-stick during the file processing, obviously an error occurs that you cannot really account for in your code. Every runtime error raises a so-called “exception,” and you can “capture” such exceptions if you want to deal with them in your program, rather than make the program end abruptly.

“Binary files” is the term used to refer to all files that are not text files. Executable programs are binary files, as are image files, movies, word processor documents, and many other file types. It is not common to use Python to process binary files (usually binary files are not handled by general-purpose programming languages, but by special-purpose programs), but it is possible. This chapter will explain how to deal with binary files.

Chapter 18 discussed dealing with binary files. When binary files are used, you are no longer working with characters and numbers; rather, you are working with bytes. To manipulate information on the level of bytes, Python offers a number of so-called “bitwise operators.” You will not need these often, but when you delve into binary file manipulation, they might come in handy.

The chapters until this point covered an approach to programming that is often referred to as “structured programming” or “imperative programming,” wherein a program is considered a sequence of statements, decisions, and loops. You can solve any programming problem with a structured programming approach. However, in the last decades several other programming “paradigms” have been coined up, which help designing and implementing large-scale programs. One of the most successful paradigms is “object orientation,” and most modern programming languages support the object oriented paradigm. Python is, in fact, an object oriented language.

While object orientation tends to provide a natural way to look at problems and solutions, designing an object oriented program can be quite hard. The reason that it is hard, is that you have to really think about your approach to a problem in all of its aspects, before you start coding. For bigger problems, this can be daunting, especially when you lack experience with programming. However, for bigger problems you have to spend a lot of time designing your solution anyway, and an object oriented approach may be quite helpful in creating it. Moreover, you will find that most modules provide object oriented implementations, and that object orientation can be helpful for many smaller problems too.

Since object orientation is a broad topic, several chapters will be spent on it, of which this one is the first. It discusses the basics of object orientation, leaving the more specialized (and powereful!) aspects of object orientation for later chapters.

Operator overloading is a powerful feature of object orientation that allows you to integrate your new classes into programs in a natural way. Operator overloading is always based on the definition of some special methods, that have the typical __<name>__() structure.

Inheritance allows you to create new classes based on existing ones, just by indicating the difference. It is an extremely powerful concept that allows for the creation of highly flexible, easily maintainable programs.

Iterators allow your classes to be used in for … in … statements. Generators are an easy way to create iterators.

In Chapter 15 I mentioned that when handling large volumes of data spread over multiple files, you occasionally might want to write a Python program that supports command-line processing. This chapter tells you how to do that.

When you are facing a problem with text mining, data processing, finding patterns in large collections of data, or scraping data from web pages, and you explore the Internet to find a solution to your problem, you will find that often the very first answer given to questions in this respect is “Why don’t you use regular expressions?” or even “Just use regex,” without further explanations. Rather smug answers, as many people have never heard of regular expressions, and if they have, might find them scary and incomprehensible. In fact, at first glance they come over as so arcane and confusing that most people rather shy away than delve into them. Which is a pity, as regular expressions are a powerful tool that should not be missing in the toolbox of anyone who deals with unstructured data on a regular basis.

In this chapter I will explain how to write and use basic regular expressions with Python. You will find them indeed a powerful way to quickly express and discover complex and diverse patterns in data, providing access to functionalities that would be very hard to implement in vanilla Python. While this chapter does not contain a complete overview of regular expressions, after studying it you will be able to understand and use regular expressions for most, if not all, pattern-matching problems that you encounter in practice, and be confident in telling the uninitiated: “You should use regular expressions to solve your problems.” Now you can feel smug too!

Data is usually stored in files, which are constructed according to a specific file format. Standardized file formats often are supported by Python using a particular module. In this chapter, I discuss some of more common file formats and the modules that support them.

At this point in the book, everything that you really need to know to be an accomplished Python programmer – or maybe even an accomplished programmer in general – has been covered. I want to quickly highlight a few useful modules that do not need a chapter of their own. I will not give many details; once you know what the purpose of a module is, you can look up more information on it in the Python reference.