User:Yasondinalt/Python Programming-Print version
This is the print version of Python Programming You won't see this message or any elements not part of the book's content when you print or preview this page. |
Python is a general purpose programming language.
Table of contents
Introduction
Learning to program in Python
Python concepts
Rocking the Python (Modules)
- Regular Expression
- Graphical User Interfaces in Python
- Python Programming/Game Programming in Python
- Socket programming
- Files (I/O)
- Databases
- Extracting info from web pages
- Threading
- Extending with C
- Extending with C++
- Extending with ctypes
- WSGI web programming
References
Authors
License
Overview
Python is a high-level, structured, open-source programming language that can be used for a wide variety of programming tasks. Python was created by Guido Van Rossum in the early 1990s; its following has grown steadily and interest has increased markedly in the last few years or so. It is named after Monty Python's Flying Circus comedy program.
Python is used extensively for system administration (many vital components of Linux distributions are written in it); also, it is a great language to teach programming to novices. NASA has used Python for its software systems and has adopted it as the standard scripting language for its Integrated Planning System. Python is also extensively used by Google to implement many components of its Web Crawler and Search Engine & Yahoo! for managing its discussion groups.
Python within itself is an interpreted programming language that is automatically compiled into bytecode before execution (the bytecode is then normally saved to disk, just as automatically, so that compilation need not happen again until and unless the source gets changed). It is also a dynamically typed language that includes (but does not require one to use) object-oriented features and constructs.
The most unusual aspect of Python is that whitespace is significant; instead of block delimiters (braces → "{}" in the C family of languages), indentation is used to indicate where blocks begin and end.
For example, the following Python code can be interactively typed at an interpreter prompt, display the famous "Hello World!" on the user screen:
>>> print ("Hello World!")
Hello World!
Another great feature of Python is its availability for all platforms. Python can run on Microsoft Windows, Macintosh and all Linux distributions with ease. This makes the programs very portable, as any program written for one platform can easily be used on another.
Python provides a powerful assortment of built-in types (e.g., lists, dictionaries and strings), a number of built-in functions, and a few constructs, mostly statements. For example, loop constructs that can iterate over items in a collection instead of being limited to a simple range of integer values. Python also comes with a powerful standard library, which includes hundreds of modules to provide routines for a wide variety of services including regular expressions and TCP/IP sessions.
Python is used and supported by a large Python Community that exists on the Internet. The mailing lists and news groups like the tutor list actively support and help new python programmers. While they discourage doing homework for you, they are quite helpful and are populated by the authors of many of the Python textbooks currently available on the market.
Python 2 vs. Python 3: Years ago, the Python developers made the decision to come up with a major new version of Python, which became the 3.x series of versions. The 3.x versions are backward-incompatible with Python 2.x: certain old features (like the handling of Unicode strings) were deemed to be too unwieldy or broken to be worth carrying forward. Instead, new, cleaner ways of achieving the same results were added. See also Python 2 vs. Python 3 chapter.
Getting Python
To program in Python, you need a Python interpreter to run your code—we will discuss interpreters later. If it's not already installed, or if the version you are using is obsolete, you will need to obtain and install Python using the methods below. The current Python versions are 3.x; versions 2.x are discontinued and no longer maintained.
Installing Python in Windows
Go to the Python Homepage and get the proper version for your platform. Download it, read the instructions and get it installed.
To run Python from the command line, you will need to have the python directory in your PATH. You can instruct the Python installer to add Python to the path, but if you do not do that, you can add it manually. The PATH variable can be modified from the Window's System control panel. To expand the PATH in Windows 7:
- Go to Start.
- Right click on computer.
- Click on properties.
- Click on 'Advanced System Settings'
- Click on 'Environmental Variables'.
- In the system variables select Path and edit it, by appending a ';' (without quote) and adding 'C:\python27'(without quote).
If you prefer having a temporary environment, you can create a new command prompt short-cut that automatically executes the following statement:
PATH %PATH%;c:\python27
If you downloaded a different version (such as Python 3.1), change the "27" for the version of Python you have (27 is 2.7.x, the current version of Python 2.)
Cygwin
By default, the Cygwin installer for Windows does not include Python in the downloads. However, it can be selected from the list of packages.
Installing Python on Mac
Users on Mac OS X will find that it already ships with Python 2.3 (OS X 10.4 Tiger) or Python 2.6.1 (OS X Snow Leopard), but if you want the more recent version head to Python Download Page follow the instruction on the page and in the installers. As a bonus you will also install the Python IDE.
Installing Python on Unix environments
Python is available as a package for most Linux distributions. In some cases, the distribution CD will contain the python package for installation, while other distributions require downloading the source code and using the compilation scripts.
Gentoo Linux
Gentoo includes Python by default—the package management system Portage depends on Python.
Ubuntu Linux
Users of Ubuntu will notice that Python comes installed by default, only it sometimes is not the latest version. To check which version of Python is installed, type
python -V
into the terminal.
Arch Linux
Arch Linux does not come with Python pre-installed by default, but it is easily available for installation through the package manager to pacman. As root (or using sudo if you've installed and configured it), type:
pacman -S python
This will be update package databases and install Python 3. Python 2 can be installed with:
pacman -S python2
Other versions can be built from source from the Arch User Repository.
Source code installations
Some platforms do not have a version of Python installed, and do not have pre-compiled binaries. In these cases, you will need to download the source code from the official site. Once the download is complete, you will need to unpack the compressed archive into a folder.
To build Python, simply run the configure script (requires the Bash shell) and compile using make.
Other Distributions
Python, also referred to as CPython to avoid confusion, is written in the C programming language, and is the official reference implementation. CPython can run on various platforms due to its portability.
Apart from CPython there are also other implementations that run on top of a virtual machine. For example, on Java's JRE (Java Runtime Environment) or Microsoft's .NET CLR (Common Language Runtime). Both can access and use the libraries available on their platform. Specifically, they make use of reflection that allows complete inspection and use of all classes and objects for their very technology.
Python Implementations (Platforms)
Environment | Description | Get From |
---|---|---|
Jython | Java Version of Python | Jython |
IronPython | C# Version of Python | IronPython |
Integrated Development Environments (IDE)
It's common to use a simple text editor for writing Python code, but you may feel the need to upgrade to a more advanced IDE. CPython ships with IDLE; however, IDLE is not considered user-friendly.[1] For Linux, KDevelop and Spyder are popular. For Windows, PyScripter is free, quick to install, and comes included with PortablePython.
Some Integrated Development Environments (IDEs) for Python
Environment | Description | Get From |
---|---|---|
ActivePython | Highly flexible, Pythonwin IDE | ActivePython |
Anjuta | IDE Linux/Unix | Anjuta |
Eclipse (PyDev plugin) | Open-source IDE | Eclipse |
Eric | Open-source Linux/Windows IDE. | Eric |
KDevelop | Cross-language IDE for KDE | KDevelop |
Ninja-IDE | Cross-platform open-source IDE. | Nina-IDE |
PyScripter | Free Windows IDE (portable) | PyScripter |
Pythonwin | Windows-oriented environment | Pythonwin |
Spyder | Free cross-platform IDE (math-oriented) | Spyder |
VisualWx | Free GUI Builder | VisualWx |
The Python official wiki has a complete list of IDEs.
There are several commercial IDEs such as Komodo, BlackAdder, Code Crusader, Code Forge, and PyCharm. However, for beginners learning to program, purchasing a commercial IDE is unnecessary.
Trying Python online
You can try Python online, thereby avoiding the need to install. The online Python shell at Python's official site provides a web Python REPL (read–eval–print loop).
Keeping Up to Date
Python has a very active community and the language itself is evolving continuously. Make sure to check python.org for recent releases and relevant tools. The website is an invaluable asset.
Public Python-related mailing lists are hosted at mail.python.org. Two examples of such mailing lists are the Python-announce-list to keep up with newly released third party-modules or software for Python and the general discussion list Python-list. These lists are mirrored to the Usenet newsgroups comp.lang.python.announce & comp.lang.python.
Notes
Interactive mode
Python has two basic modes: script and interactive. The normal mode is the mode where the scripted and finished .py
files are run in the Python interpreter. Interactive mode is a command line shell which gives immediate feedback for each statement, while running previously fed statements in active memory. As new lines are fed into the interpreter, the fed program is evaluated both in part and in whole.
Interactive mode is a good way to play around and try variations on syntax.
On macOS or linux, open a terminal and simply type "python". On Windows, bring up the command prompt and type "py", or start an interactive Python session by selecting "Python (command line)", "IDLE", or similar program from the task bar / app menu. IDLE is a GUI which includes both an interactive mode and options to edit and run files.
Python should print something like this:
$ python Python 3.0b3 (r30b3:66303, Sep 8 2008, 14:01:02) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>>
(If Python doesn't run, make sure it is installed and your path is set correctly. See Getting Python.)
The >>> is Python's way of telling you that you are in interactive mode. In interactive mode what you type is immediately run. Try typing 1+1 in. Python will respond with 2. Interactive mode allows you to test out and see what Python will do. If you ever feel the need to play with new Python statements, go into interactive mode and try them out.
A sample interactive session:
>>> 5 5 >>> print(5*7) 35 >>> "hello" * 2 'hellohello' >>> "hello".__class__ <type 'str'>
However, you need to be careful in the interactive environment to avoid confusion. For example, the following is a valid Python script:
if 1:
print("True")
print("Done")
If you try to enter this as written in the interactive environment, you might be surprised by the result:
>>> if 1: ... print("True") ... print("Done") File "<stdin>", line 3 print("Done") ^ SyntaxError: invalid syntax
What the interpreter is saying is that the indentation of the second print was unexpected. You should have entered a blank line to end the first (i.e., "if") statement, before you started writing the next print statement. For example, you should have entered the statements as though they were written:
if 1:
print("True")
print("Done")
Which would have resulted in the following:
>>> if 1: ... print("True") ... True >>> print("Done") Done >>>
Interactive mode
Instead of Python exiting when the program is finished, you can use the -i flag to start an interactive session. This can be very useful for debugging and prototyping.
python -i hello.py
For i in range(-1,-5,-1):
print(i)
Creating Python programs
Welcome to Python! This tutorial will show you how to start writing programs.
Python programs are nothing more than text files, and they may be edited with a standard text editor program.[1] What text editor you use will probably depend on your operating system: any text editor can create Python programs. However, it is easier to use a text editor that includes Python syntax highlighting.
Hello, World
The very first program that beginning programmers usually write or learn is the "Hello, World!" program. This program simply outputs the phrase "Hello, World!" then terminates itself. Let's write "Hello, World!" in Python!
Open up your text editor and create a new file called hello.py
containing just this line (you can copy-paste if you want):
print('Hello, World!')
The below line is used for Python 3.x.x
print("Hello, World!")
You can also put the below line to pause the program at the end until you press anything.
input()
This program uses the print
function, which simply outputs its parameters to the terminal. By default, print
appends a newline
character to its output, which simply moves the cursor to the next line.
Now that you've written your first program, let's run it in Python! This process differs slightly depending on your operating system.
Windows
- Create a folder on your computer to use for your Python programs, such as
C:\pythonpractice
, and save yourhello.py
program in that folder. - In the Start menu, select "Run...", and type in
cmd
. This will cause the Windows terminal to open. - Type
cd \pythonpractice
to change directory to yourpythonpractice
folder, and hit Enter. - Type
hello.py
to run your program!
If it didn't work, make sure your PATH contains the python directory. See Getting Python.
Mac
- Create a folder on your computer to use for your Python programs. A good suggestion would be to name it
pythonpractice
and place it in your Home folder (the one that contains folders for Documents, Movies, Music, Pictures, etc). Save yourhello.py
program into it. Open the Applications folder, go into the Utilities folder, and open the Terminal program. - Type
cd pythonpractice
to change directory to yourpythonpractice
folder, and hit Enter. - Type
python ./hello.py
to run your program!
Linux
- Create a folder on your computer to use for your Python programs, such as
~/pythonpractice
, and save yourhello.py
program in that folder. - Open up the terminal program. In KDE, open the main menu and select "Run Command..." to open Konsole. In GNOME, open the main menu, open the Applications folder, open the Accessories folder, and select Terminal.
- Type
cd ~/pythonpractice
to change directory to yourpythonpractice
folder, and hit Enter. - Don't forget to make the script executable by chmod +x.
- Type
python ./hello.py
to run your program!
Linux (advanced)
- Create a folder on your computer to use for your Python programs, such as
~/pythonpractice
.
- Open up your favorite text editor and create a new file called
hello.py
containing just the following 2 lines (you can copy-paste if you want):[2]
#! /usr/bin/python
print('Hello, world!')
- save your
hello.py
program in the~/pythonpractice
folder. - Open up the terminal program. In KDE, open the main menu and select "Run Command..." to open Konsole. In GNOME, open the main menu, open the Applications folder, open the Accessories folder, and select Terminal.
- Type
cd ~/pythonpractice
to change directory to yourpythonpractice
folder, and hit Enter. - Type
chmod a+x hello.py
to tell Linux that it is an executable program. - Type
./hello.py
to run your program! - In addition, you can also use
ln -s hello.py /usr/bin/hello
to make a symbolic linkhello.py
to/usr/bin
under the namehello
, then run it by simply executinghello
.
Note that this mainly should be done for complete, compiled programs, if you have a script that you made and use frequently, then it might be a good idea to put it somewhere in your home directory and put a link to it in /usr/bin. If you want a playground, a good idea is to invoke mkdir ~/.local/bin
and then put scripts in there. To make ~/.local/bin content executable the same way /usr/bin does type $PATH = $PATH:~/local/bin
(you can add this line to your shell rc file, for example ~/.bashrc).
Result
The program should print:
Hello, world!
Congratulations! You're well on your way to becoming a Python programmer.
Exercises
- Modify the
hello.py
program to say hello to someone from your family or your friends (or to Ada Lovelace). - Change the program so that after the greeting, it asks, "How did you get here?".
- Re-write the original program to use two
print
statements: one for "Hello" and one for "world". The program should still only print out on one line.
Notes
- ↑ Sometimes, Python programs are distributed in compiled form. We won't have to worry about that for quite a while.
- ↑ A Quick Introduction to Unix/My First Shell Script explains what a hash bang line does.
Basic syntax
There are five fundamental concepts in Python.
Semicolons
Python does not normally use semicolons, but they are allowed to separate statements on the same line, if your code has semicolons; your code isn't "Pythonic"
Case Sensitivity
All variables are case-sensitive. Python treats 'number' and 'Number' as separate, unrelated entities.
Spaces and tabs don't mix
Instead of block delimiters (braces → "{}" in the C family of languages), indentation is used to indicate where blocks begin and end. Because whitespace is significant, remember that spaces and tabs don't mix, so use only one or the other when indenting your programs. A common error is to mix them. While they may look the same in editor, the interpreter will read them differently and it will result in either an error or unexpected behavior. Most decent text editors can be configured to let tab key emit spaces instead.
Python's Style Guideline described that the preferred way is using 4 spaces.
Tips: If you invoked python from the command-line, you can give -t or -tt argument to python to make python issue a warning or error on inconsistent tab usage.
pythonprogrammer@wikibook:~$ python -tt myscript.py
This will issue an error if you have mixed spaces and tabs.
Objects
In Python, like all object-oriented languages, there are aggregations of code and data called objects, which typically represent the pieces in a conceptual model of a system.
Objects in Python are created (i.e., instantiated) from templates called classes (which are covered later, as much of the language can be used without understanding classes). They have attributes, which represent the various pieces of code and data which make up the object. To access attributes, one writes the name of the object followed by a period (henceforth called a dot), followed by the name of the attribute.
An example is the 'upper' attribute of strings, which refers to the code that returns a copy of the string in which all the letters are uppercase. To get to this, it is necessary to have a way to refer to the object (in the following example, the way is the literal string that constructs the object).
'bob'.upper
Code attributes are called methods. So in this example, upper is a method of 'bob' (as it is of all strings). To execute the code in a method, use a matched pair of parentheses surrounding a comma separated list of whatever arguments the method accepts (upper doesn't accept any arguments). So to find an uppercase version of the string 'bob', one could use the following:
'bob'.upper()
Scope
In a large system, it is important that one piece of code does not affect another in difficult to predict ways. One of the simplest ways to further this goal is to prevent one programmer's choice of a name from blocking another's use of that name. The concept of scope was invented to do this. A scope is a "region" of code in which a name can be used and outside of which the name cannot be easily accessed. There are two ways of delimiting regions in Python: functions or modules. They each have different ways of providing their content outside of their scope. Functions return data as the result of execution. Modules leads us to another concept, namespace.
Namespaces
It would be possible to teach Python without the concept of namespaces because they are so similar to attributes, which we have already mentioned, but the concept of namespaces is one that transcends any particular programming language, and so it is important to teach. To begin with, there is a built-in function dir() that can be used to help one understand the concept of namespaces. When you first start the Python interpreter (i.e., in interactive mode), you can list the objects in the current (or default) namespace using this function.
Python 2.3.4 (#53, Oct 18 2004, 20:35:07) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> dir()
['__builtins__', '__doc__', '__name__']
This function can also be used to show the names available within a module's namespace. To demonstrate this, first we can use the type() function to show what kind of object __builtins__ is:
>>> type(__builtins__)
<type 'module'>
Since it is a module, it has a namespace. We can list the names within the __builtins__ namespace, again using the dir() function (note that the complete list of names has been abbreviated):
>>> dir(__builtins__)
['ArithmeticError', ... 'copyright', 'credits', ... 'help', ... 'license', ... 'zip']
>>>
Namespaces are a simple concept. A namespace is a particular place in which names specific to a module reside. Each name within a namespace is distinct from names outside of that namespace. This layering of namespaces is called scope. A name is placed within a namespace when that name is given a value. For example:
>>> dir()
['__builtins__', '__doc__', '__name__']
>>> name = "Bob"
>>> import math
>>> dir()
['__builtins__', '__doc__', '__name__', 'math', 'name']
Note that I was able to add the "name" variable to the namespace using a simple assignment statement. The import statement was used to add the "math" name to the current namespace. To see what math is, we can simply:
>>> math
<module 'math' (built-in)>
Since it is a module, it also has a namespace. To display the names within this namespace, we:
>>> dir(math)
['__doc__', '__name__', 'acos', 'asin', 'atan', 'atan2', 'ceil', 'cos', 'cosh', 'degrees', 'e',
'exp', 'fabs', 'floor', 'fmod', 'frexp', 'hypot', 'ldexp', 'log', 'log10', 'modf', 'pi', 'pow',
'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh']
>>>
If you look closely, you will notice that both the default namespace and the math module namespace have a '__name__' object. The fact that each layer can contain an object with the same name is what scope is all about. To access objects inside a namespace, simply use the name of the module, followed by a dot, followed by the name of the object. This allows us to differentiate between the __name__ object within the current namespace, and that of the object with the same name within the math module. For example:
>>> print (__name__)
__main__
>>> print (math.__name__)
math
>>> print (math.__doc__)
This module is always available. It provides access to the
mathematical functions defined by the C standard.
>>> print (math.pi)
3.1415926535897931
Data types
Data types determine whether an object can do something, or whether it just would not make sense. Other programming languages often determine whether an operation makes sense for an object by making sure the object can never be stored somewhere where the operation will be performed on the object (this type system is called static typing). Python does not do that. Instead it stores the type of an object with the object, and checks when the operation is performed whether that operation makes sense for that object (this is called dynamic typing).
Built-in Data types
Python's built-in (or standard) data types can be grouped into several classes. Sticking to the hierarchy scheme used in the official Python documentation these are numeric types, sequences, sets and mappings (and a few more not discussed further here). Some of the types are only available in certain versions of the language as noted below.
- boolean: the type of the built-in values
True
andFalse
. Useful in conditional expressions, and anywhere else you want to represent the truth or falsity of some condition. Mostly interchangeable with the integers 1 and 0. In fact, conditional expressions will accept values of any type, treating special ones like booleanFalse
, integer 0 and the empty string""
as equivalent toFalse
, and all other values as equivalent toTrue
.
Numeric types:
- int: Integers; equivalent to C longs in Python 2.x, non-limited length in Python 3.x
- long: Long integers of non-limited length; exists only in Python 2.x
- float: Floating-Point numbers, equivalent to C doubles
- complex: Complex Numbers
Sequences:
- str: String; represented as a sequence of 8-bit characters in Python 2.x, but as a sequence of Unicode characters (in the range of U+0000 - U+10FFFF) in Python 3.x
- bytes: a sequence of integers in the range of 0-255; only available in Python 3.x
- byte array: like bytes, but mutable (see below); only available in Python 3.x
- list
- tuple
Sets:
- set: an unordered collection of unique objects; available as a standard type since Python 2.6
- frozen set: like set, but immutable (see below); available as a standard type since Python 2.6
Mappings:
- dict: Python dictionaries, also called hashmaps or associative arrays, which means that an element of the list is associated with a definition, rather like a Map in Java
Some others, such as type and callables
Mutable vs Immutable Objects
In general, data types in Python can be distinguished based on whether objects of the type are mutable or immutable. The content of objects of immutable types cannot be changed after they are created.
Some immutable types | Some mutable types |
---|---|
|
|
Only mutable objects support methods that change the object in place, such as reassignment of a sequence slice, which will work for lists, but raise an error for tuples and strings.
It is important to understand that variables in Python are really just references to objects in memory. If you assign an object to a variable as below,
a = 1
s = 'abc'
l = ['a string', 456, ('a', 'tuple', 'inside', 'a', 'list')]
all you really do is make this variable (a, s, or l) point to the object (1, 'abc', ['a string', 456, ('a', 'tuple', 'inside', 'a', 'list')]), which is kept somewhere in memory, as a convenient way of accessing it. If you reassign a variable as below
a = 7
s = 'xyz'
l = ['a simpler list', 99, 10]
you make the variable point to a different object (newly created ones in our examples). As stated above, only mutable objects can be changed in place (l[0] = 1 is ok in our example, but s[0] = 'a' raises an error). This becomes tricky, when an operation is not explicitly asking for a change to happen in place, as is the case for the += (increment) operator, for example. When used on an immutable object (as in a += 1 or in s += 'qwertz'), Python will silently create a new object and make the variable point to it. However, when used on a mutable object (as in l += [1,2,3]), the object pointed to by the variable will be changed in place. While in most situations, you do not have to know about this different behavior, it is of relevance when several variables are pointing to the same object. In our example, assume you set p = s and m = l, then s += 'etc' and l += [9,8,7]. This will change s and leave p unaffected, but will change both m and l since both point to the same list object. Python's built-in id() function, which returns a unique object identifier for a given variable name, can be used to trace what is happening under the hood.
Typically, this behavior of Python causes confusion in functions. As an illustration, consider this code:
def append_to_sequence (myseq):
myseq += (9,9,9)
return myseq
tuple1 = (1,2,3) # tuples are immutable
list1 = [1,2,3] # lists are mutable
tuple2 = append_to_sequence(tuple1)
list2 = append_to_sequence(list1)
print('tuple1 = ', tuple1) # outputs (1, 2, 3)
print('tuple2 = ', tuple2) # outputs (1, 2, 3, 9, 9, 9)
print('list1 = ', list1) # outputs [1, 2, 3, 9, 9, 9]
print('list2 = ', list2) # outputs [1, 2, 3, 9, 9, 9]
This will give the above indicated, and usually unintended, output. myseq is a local variable of the append_to_sequence function, but when this function gets called, myseq will nevertheless point to the same object as the variable that we pass in (t or l in our example). If that object is immutable (like a tuple), there is no problem. The += operator will cause the creation of a new tuple, and myseq will be set to point to it. However, if we pass in a reference to a mutable object, that object will be manipulated in place (so myseq and l, in our case, end up pointing to the same list object).
Links:
- 3.1. Objects, values and types, The Python Language Reference, docs.python.org
- 5.6.4. Mutable Sequence Types, The Python Standard Library, docs.python.org
Creating Objects of Defined Types
Literal integers can be entered in three ways:
- decimal numbers can be entered directly
- hexadecimal numbers can be entered by prepending a 0x or 0X (0xff is hex FF, or 255 in decimal)
- the format of octal literals depends on the version of Python:
- Python 2.x: octals can be entered by prepending a zero ( 0 ) (0732 is octal 732, or 474 in decimal)
- Python 3.x: octals can be entered by prepending a zero followed by the letter O (0o or 0O) (0o732 is octal 732, or 474 in decimal)
Floating point numbers can be entered directly.
Long integers are entered either directly (1234567891011121314151617181920 is a long integer) or by appending an L (0L is a long integer). Computations involving short integers that overflow are automatically turned into long integers.
Complex numbers are entered by adding a real number and an imaginary one, which is entered by appending a j (i.e. 10+5j is a complex number. So is 10j). Note that j by itself does not constitute a number. If this is desired, use 1j.
Strings can be either single or triple quoted strings. The difference is in the starting and ending delimiters, and in that single quoted strings cannot span more than one line. Single quoted strings are entered by entering either a single quote (') or a double quote (") followed by its match. So therefore
'foo' works, and "moo" works as well, but 'bar" does not work, and "baz' does not work either. "quux'' is right out.
Triple quoted strings are like single quoted strings, but can span more than one line. Their starting and ending delimiters must also match. They are entered with three consecutive single or double quotes, so
'''foo''' works, and """moo""" works as well, but '"'bar'"' does not work, and """baz''' does not work either. '"'quux"'" is right out.
Tuples are entered in parentheses, with commas between the entries:
(10, 'Mary had a little lamb')
Also, the parenthesis can be left out when it's not ambiguous to do so:
10, 'whose fleece was as white as snow'
Note that one-element tuples can be entered by surrounding the entry with parentheses and adding a comma like so:
('this is a singleton tuple',)
Lists are similar, but with brackets:
['abc', 1,2,3]
Dicts are created by surrounding with curly braces a list of key/value pairs separated from each other by a colon and from the other entries with commas:
{ 'hello': 'world', 'weight': 'African or European?' }
Any of these composite types can contain any other, to any depth:
((((((((('bob',),['Mary', 'had', 'a', 'little', 'lamb']), { 'hello' : 'world' } ),),),),),),)
Null object
The Python analogue of null pointer known from other programming languages is None. None is not a null pointer or a null reference but an actual object of which there is only one instance. One of the uses of None is in default argument values of functions, for which see User:Yasondinalt/Functions#Default_Argument_Values. Comparisons to None are usually made using is rather than ==.
Testing for None and assignment:
if item is None:
...
another = None
if not item is None:
...
if item is not None: # Also possible
...
Using None in a default argument value:
def log(message, type = None):
...
PEP8 states that "Comparisons to singletons like None should always be done with is or is not, never the equality operators." Therefore, "if item == None:" is inadvisable. A class can redefine the equality operator (==) such that instances of it will equal None.
You can verify that None is an object by dir(None) or id(None).
See also Operators#Identity chapter.
Links:
- 4. Built-in Constants, docs.python.org
- 3.11.7 The Null Object, docs.python.org
- Python None comparison: should I use “is” or ==?, stackoverflow.com
- PEP 0008 -- Style Guide for Python Code, python.org
Type conversion
Type conversion in Python by example:
v1 = int(2.7) # 2
v2 = int(-3.9) # -3
v3 = int("2") # 2
v4 = int("11", 16) # 17, base 16
v5 = long(2) # Python 2.x only, not Python 3.x
v6 = float(2) # 2.0
v7 = float("2.7") # 2.7
v8 = float("2.7E-2") # 0.027
v9 = float(False) # 0.0
vA = float(True) # 1.0
vB = str(4.5) # "4.5"
vC = str([1, 3, 5]) # "[1, 3, 5]"
vD = bool(0) # False; bool fn since Python 2.2.1
vE = bool(3) # True
vF = bool([]) # False - empty list
vG = bool([False]) # True - non-empty list
vH = bool({}) # False - empty dict; same for empty tuple
vI = bool("") # False - empty string
vJ = bool(" ") # True - non-empty string
vK = bool(None) # False
vL = bool(len) # True
vM = set([1, 2])
vN = set((1, 2)) # Converts any sequence, not just a list
vO = set("abc") # {'c', 'b', 'a'}
vP = set(b"abc") # {97, 98, 99}
vQ = list(vM)
vR = list({1: "a", 2: "b"}) # dict -> list of keys
vS = tuple(vQ)
vT = list("abc") # ['a', 'b', 'c']
print(v1, v2, v3, type(v1), type(v2), type(v3))
Implicit type conversion:
int1 = 4
float1 = int1 + 2.1 # 4 converted to float
# str1 = "My int:" + int1 # Error: no implicit type conversion from int to string
str1 = "My int:" + str(int1)
int2 = 4 + True # 5: bool is implicitly converted to int
float2 = 4.5 + True # 5.5: True is converted to 1, which is converted to 1.0
Keywords: type casting.
Links:
- 2. Built-in Functions # bool, docs.python.org
- 2. Built-in Functions # list, docs.python.org
- 2. Built-in Functions # float, docs.python.org
- 2. Built-in Functions # int, docs.python.org
- 2. Built-in Functions # set, docs.python.org
- 2. Built-in Functions # str, docs.python.org
- 2. Built-in Functions # type, docs.python.org
- 2. Built-in Functions # tuple, docs.python.org
Exercises
- Write a program that instantiates a single object, adds [1,2] to the object, and returns the result.
- Find an object that returns an output of the same length (if one exists?).
- Find an object that returns an output length 2 greater than it started.
- Find an object that causes an error.
- Find two data types X and Y such that X = X + Y will cause an error, but X += Y will not.
Numbers
Python 2.x supports 4 built-in numeric types - int, long, float and complex. Of these, the long type has been dropped in Python 3.x - the int type is now of unlimited length by default. You don’t have to specify what type of variable you want; Python does that automatically.
- Int: The basic integer type in python, equivalent to the hardware 'c long' for the platform you are using in Python 2.x, unlimited in length in Python 3.x.
- Long: Integer type with unlimited length. In python 2.2 and later, Ints are automatically turned into long ints when they overflow. Dropped since Python 3.0, use int type instead.
- Float: This is a binary floating point number. Longs and Ints are automatically converted to floats when a float is used in an expression, and with the true-division / operator. In CPython, floats are usually implemented using the C languages double, which often yields 52 bits of significand, 11 bits of exponent, and 1 sign bit, but this is machine dependent.
- Complex: This is a complex number consisting of two floats. Complex literals are written as a + bj where a and b are floating-point numbers denoting the real and imaginary parts respectively.
In general, the number types are automatically 'up cast' in the following order:
Int → Long → Float → Complex. The farther to the right you go, the higher the precedence.
>>> x = 5
>>> type(x)
<type 'int'>
>>> x = 187687654564658970978909869576453
>>> type(x)
<type 'long'>
>>> x = 1.34763
>>> type(x)
<type 'float'>
>>> x = 5 + 2j
>>> type(x)
<type 'complex'>
The result of divisions is somewhat confusing. In Python 2.x, using the / operator on two integers will return another integer, using floor division. For example, 5/2 will give you 2. At least one of the operands must be float to get true division, e.g. 5/2. or 5./2 (the dot makes a number a float) will yield 2.5. Starting with Python 2.2 this behavior can be changed to true division by the future division statement from __future__ import division. In Python 3.x, the result of using the / operator is always true division (you can ask for floor division explicitly by using the // operator since Python 2.2).
This illustrates the behavior of the / operator in Python 2.2+:
>>> 5/2
2
>>> 5/2.
2.5
>>> 5./2
2.5
>>> from __future__ import division
>>> 5/2
2.5
>>> 5//2
2
For operations on numbers, see chapters Basic Math and Math.
Links
- 5.4. Numeric Types — int, float, long, complex, docs.python.org
Strings
Overview
Strings in Python at a glance:
str1 = "Hello" # A new string using double quotes
str2 = 'Hello' # Single quotes do the same
str3 = "Hello\tworld\n" # One with a tab and a newline
str4 = str1 + " world" # Concatenation
str5 = str1 + str(4) # Concatenation with a number
str6 = str1[2] # 3rd character
str6a = str1[-1] # Last character
#str1[0] = "M" # No way; strings are immutable
for char in str1: print(char) # For each character
str7 = str1[1:] # Without the 1st character
str8 = str1[:-1] # Without the last character
str9 = str1[1:4] # Substring: 2nd to 4th character
str10 = str1 * 3 # Repetition
str11 = str1.lower() # Lowercase
str12 = str1.upper() # Uppercase
str13 = str1.rstrip() # Strip right (trailing) whitespace
str14 = str1.replace('l','h') # Replacement
list15 = str1.split('l') # Splitting
if str1 == str2: print("Equ") # Equality test
if "el" in str1: print("In") # Substring test
length = len(str1) # Length
pos1 = str1.find('llo') # Index of substring or -1
pos2 = str1.rfind('l') # Index of substring, from the right
count = str1.count('l') # Number of occurrences of a substring
print(str1, str2, str3, str4, str5, str6, str7, str8, str9, str10)
print(str11, str12, str13, str14, list15)
print(length, pos1, pos2, count)
See also chapter Regular Expression for advanced pattern matching on strings in Python.
String operations
Equality
Two strings are equal if they have exactly the same contents, meaning that they are both the same length and each character has a one-to-one positional correspondence. Many other languages compare strings by identity instead; that is, two strings are considered equal only if they occupy the same space in memory. Python uses the is
operator to test the identity of strings and any two objects in general.
Examples:
>>> a = 'hello'; b = 'hello' # Assign 'hello' to a and b.
>>> a == b # check for equality
True
>>> a == 'hello' #
True
>>> a == "hello" # (choice of delimiter is unimportant)
True
>>> a == 'hello ' # (extra space)
False
>>> a == 'Hello' # (wrong case)
False
Numerical
There are two quasi-numerical operations which can be done on strings -- addition and multiplication. String addition is just another name for concatenation, which is simply sticking the strings together. String multiplication is repetitive addition, or concatenation. So:
>>> c = 'a'
>>> c + 'b'
'ab'
>>> c * 5
'aaaaa'
Containment
There is a simple operator 'in' that returns True if the first operand is contained in the second. This also works on substrings:
>>> x = 'hello'
>>> y = 'ell'
>>> x in y
False
>>> y in x
True
Note that 'print(x in y)' would have also returned the same value.
Indexing and Slicing
Much like arrays in other languages, the individual characters in a string can be accessed by an integer representing its position in the string. The first character in string s would be s[0] and the nth character would be at s[n-1].
>>> s = "Xanadu"
>>> s[1]
'a'
Unlike arrays in other languages, Python also indexes the arrays backwards, using negative numbers. The last character has index -1, the second to last character has index -2, and so on.
>>> s[-4]
'n'
We can also use "slices" to access a substring of s. s[a:b] will give us a string starting with s[a] and ending with s[b-1].
>>> s[1:4]
'ana'
None of these are assignable.
>>> print(s)
>>> s[0] = 'J'
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: object does not support item assignment
>>> s[1:3] = "up"
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: object does not support slice assignment
>>> print(s)
Outputs (assuming the errors were suppressed):
Xanadu Xanadu
Another feature of slices is that if the beginning or end is left empty, it will default to the first or last index, depending on context:
>>> s[2:]
'nadu'
>>> s[:3]
'Xan'
>>> s[:]
'Xanadu'
You can also use negative numbers in slices:
>>> print(s[-2:])
'du'
To understand slices, it's easiest not to count the elements themselves. It is a bit like counting not on your fingers, but in the spaces between them. The list is indexed like this:
Element: 1 2 3 4 Index: 0 1 2 3 4 -4 -3 -2 -1
So, when we ask for the [1:3] slice, that means we start at index 1, and end at index 2, and take everything in between them. If you are used to indexes in C or Java, this can be a bit disconcerting until you get used to it.
String constants
String constants can be found in the standard string module. An example is string.digits, which equals to '0123456789'.
Links:
- Python documentation of "string" module -- python.org
String methods
There are a number of methods or built-in string functions:
- capitalize
- center
- count
- decode
- encode
- endswith
- expandtabs
- find
- index
- isalnum
- isalpha
- isdigit
- islower
- isspace
- istitle
- isupper
- join
- ljust
- lower
- lstrip
- replace
- rfind
- rindex
- rjust
- rstrip
- split
- splitlines
- startswith
- strip
- swapcase
- title
- translate
- upper
- zfill
Only emphasized items will be covered.
is*
isalnum(), isalpha(), isdigit(), islower(), isupper(), isspace(), and istitle() fit into this category.
The length of the string object being compared must be at least 1, or the is* methods will return False. In other words, a string object of len(string) == 0, is considered "empty", or False.
- isalnum returns True if the string is entirely composed of alphabetic and/or numeric characters (i.e. no punctuation).
- isalpha and isdigit work similarly for alphabetic characters or numeric characters only.
- isspace returns True if the string is composed entirely of whitespace.
- islower, isupper, and istitle return True if the string is in lowercase, uppercase, or titlecase respectively. Uncased characters are "allowed", such as digits, but there must be at least one cased character in the string object in order to return True. Titlecase means the first cased character of each word is uppercase, and any immediately following cased characters are lowercase. Curiously, 'Y2K'.istitle() returns True. That is because uppercase characters can only follow uncased characters. Likewise, lowercase characters can only follow uppercase or lowercase characters. Hint: whitespace is uncased.
Example:
>>> '2YK'.istitle()
False
>>> 'Y2K'.istitle()
True
>>> '2Y K'.istitle()
True
Title, Upper, Lower, Swapcase, Capitalize
Returns the string converted to title case, upper case, lower case, inverts case, or capitalizes, respectively.
The title method capitalizes the first letter of each word in the string (and makes the rest lower case). Words are identified as substrings of alphabetic characters that are separated by non-alphabetic characters, such as digits, or whitespace. This can lead to some unexpected behavior. For example, the string "x1x" will be converted to "X1X" instead of "X1x".
The swapcase method makes all uppercase letters lowercase and vice versa.
The capitalize method is like title except that it considers the entire string to be a word. (i.e. it makes the first character upper case and the rest lower case)
Example:
s = 'Hello, wOrLD'
print(s) # 'Hello, wOrLD'
print(s.title()) # 'Hello, World'
print(s.swapcase()) # 'hELLO, WoRld'
print(s.upper()) # 'HELLO, WORLD'
print(s.lower()) # 'hello, world'
print(s.capitalize())# 'Hello, world'
Keywords: to lower case, to upper case, lcase, ucase, downcase, upcase.
count
Returns the number of the specified substrings in the string. i.e.
>>> s = 'Hello, world'
>>> s.count('o') # print the number of 'o's in 'Hello, World' (2)
2
Hint: .count() is case-sensitive, so this example will only count the number of lowercase letter 'o's. For example, if you ran:
>>> s = 'HELLO, WORLD'
>>> s.count('o') # print the number of lowercase 'o's in 'HELLO, WORLD' (0)
0
strip, rstrip, lstrip
Returns a copy of the string with the leading (lstrip) and trailing (rstrip) whitespace removed. strip removes both.
>>> s = '\t Hello, world\n\t '
>>> print(s)
Hello, world
>>> print(s.strip())
Hello, world
>>> print(s.lstrip())
Hello, world
# ends here
>>> print(s.rstrip())
Hello, world
Note the leading and trailing tabs and newlines.
Strip methods can also be used to remove other types of characters.
import string
s = 'www.wikibooks.org'
print(s)
print(s.strip('w')) # Removes all w's from outside
print(s.strip(string.lowercase)) # Removes all lowercase letters from outside
print(s.strip(string.printable)) # Removes all printable characters
Outputs:
www.wikibooks.org .wikibooks.org .wikibooks.
Note that string.lowercase and string.printable require an import string statement
ljust, rjust, center
left, right or center justifies a string into a given field size (the rest is padded with spaces).
>>> s = 'foo'
>>> s
'foo'
>>> s.ljust(7)
'foo '
>>> s.rjust(7)
' foo'
>>> s.center(7)
' foo '
join
Joins together the given sequence with the string as separator:
>>> seq = ['1', '2', '3', '4', '5']
>>> ' '.join(seq)
'1 2 3 4 5'
>>> '+'.join(seq)
'1+2+3+4+5'
map may be helpful here: (it converts numbers in seq into strings)
>>> seq = [1,2,3,4,5]
>>> ' '.join(map(str, seq))
'1 2 3 4 5'
now arbitrary objects may be in seq instead of just strings.
find, index, rfind, rindex
The find and index methods return the index of the first found occurrence of the given subsequence. If it is not found, find returns -1 but index raises a ValueError. rfind and rindex are the same as find and index except that they search through the string from right to left (i.e. they find the last occurrence)
>>> s = 'Hello, world'
>>> s.find('l')
2
>>> s[s.index('l'):]
'llo, world'
>>> s.rfind('l')
10
>>> s[:s.rindex('l')]
'Hello, wor'
>>> s[s.index('l'):s.rindex('l')]
'llo, wor'
Because Python strings accept negative subscripts, index is probably better used in situations like the one shown because using find instead would yield an unintended value.
replace
Replace works just like it sounds. It returns a copy of the string with all occurrences of the first parameter replaced with the second parameter.
>>> 'Hello, world'.replace('o', 'X')
'HellX, wXrld'
Or, using variable assignment:
string = 'Hello, world'
newString = string.replace('o', 'X')
print(string)
print(newString)
Outputs:
Hello, world HellX, wXrld
Notice, the original variable (string
) remains unchanged after the call to replace
.
expandtabs
Replaces tabs with the appropriate number of spaces (default number of spaces per tab = 8; this can be changed by passing the tab size as an argument).
s = 'abcdefg\tabc\ta'
print(s)
print(len(s))
t = s.expandtabs()
print(t)
print(len(t))
Outputs:
abcdefg abc a 13 abcdefg abc a 17
Notice how (although these both look the same) the second string (t) has a different length because each tab is represented by spaces not tab characters.
To use a tab size of 4 instead of 8:
v = s.expandtabs(4)
print(v)
print(len(v))
Outputs:
abcdefg abc a 13
Please note each tab is not always counted as eight spaces. Rather a tab "pushes" the count to the next multiple of eight. For example:
s = '\t\t'
print(s.expandtabs().replace(' ', '*'))
print(len(s.expandtabs()))
Output:
**************** 16
s = 'abc\tabc\tabc'
print(s.expandtabs().replace(' ', '*'))
print(len(s.expandtabs()))
Outputs:
abc*****abc*****abc 19
split, splitlines
The split method returns a list of the words in the string. It can take a separator argument to use instead of whitespace.
>>> s = 'Hello, world'
>>> s.split()
['Hello,', 'world']
>>> s.split('l')
['He', '', 'o, wor', 'd']
Note that in neither case is the separator included in the split strings, but empty strings are allowed.
The splitlines method breaks a multiline string into many single line strings. It is analogous to split('\n') (but accepts '\r' and '\r\n' as delimiters as well) except that if the string ends in a newline character, splitlines ignores that final character (see example).
>>> s = """
... One line
... Two lines
... Red lines
... Blue lines
... Green lines
... """
>>> s.split('\n')
['', 'One line', 'Two lines', 'Red lines', 'Blue lines', 'Green lines', '']
>>> s.splitlines()
['', 'One line', 'Two lines', 'Red lines', 'Blue lines', 'Green lines']
The method split also accepts multi-character string literals:
txt = 'May the force be with you'
spl = txt.split('the')
print(spl)
# ['May ', ' force be with you']
Unicode
In Python 3.x, all strings (the type str) contain Unicode per default.
In Python 2.x, there is a dedicated unicode type in addition to the str type: u = u"Hello"; type(u) is unicode.
The topic name in the internal help is UNICODE.
Examples for Python 3.x:
- v = "Hello Günther"
- Uses a Unicode code point directly in the source code; that has to be in UTF-8 encoding.
- v = "Hello G\xfcnther"
- Specifies 8-bit Unicode code point using \xfc.
- v = "Hello G\u00fcnther"
- Specifies 16-bit Unicode code point using \u00fc.
- v = "Hello G\U000000fcnther"
- Specifies 32-bit Unicode code point using \U000000fc, the U being capitalized.
- v = "Hello G\N{LATIN SMALL LETTER U WITH DIAERESIS}nther"
- Specifies a Unicode code point using \N followed by the unicode point name.
- v = "Hello G\N{latin small letter u with diaeresis}nther"
- The code point name can be in lowercase.
- n = unicodedata.name(chr(252))
- Obtains Unicode code point name given a Unicode character, here of ü.
- v = "Hello G" + chr(252) + "nther"
- chr() accepts Unicode code points and returns a string having one Unicode character.
- c = ord("ü")
- Yields the code point number.
- b = "Hello Günther".encode("UTF-8")
- Creates a byte sequence (bytes) out of a Unicode string.
- b = "Hello Günther".encode("UTF-8"); u = b.decode("UTF-8")
- Decodes bytes into a Unicode string via decode() method.
- v = b"Hello " + "G\u00fcnther"
- Throws TypeError: can't concat bytes to str.
- v = b"Hello".decode("ASCII") + "G\u00fcnther"
- Now it works.
- f = open("File.txt", encoding="UTF-8"); lines = f.readlines(); f.close()
- Opens a file for reading with a specific encoding and reads from it. If no encoding is specified, the one of locale.getpreferredencoding() is used.
- f = open("File.txt", "w", encoding="UTF-8"); f.write("Hello G\u00fcnther"); f.close()
- Writes to a file in a specified encoding.
- f = open("File.txt", encoding="UTF-8-sig"); lines = f.readlines(); f.close()
- The -sig encoding means that any leading byte order mark (BOM) is automatically stripped.
- f = tokenize.open("File.txt"); lines = f.readlines(); f.close()
- Automatically detects encoding based on an encoding marker present in the file, such as BOM, stripping the marker.
- f = open("File.txt", "w", encoding="UTF-8-sig"); f.write("Hello G\u00fcnther"); f.close()
- Writes to a file in UTF-8, writing BOM at the beginning.
Examples for Python 2.x:
- v = u"Hello G\u00fcnther"
- Specifies 16-bit Unicode code point using \u00fc.
- v = u"Hello G\U000000fcnther"
- Specifies 32-bit Unicode code point using \U000000fc, the U being capitalized.
- v = u"Hello G\N{LATIN SMALL LETTER U WITH DIAERESIS}nther"
- Specifies a Unicode code point using \N followed by the unicode point name.
- v = u"Hello G\N{latin small letter u with diaeresis}nther"
- The code point name can be in lowercase.
- unicodedata.name(unichr(252))
- Obtains Unicode code point name given a Unicode character, here of ü.
- v = "Hello G" + unichr(252) + "nther"
- chr() accepts Unicode code points and returns a string having one Unicode character.
- c = ord(u"ü")
- Yields the code point number.
- b = u"Hello Günther".encode("UTF-8")
- Creates a byte sequence (str) out of a Unicode string. type(b) is str.
- b = u"Hello Günther".encode("UTF-8"); u = b.decode("UTF-8")
- Decodes bytes (type str) into a Unicode string via decode() method.
- v = "Hello" + u"Hello G\u00fcnther"
- Concatenates str (bytes) and Unicode string without an error.
- f = codecs.open("File.txt", encoding="UTF-8"); lines = f.readlines(); f.close()
- Opens a file for reading with a specific encoding and reads from it. If no encoding is specified, the one of locale.getpreferredencoding() is used[VERIFY].
- f = codecs.open("File.txt", "w", encoding="UTF-8"); f.write(u"Hello G\u00fcnther"); f.close()
- Writes to a file in a specified encoding.
- Unlike the Python 3 variant, if told to write newline via \n, does not write operating system specific newline but rather literal \n; this makes a difference e.g. on Windows.
- To ensure text mode like operation one can write os.linesep.
- f = codecs.open("File.txt", encoding="UTF-8-sig"); lines = f.readlines(); f.close()
- The -sig encoding means that any leading byte order mark (BOM) is automatically stripped.
Links:
- Unicode HOWTO for Python 3, docs.python.org
- Unicode HOWTO for Python 2, docs.python.org
- Processing Text Files in Python 3, curiousefficiency.org
- PEP 263 – Defining Python Source Code Encodings, python.org
- unicodedata — Unicode Database in Python Library Reference, docs.python.org
- Get a list of all the encodings Python can encode to, stackoverflow.com
External links
- "String Methods" chapter -- python.org
- Python documentation of "string" module -- python.org
Lists
A list in Python is an ordered group of items (or elements). It is a very general structure, and list elements don't have to be of the same type: you can put numbers, letters, strings and nested lists all on the same list.
Overview
Lists in Python at a glance:
list1 = [] # A new empty list
list2 = [1, 2, 3, "cat"] # A new non-empty list with mixed item types
list1.append("cat") # Add a single member, at the end of the list
list1.extend(["dog", "mouse"]) # Add several members
list1.insert(0, "fly") # Insert at the beginning
list1[0:0] = ["cow", "doe"] # Add members at the beginning
doe = list1.pop(1) # Remove item at index
if "cat" in list1: # Membership test
list1.remove("cat") # Remove AKA delete
#list1.remove("elephant") - throws an error
for item in list1: # Iteration AKA for each item
print(item)
print("Item count:", len(list1))# Length AKA size AKA item count
list3 = [6, 7, 8, 9]
for i in range(0, len(list3)): # Read-write iteration AKA for each item
list3[i] += 1 # Item access AKA element access by index
last = list3[-1] # Last item
nextToLast = list3[-2] # Next-to-last item
isempty = len(list3) == 0 # Test for emptiness
set1 = set(["cat", "dog"]) # Initialize set from a list
list4 = list(set1) # Get a list from a set
list5 = list4[:] # A shallow list copy
list4equal5 = list4==list5 # True: same by value
list4refEqual5 = list4 is list5 # False: not same by reference
list6 = list4[:]
del list6[:] # Clear AKA empty AKA erase
list7 = [1, 2] + [2, 3, 4] # Concatenation
print(list1, list2, list3, list4, list5, list6, list7)
print(list4equal5, list4refEqual5)
print(list3[1:3], list3[1:], list3[:2]) # Slices
print(max(list3 ), min(list3 ), sum(list3)) # Aggregates
print([x for x in range(10)]) # List comprehension
print([x for x in range(10) if x % 2 == 1])
print([x for x in range(10) if x % 2 == 1 if x < 5])
print([x + 1 for x in range(10) if x % 2 == 1])
print([x + y for x in '123' for y in 'abc'])
List creation
There are two different ways to make a list in Python. The first is through assignment ("statically"), the second is using list comprehensions ("actively").
Plain creation
To make a static list of items, write them between square brackets. For example:
[ 1,2,3,"This is a list",'c',Donkey("kong") ]
Observations:
- The list contains items of different data types: integer, string, and Donkey class.
- Objects can be created 'on the fly' and added to lists. The last item is a new instance of Donkey class.
Creation of a new list whose members are constructed from non-literal expressions:
a = 2
b = 3
myList = [a+b, b+a, len(["a","b"])]
List comprehensions
Using list comprehension, you describe the process using which the list should be created. To do that, the list is broken into two pieces. The first is a picture of what each element will look like, and the second is what you do to get it.
For instance, let's say we have a list of words:
listOfWords = ["this","is","a","list","of","words"]
To take the first letter of each word and make a list out of it using list comprehension, we can do this:
>>> listOfWords = ["this","is","a","list","of","words"]
>>> items = [ word[0] for word in listOfWords ]
>>> print(items)
['t', 'i', 'a', 'l', 'o', 'w']
List comprehension supports more than one for statement. It will evaluate the items in all of the objects sequentially and will loop over the shorter objects if one object is longer than the rest.
>>> item = [x+y for x in 'cat' for y in 'pot']
>>> print(item)
['cp', 'co', 'ct', 'ap', 'ao', 'at', 'tp', 'to', 'tt']
List comprehension supports an if statement, to only include members into the list that fulfill a certain condition:
>>> print([x+y for x in 'cat' for y in 'pot'])
['cp', 'co', 'ct', 'ap', 'ao', 'at', 'tp', 'to', 'tt']
>>> print([x+y for x in 'cat' for y in 'pot' if x != 't' and y != 'o' ])
['cp', 'ct', 'ap', 'at']
>>> print([x+y for x in 'cat' for y in 'pot' if x != 't' or y != 'o' ])
['cp', 'co', 'ct', 'ap', 'ao', 'at', 'tp', 'tt']
In version 2.x, Python's list comprehension does not define a scope. Any variables that are bound in an evaluation remain bound to whatever they were last bound to when the evaluation was completed. In version 3.x Python's list comprehension uses local variables:
>>> print(x, y) #Input to python version 2
t t #Output using python 2
>>> print(x, y) #Input to python version 3
NameError: name 'x' is not defined #Python 3 returns an error because x and y were not leaked
This is exactly the same as if the comprehension had been expanded into an explicitly-nested group of one or more 'for' statements and 0 or more 'if' statements.
List creation shortcuts
You can initialize a list to a size, with an initial value for each element:
>>> zeros=[0]*5
>>> print zeros
[0, 0, 0, 0, 0]
This works for any data type:
>>> foos=['foo']*3
>>> print(foos)
['foo', 'foo', 'foo']
But there is a caveat. When building a new list by multiplying, Python copies each item by reference. This poses a problem for mutable items, for instance in a multidimensional array where each element is itself a list. You'd guess that the easy way to generate a two dimensional array would be:
listoflists=[ [0]*4 ] *5
and this works, but probably doesn't do what you expect:
>>> listoflists=[ [0]*4 ] *5
>>> print(listoflists)
[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
>>> listoflists[0][2]=1
>>> print(listoflists)
[[0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0]]
What's happening here is that Python is using the same reference to the inner list as the elements of the outer list. Another way of looking at this issue is to examine how Python sees the above definition:
>>> innerlist=[0]*4
>>> listoflists=[innerlist]*5
>>> print(listoflists)
[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
>>> innerlist[2]=1
>>> print(listoflists)
[[0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0]]
Assuming the above effect is not what you intend, one way around this issue is to use list comprehensions:
>>> listoflists=[[0]*4 for i in range(5)]
>>> print(listoflists)
[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
>>> listoflists[0][2]=1
>>> print(listoflists)
[[0, 0, 1, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
List size
To find the length of a list use the built in len() method.
>>> len([1,2,3])
3
>>> a = [1,2,3,4]
>>> len( a )
4
Combining lists
Lists can be combined in several ways. The easiest is just to 'add' them. For instance:
>>> [1,2] + [3,4]
[1, 2, 3, 4]
Another way to combine lists is with extend. If you need to combine lists inside of a lambda, extend is the way to go.
>>> a = [1,2,3]
>>> b = [4,5,6]
>>> a.extend(b)
>>> print(a)
[1, 2, 3, 4, 5, 6]
The other way to append a value to a list is to use append. For example:
>>> p=[1,2]
>>> p.append([3,4])
>>> p
[1, 2, [3, 4]]
>>> # or
>>> print(p)
[1, 2, [3, 4]]
However, [3,4] is an element of the list, and not part of the list. append always adds one element only to the end of a list. So if the intention was to concatenate two lists, always use extend.
Getting pieces of lists (slices)
Continuous slices
Like strings, lists can be indexed and sliced:
>>> list = [2, 4, "usurp", 9.0, "n"]
>>> list[2]
'usurp'
>>> list[3:]
[9.0, 'n']
Much like the slice of a string is a substring, the slice of a list is a list. However, lists differ from strings in that we can assign new values to the items in a list:
>>> list[1] = 17
>>> list
[2, 17, 'usurp', 9.0, 'n']
We can assign new values to slices of the lists, which don't even have to be the same length:
>>> list[1:4] = ["opportunistic", "elk"]
>>> list
[2, 'opportunistic', 'elk', 'n']
It's even possible to append items onto the start of lists by assigning to an empty slice:
>>> list[:0] = [3.14, 2.71]
>>> list
[3.14, 2.71, 2, 'opportunistic', 'elk', 'n']
Similarly, you can append to the end of the list by specifying an empty slice after the end:
>>> list[len(list):] = ['four', 'score']
>>> list
[3.14, 2.71, 2, 'opportunistic', 'elk', 'n', 'four', 'score']
You can also completely change the contents of a list:
>>> list[:] = ['new', 'list', 'contents']
>>> list
['new', 'list', 'contents']
The right-hand side of a list assignment statement can be any iterable type:
>>> list[:2] = ('element',('t',),[])
>>> list
['element', ('t',), [], 'contents']
With slicing you can create copy of list since slice returns a new list:
>>> original = [1, 'element', []]
>>> list_copy = original[:]
>>> list_copy
[1, 'element', []]
>>> list_copy.append('new element')
>>> list_copy
[1, 'element', [], 'new element']
>>> original
[1, 'element', []]
Note, however, that this is a shallow copy and contains references to elements from the original list, so be careful with mutable types:
>>> list_copy[2].append('something')
>>> original
[1, 'element', ['something']]
Non-Continuous slices
It is also possible to get non-continuous parts of an array. If one wanted to get every n-th occurrence of a list, one would use the :: operator. The syntax is a:b:n where a and b are the start and end of the slice to be operated upon.
>>> list = [i for i in range(10) ]
>>> list
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list[::2]
[0, 2, 4, 6, 8]
>>> list[1:7:2]
[1, 3, 5]
Comparing lists
Lists can be compared for equality.
>>> [1,2] == [1,2]
True
>>> [1,2] == [3,4]
False
Lists can be compared using a less-than operator, which uses lexicographical order:
>>> [1,2] < [2,1]
True
>>> [2,2] < [2,1]
False
>>> ["a","b"] < ["b","a"]
True
Sorting lists
Sorting at a glance:
list1 = [2, 3, 1, 'a', 'B']
list1.sort() # list1 gets modified, case sensitive
list2 = sorted(list1) # list1 is unmodified; since Python 2.4
list3 = sorted(list1, key=lambda x: x.lower()) # case insensitive ; will give error as not all elements of list are strings and .lower() is not applicable
list4 = sorted(list1, reverse=True) # Reverse sorting order: descending
print(list1, list2, list3, list4)
Sorting lists is easy with a sort method.
>>> list1 = [2, 3, 1, 'a', 'b']
>>> list1.sort()
>>> list1
[1, 2, 3, 'a', 'b']
Note that the list is sorted in place, and the sort() method returns None to emphasize this side effect.
If you use Python 2.4 or higher there are some more sort parameters:
- sort(cmp,key,reverse)
- cmp : function to determine the relative order between any two elements
- key : function to obtain the value against which to sort for any element.
- reverse : sort(reverse=True) or sort(reverse=False)
Python also includes a sorted() function.
>>> list1 = [5, 2, 3, 'q', 'p']
>>> sorted(list1)
[2, 3, 5, 'p', 'q']
>>> list1
[5, 2, 3, 'q', 'p']
Note that unlike the sort() method, sorted(list) does not sort the list in place, but instead returns the sorted list. The sorted() function, like the sort() method also accepts the reverse parameter.
Links:
- 2. Built-in Functions # sorted, docs.python.org
- Sorting HOW TO, docs.python.org
Iteration
Iteration over lists:
Read-only iteration over a list, AKA for each element of the list:
list1 = [1, 2, 3, 4]
for item in list1:
print(item)
Writable iteration over a list:
list1 = [1, 2, 3, 4]
for i in range(0, len(list1)):
list1[i]+=1 # Modify the item at an index as you see fit
print(list)
From a number to a number with a step:
for i in range(1, 13+1, 3): # For i=1 to 13 step 3
print(i)
for i in range(10, 5-1, -1): # For i=10 to 5 step -1
print(i)
For each element of a list satisfying a condition (filtering):
for item in list:
if not condition(item):
continue
print(item)
See also User:Yasondinalt/Loops#For_Loops.
Removing
Removing aka deleting an item at an index (see also #pop(i)):
list1 = [1, 2, 3, 4]
list1.pop() # Remove the last item
list1.pop(0) # Remove the first item , which is the item at index 0
print(list1)
list1 = [1, 2, 3, 4]
del list1[1] # Remove the 2nd element; an alternative to list.pop(1)
print(list1)
Removing an element by value:
list1 = ["a", "a", "b"]
list1.remove("a") # Removes only the 1st occurrence of "a"
print(list1)
Creating a new list by copying a filtered selection of items from the old list:
list1 = [1, 2, 3, 4]
newlist = [item for item in list1 if item > 2]
print(newlist)
This uses a list comprehension.
Update a list by retaining a filtered selection of items in it by using "[:]":
list1 = [1, 2, 3, 4]
sameList = list1
list1[:] = [item for item in list1 if item > 2]
print(sameList, sameList is list1)
For more complex condition a separate function can be used to define the criteria:
list1 = [1, 2, 3, 4]
def keepingCondition(item):
return item > 2
sameList = list1
list1[:] = [item for item in list1 if keepingCondition(item)]
print(sameList, sameList is list1)
Removing items while iterating a list usually leads to unintended outcomes unless you do it carefully by using an index:
list1 = [1, 2, 3, 4]
index = len(list1)
while index > 0:
index -= 1
if not list1[index] < 2:
list1.pop(index)
Links:
- Remove items from a list while iterating, stackoverflow.com
Aggregates
There are some built-in functions for arithmetic aggregates over lists. These include minimum, maximum, and sum:
list = [1, 2, 3, 4]
print(max(list), min(list), sum(list))
average = sum(list) / float(len(list)) # Provided the list is non-empty
# The float above ensures the division is a float one rather than integer one.
print(average)
The max and min functions also apply to lists of strings, returning maximum and minimum with respect to alphabetical order:
list = ["aa", "ab"]
print(max(list), min(list)) # Prints "ab aa"
Copying
Copying AKA cloning of lists:
Making a shallow copy:
list1= [1, 'element']
list2 = list1[:] # Copy using "[:]"
list2[0] = 2 # Only affects list2, not list1
print(list1[0]) # Displays 1
# By contrast
list1 = [1, 'element']
list2 = list1
list2[0] = 2 # Modifies the original list
print(list1[0]) # Displays 2
The above does not make a deep copy, which has the following consequence:
list1 = [1, [2, 3]] # Notice the second item being a nested list
list2 = list1[:] # A shallow copy
list2[1][0] = 4 # Modifies the 2nd item of list1 as well
print(list1[1][0]) # Displays 4 rather than 2
Making a deep copy:
import copy
list1 = [1, [2, 3]] # Notice the second item being a nested list
list2 = copy.deepcopy(list1) # A deep copy
list2[1][0] = 4 # Leaves the 2nd item of list1 unmodified
print list1[1][0] # Displays 2
See also #Continuous slices.
Links:
- 8.17. copy — Shallow and deep copy operations at docs.python.org
Clearing
Clearing a list:
del list1[:] # Clear a list
list1 = [] # Not really clear but rather assign to a new empty list
Clearing using a proper approach makes a difference when the list is passed as an argument:
def workingClear(ilist):
del ilist[:]
def brokenClear(ilist):
ilist = [] # Lets ilist point to a new list, losing the reference to the argument list
list1=[1, 2]; workingClear(list1); print(list1)
list1=[1, 2]; brokenClear(list1); print(list1)
Keywords: emptying a list, erasing a list, clear a list, empty a list, erase a list.
Removing duplicate items
Removing duplicate items from a list (keeping only unique items) can be achieved as follows.
If each item in the list is hashable, using list comprehension, which is fast:
list1 = [1, 4, 4, 5, 3, 2, 3, 2, 1]
seen = {}
list1[:] = [seen.setdefault(e, e) for e in list1 if e not in seen]
If each item in the list is hashable, using index iteration, much slower:
list1 = [1, 4, 4, 5, 3, 2, 3, 2, 1]
seen = set()
for i in range(len(list1) - 1, -1, -1):
if list1[i] in seen:
list1.pop(i)
seen.add(list1[i])
If some items are not hashable, the set of visited items can be kept in a list:
list1 = [1, 4, 4, ["a", "b"], 5, ["a", "b"], 3, 2, 3, 2, 1]
seen = []
for i in range(len(list1) - 1, -1, -1):
if list1[i] in seen:
list1.pop(i)
seen.append(list1[i])
If each item in the list is hashable and preserving element order does not matter:
list1 = [1, 4, 4, 5, 3, 2, 3, 2, 1]
list1[:] = list(set(list1)) # Modify list1
list2 = list(set(list1))
In the above examples where index iteration is used, scanning happens from the end to the beginning. When these are rewritten to scan from the beginning to the end, the result seems hugely slower.
Links:
- How do you remove duplicates from a list? at python.org Programming FAQ
- Remove duplicates from a sequence (Python recipe) at activestate.com
- Removing duplicates in lists at stackoverflow.com
List methods
append(x)
Add item x onto the end of the list.
>>> list = [1, 2, 3]
>>> list.append(4)
>>> list
[1, 2, 3, 4]
See pop(i)
pop(i)
Remove the item in the list at the index i and return it. If i is not given, remove the last item in the list and return it.
>>> list = [1, 2, 3, 4]
>>> a = list.pop(0)
>>> list
[2, 3, 4]
>>> a
1
>>> b = list.pop()
>>>list
[2, 3]
>>> b
4
Operators
+
To concatenate two lists.
*
To create a new list by concatenating the given list the given number of times. i.e. list1 * 0 == []; list1 * 3 == list1 + list1 + list1;
in
The operator 'in' is used for two purposes; either to iterate over every item in a list in a for loop, or to check if a value is in a list returning true or false.
>>> list = [1, 2, 3, 4]
>>> if 3 in list:
>>> ....
>>> l = [0, 1, 2, 3, 4]
>>> 3 in l
True
>>> 18 in l
False
>>>for x in l:
>>> print(x)
0
1
2
3
4
Difference
To get the difference between two lists, just iterate:
a = [0, 1, 2, 3, 4, 4]
b = [1, 2, 3, 4, 4, 5]
print([item for item in a if item not in b])
# [0]
Intersection
To get the intersection between two lists (by preserving its elements order, and their doubles), apply the difference with the difference:
a = [0, 1, 2, 3, 4, 4]
b = [1, 2, 3, 4, 4, 5]
dif = [item for item in a if item not in b]
print([item for item in a if item not in dif])
# [1, 2, 3, 4, 4]
# Note that using the above on:
a = [1, 1]; b = [1]
# will result in [1, 1]
# Similarly
a = [1]; b = [1, 1]
# will result in [1]
Exercises
- Use a list comprehension to construct the list ['ab', 'ac', 'ad', 'bb', 'bc', 'bd'].
- Use a slice on the above list to construct the list ['ab', 'ad', 'bc'].
- Use a list comprehension to construct the list ['1a', '2a', '3a', '4a'].
- Simultaneously remove the element '2a' from the above list and print it.
- Copy the above list and add '2a' back into the list such that the original is still missing it.
- Use a list comprehension to construct the list ['abe', 'abf', 'ace', 'acf', 'ade', 'adf', 'bbe', 'bbf', 'bce', 'bcf', 'bde', 'bdf']
Solutions
Question 1 :
List1 = [a + b for a in 'ab' for b in 'bcd'] print(List1) >>> ['ab', 'ac', 'ad', 'bb', 'bc', 'bd']
Question 2 :
List2 = List1[::2] print(List2) >>> ['ab', 'ad', 'bc']
Question 3 :
List3 = [a + b for a in '1234' for b in 'a'] print(List3) >>> ['1a', '2a', '3a', '4a']
Question 4 :
print(List3.pop(List3.index('3a'))) print(List3) >>> 3a >>> ['1a', '2a', '4a']
Question 5 :
List4 = List3[:] List4.insert(2, '3a') print(List4) >>> ['1a', '2a', '3a', '4a']
Question 6 :
List5 = [a + b + c for a in 'ab' for b in 'bcd' for c in 'ef'] print(List5) >>> ['abe', 'abf', 'ace', 'acf', 'ade', 'adf', 'bbe', 'bbf', 'bce', 'bcf', 'bde', 'bdf']
External links
- Python documentation, chapter "Sequence Types" -- python.org
- Python Tutorial, chapter "Lists" -- python.org
}}
Dictionaries
A dictionary in Python is a collection of unordered values accessed by key rather than by index. The keys have to be hashable: integers, floating point numbers, strings, tuples, and, frozensets are hashable. Lists, dictionaries, and sets other than frozensets are not hashable. Dictionaries were available as early as in Python 1.4.
Overview
Dictionaries in Python at a glance:
dict1 = {} # Create an empty dictionary
dict2 = dict() # Create an empty dictionary 2
dict2 = {"r": 34, "i": 56} # Initialize to non-empty value
dict3 = dict([("r", 34), ("i", 56)]) # Init from a list of tuples
dict4 = dict(r=34, i=56) # Initialize to non-empty value 3
dict1["temperature"] = 32 # Assign value to a key
if "temperature" in dict1: # Membership test of a key AKA key exists
del dict1["temperature"] # Delete AKA remove
equalbyvalue = dict2 == dict3
itemcount2 = len(dict2) # Length AKA size AKA item count
isempty2 = len(dict2) == 0 # Emptiness test
for key in dict2: # Iterate via keys
print (key, dict2[key]) # Print key and the associated value
dict2[key] += 10 # Modify-access to the key-value pair
for key in sorted(dict2): # Iterate via keys in sorted order of the keys
print (key, dict2[key]) # Print key and the associated value
for value in dict2.values(): # Iterate via values
print (value)
for key, value in dict2.items(): # Iterate via pairs
print (key, value)
dict5 = {} # {x: dict2[x] + 1 for x in dict2 } # Dictionary comprehension in Python 2.7 or later
dict6 = dict2.copy() # A shallow copy
dict6.update({"i": 60, "j": 30}) # Add or overwrite; a bit like list's extend
dict7 = dict2.copy()
dict7.clear() # Clear AKA empty AKA erase
sixty = dict6.pop("i") # Remove key i, returning its value
print (dict1, dict2, dict3, dict4, dict5, dict6, dict7, equalbyvalue, itemcount2, sixty)
Dictionary notation
Dictionaries may be created directly or converted from sequences. Dictionaries are enclosed in curly braces, {}
>>> d = {'city':'Paris', 'age':38, (102,1650,1601):'A matrix coordinate'}
>>> seq = [('city','Paris'), ('age', 38), ((102,1650,1601),'A matrix coordinate')]
>>> d
{'city': 'Paris', 'age': 38, (102, 1650, 1601): 'A matrix coordinate'}
>>> dict(seq)
{'city': 'Paris', 'age': 38, (102, 1650, 1601): 'A matrix coordinate'}
>>> d == dict(seq)
True
Also, dictionaries can be easily created by zipping two sequences.
>>> seq1 = ('a','b','c','d')
>>> seq2 = [1,2,3,4]
>>> d = dict(zip(seq1,seq2))
>>> d
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
Operations on Dictionaries
The operations on dictionaries are somewhat unique. Slicing is not supported, since the items have no intrinsic order.
>>> d = {'a':1,'b':2, 'cat':'Fluffers'}
>>> d.keys()
['a', 'b', 'cat']
>>> d.values()
[1, 2, 'Fluffers']
>>> d['a']
1
>>> d['cat'] = 'Mr. Whiskers'
>>> d['cat']
'Mr. Whiskers'
>>> 'cat' in d
True
>>> 'dog' in d
False
Combining two Dictionaries
You can combine two dictionaries by using the update method of the primary dictionary. Note that the update method will merge existing elements if they conflict.
>>> d = {'apples': 1, 'oranges': 3, 'pears': 2}
>>> ud = {'pears': 4, 'grapes': 5, 'lemons': 6}
>>> d.update(ud)
>>> d
{'grapes': 5, 'pears': 4, 'lemons': 6, 'apples': 1, 'oranges': 3}
>>>
Deleting from dictionary
del dictionaryName[membername]
Exercises
Write a program that:
- Asks the user for a string, then creates the following dictionary. The values are the letters in the string, with the corresponding key being the place in the string. https://docs.python.org/2/tutorial/datastructures.html#looping-techniques
- Replaces the entry whose key is the integer 3, with the value "Pie".
- Asks the user for a string of digits, then prints out the values corresponding to those digits.
External links
- 5.5. Dictionaries in Tutorial, docs.python.org
- 5.8. Mapping Types in Library Doc, docs.python.org
Sets
Starting with version 2.3, Python comes with an implementation of the mathematical set. Initially this implementation had to be imported from the standard module set, but with Python 2.6 the types set and frozenset became built-in types. A set is an unordered collection of objects, unlike sequence objects such as lists and tuples, in which each element is indexed. Sets cannot have duplicate members - a given object appears in a set 0 or 1 times. All members of a set have to be hashable, just like dictionary keys. Integers, floating point numbers, tuples, and strings are hashable; dictionaries, lists, and other sets (except frozensets) are not.
Overview
Sets in Python at a glance:
set1 = set() # A new empty set
set1.add("cat") # Add a single member
set1.update(["dog", "mouse"]) # Add several members, like list's extend
set1 |= set(["doe", "horse"]) # Add several members 2, like list's extend
if "cat" in set1: # Membership test
set1.remove("cat")
#set1.remove("elephant") - throws an error
set1.discard("elephant") # No error thrown
print(set1)
for item in set1: # Iteration AKA for each element
print(item)
print("Item count:", len(set1))# Length AKA size AKA item count
#1stitem = set1[0] # Error: no indexing for sets
isempty = len(set1) == 0 # Test for emptiness
set1 = {"cat", "dog"} # Initialize set using braces; since Python 2.7
#set1 = {} # No way; this is a dict
set1 = set(["cat", "dog"]) # Initialize set from a list
set2 = set(["dog", "mouse"])
set3 = set1 & set2 # Intersection
set4 = set1 | set2 # Union
set5 = set1 - set3 # Set difference
set6 = set1 ^ set2 # Symmetric difference
issubset = set1 <= set2 # Subset test
issuperset = set1 >= set2 # Superset test
set7 = set1.copy() # A shallow copy
set7.remove("cat")
print(set7.pop()) # Remove an arbitrary element
set8 = set1.copy()
set8.clear() # Clear AKA empty AKA erase
set9 = {x for x in range(10) if x % 2} # Set comprehension; since Python 2.7
print(set1, set2, set3, set4, set5, set6, set7, set8, set9, issubset, issuperset)
Constructing Sets
One way to construct sets is by passing any sequential object to the "set" constructor.
>>> set([0, 1, 2, 3])
set([0, 1, 2, 3])
>>> set("obtuse")
set(['b', 'e', 'o', 's', 'u', 't'])
We can also add elements to sets one by one, using the "add" function.
>>> s = set([12, 26, 54])
>>> s.add(32)
>>> s
set([32, 26, 12, 54])
Note that since a set does not contain duplicate elements, if we add one of the members of s to s again, the add function will have no effect. This same behavior occurs in the "update" function, which adds a group of elements to a set.
>>> s.update([26, 12, 9, 14])
>>> s
set([32, 9, 12, 14, 54, 26])
Note that you can give any type of sequential structure, or even another set, to the update function, regardless of what structure was used to initialize the set.
The set function also provides a copy constructor. However, remember that the copy constructor will copy the set, but not the individual elements.
>>> s2 = s.copy()
>>> s2
set([32, 9, 12, 14, 54, 26])
Membership Testing
We can check if an object is in the set using the same "in" operator as with sequential data types.
>>> 32 in s
True
>>> 6 in s
False
>>> 6 not in s
True
We can also test the membership of entire sets. Given two sets and , we check if is a subset or a superset of .
>>> s.issubset(set([32, 8, 9, 12, 14, -4, 54, 26, 19]))
True
>>> s.issuperset(set([9, 12]))
True
Note that "issubset" and "issuperset" can also accept sequential data types as arguments
>>> s.issuperset([32, 9])
True
Note that the <= and >= operators also express the issubset and issuperset functions respectively.
>>> set([4, 5, 7]) <= set([4, 5, 7, 9])
True
>>> set([9, 12, 15]) >= set([9, 12])
True
Like lists, tuples, and string, we can use the "len" function to find the number of items in a set.
Removing Items
There are three functions which remove individual items from a set, called pop, remove, and discard. The first, pop, simply removes an item from the set. Note that there is no defined behavior as to which element it chooses to remove.
>>> s = set([1,2,3,4,5,6])
>>> s.pop()
1
>>> s
set([2,3,4,5,6])
We also have the "remove" function to remove a specified element.
>>> s.remove(3)
>>> s
set([2,4,5,6])
However, removing a item which isn't in the set causes an error.
>>> s.remove(9)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
KeyError: 9
If you wish to avoid this error, use "discard." It has the same functionality as remove, but will simply do nothing if the element isn't in the set
We also have another operation for removing elements from a set, clear, which simply removes all elements from the set.
>>> s.clear()
>>> s
set([])
Iteration Over Sets
We can also have a loop move over each of the items in a set. However, since sets are unordered, it is undefined which order the iteration will follow.
>>> s = set("blerg")
>>> for n in s:
... print(n, "", end="")
...
r b e l g
Set Operations
Python allows us to perform all the standard mathematical set operations, using members of set. Note that each of these set operations has several forms. One of these forms, s1.function(s2) will return another set which is created by "function" applied to and . The other form, s1.function_update(s2), will change to be the set created by "function" of and . Finally, some functions have equivalent special operators. For example, s1 & s2 is equivalent to s1.intersection(s2)
Intersection
Any element which is in both and will appear in their intersection.
>>> s1 = set([4, 6, 9])
>>> s2 = set([1, 6, 8])
>>> s1.intersection(s2)
set([6])
>>> s1 & s2
set([6])
>>> s1.intersection_update(s2)
>>> s1
set([6])
Union
The union is the merger of two sets. Any element in or will appear in their union.
>>> s1 = set([4, 6, 9])
>>> s2 = set([1, 6, 8])
>>> s1.union(s2)
set([1, 4, 6, 8, 9])
>>> s1 | s2
set([1, 4, 6, 8, 9])
Note that union's update function is simply "update" above.
Symmetric Difference
The symmetric difference of two sets is the set of elements which are in one of either set, but not in both (also called exclusive-or in logic).
>>> s1 = set([4, 6, 9])
>>> s2 = set([1, 6, 8])
>>> s1.symmetric_difference(s2)
set([8, 1, 4, 9])
>>> s1 ^ s2
set([8, 1, 4, 9])
>>> s1.symmetric_difference_update(s2)
>>> s1
set([8, 1, 4, 9])
Set Difference
Python can also find the set difference of and , which is the elements that are in but not in .
>>> s1 = set([4, 6, 9])
>>> s2 = set([1, 6, 8])
>>> s1.difference(s2)
set([9, 4])
>>> s1 - s2
set([9, 4])
>>> s1.difference_update(s2)
>>> s1
set([9, 4])
Multiple sets
Starting with Python 2.6, "union", "intersection", and "difference" can work with multiple input. For example, using "set.intersection()":
>>> s1 = set([3, 6, 7, 9])
>>> s2 = set([6, 7, 9, 10])
>>> s3 = set([7, 9, 10, 11])
>>> set.intersection(s1, s2, s3)
set([9, 7])
frozenset
A frozenset is basically the same as a set, except that it is immutable - once it is created, its members cannot be changed. Since they are immutable, they are also hashable, which means that frozensets can be used as members in other sets and as dictionary keys. frozensets have the same functions as normal sets, except none of the functions that change the contents (update, remove, pop, etc.) are available.
>>> fs = frozenset([2, 3, 4])
>>> s1 = set([fs, 4, 5, 6])
>>> s1
set([4, frozenset([2, 3, 4]), 6, 5])
>>> fs.intersection(s1)
frozenset([4])
>>> fs.add(6)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'frozenset' object has no attribute 'add'
Exercises
- Create the set {'cat', 1, 2, 3}, call it s.
- Create the set {'c', 'a', 't', '1', '2', '3'}.
- Create the frozen set {'cat', 1, 2, 3}, call it fs.
- Create a set containing the frozenset fs, it should look like {frozenset({'cat', 2, 3, 1})}.
Reference
- Python Tutorial, section "Data Structures", subsection "Sets" -- python.org
- Python Library Reference on Set Types -- python.org
- PEP 218 -- Adding a Built-In Set Object Type, python.org, a nice concise overview of the set type
Operators
Basics
Python math works as expected:
>>> x = 2
>>> y = 3
>>> z = 5
>>> x * y
6
>>> x + y
5
>>> y - x
1
>>> x * y + z
11
>>> (x + y) * z
25
>>> 3.0 / 2.0 # True division
1.5
>>> 3 // 2 # Floor division
1
>>> 2 ** 3 # Exponentiation
8
Note that Python adheres to the PEMDAS order of operations.
Powers
There is a built in exponentiation operator **, which can take either integers, floating point or complex numbers. This occupies its proper place in the order of operations.
>>> 2**8
256
Floor Division and True Division
In Python 3.x, slash operator ("/") does true division for all types including integers, and therefore, e.g. 3/2==1.5
[1][2]. The result is of a floating-point type even if both inputs are integers: 4 / 2 yields 2.0.
In Python 3.x and latest 2.x, floor division for both integer arguments and floating-point arguments is achieved by using the double slash ("//") operator. For negative results, this is unlike the integer division in the C language since -3 // 2 == -2 in Python while -3 / 2 == -1 in C: C rounds the negative result toward zero while Python toward negative infinity.
Beware that due to the limitations of floating point arithmetic, rounding errors can cause unexpected results. For example:
>>> print(0.6/0.2) 3.0 >>> print(0.6//0.2) 2.0
For Python 2.x, dividing two integers or longs using the slash operator ("/") uses floor division (applying the floor function after division) and results in an integer or long. Thus, 5 / 2 == 2 and -3 / 2 == -2. Using "/" to do division this way is deprecated; if you want floor division, use "//" (available in Python 2.2 and later). Dividing by or into a floating point number will cause Python to use true division. Thus, to ensure true division in Python 2.x: x=3; y=2; float(x)/y == 1.5
.
Links:
- 6.7. Binary arithmetic operations, in The Python Language Reference, docs.python.org
- Python-style integer division & modulus in C, stackoverflow.com
- Integer division rounding with negatives in C++, stackoverflow.com
- In Python, what is a good way to round towards zero in integer division?, stackoverflow.com
- Why Python's Integer Division Floors, python-history.blogspot.com
Modulus
The modulus (remainder of the division of the two operands, rather than the quotient) can be found using the % operator, or by the divmod builtin function. The divmod function returns a tuple containing the quotient and remainder.
>>> 10 % 7
3
>>> -10 % 7
4
Note that -10 % 7 is equal to +4 while in the C language it is equal to -3. That is because Python floors towards negative infinity not zero. As a result, remainders add towards positive infinity. Consequently, since -10 / 7 = -1.4286 becomes floored to -2.0 the remainder becomes x such that -14 + x = -10.
Links:
- 6.7. Binary arithmetic operations, in The Python Language Reference, docs.python.org
Negation
Unlike some other languages, variables can be negated directly:
>>> x = 5
>>> -x
-5
Comparison
Operation | Means |
---|---|
< | Less than |
> | Greater than |
<= | Less than or equal to |
>= | Greater than or equal to |
== | Equal to |
!= | Not equal to |
Numbers, strings and other types can be compared for equality/inequality and ordering:
>>> 2 == 3
False
>>> 3 == 3
True
>>> 3 == '3'
False
>>> 2 < 3
True
>>> "a" < "aa"
True
Identity
The operators is
and is not
test for object identity and stand in contrast to == (equals): x is y
is true if and only if x and y are references to the same object in memory. x is not y
yields the inverse truth value.
Note that an identity test is more stringent than an equality test since two distinct objects may have the same value.
>>> [1,2,3] == [1,2,3]
True
>>> [1,2,3] is [1,2,3]
False
For the built-in immutable data types (like int, str and tuple) Python uses caching mechanisms to improve performance, i.e., the interpreter may decide to reuse an existing immutable object instead of generating a new one with the same value. The details of object caching are subject to changes between different Python versions and are not guaranteed to be system-independent, so identity checks on immutable objects like 'hello' is 'hello'
, (1,2,3) is (1,2,3)
, 4 is 2**2
may give different results on different machines.
In some Python implementations, the following results are applicable:
print(8 is 8) # True
print("str" is "str") # True
print((1, 2) is (1, 2)) # False - whyever, it is immutable
print([1, 2] is [1, 2]) # False
print(id(8) == id(8)) # True
int1 = 8
print(int1 is 8) # True
oldid = id(int1)
int1 += 2
print(id(int1) == oldid)# False
Links:
- 3. Data model, python.org
- 2. Built-in Functions # id, python.org
- 5. Expressions # is, python.org
Augmented Assignment
There is shorthand for assigning the output of an operation to one of the inputs:
>>> x = 2
>>> x # 2
2
>>> x *= 3
>>> x # 2 * 3
6
>>> x += 4
>>> x # 2 * 3 + 4
10
>>> x /= 5
>>> x # (2 * 3 + 4) / 5
2
>>> x **= 2
>>> x # ((2 * 3 + 4) / 5) ** 2
4
>>> x %= 3
>>> x # ((2 * 3 + 4) / 5) ** 2 % 3
1
>>> x = 'repeat this '
>>> x # repeat this
repeat this
>>> x *= 3 # fill with x repeated three times
>>> x
repeat this repeat this repeat this
Logical Operators
Logical operators are operators that act on booleans.
or
The or operator returns true if any one of the booleans involved are true. If none of them are true (in other words, they are all false), the or operator returns false.
if a or b:
do_this
else:
do_this
and
The and operator only returns true if all of the booleans are true. If any one of them is false, the and operator returns false.
if a and b:
do_this
else:
do_this
not
The not operator only acts on one boolean and simply returns its opposite. So, true turns into false and false into true.
if not a:
do_this
else:
do_this
The order of operations here is: not first, and second, or third. In particular, "True or True and False or False" becomes "True or False or False" which is True.
Warning, logical operators can act on things other than booleans. For instance "1 and 6" will return 6. Specifically, "and" returns either the first value considered to be false, or the last value if all are considered true. "or" returns the first true value, or the last value if all are considered false. In Python the number zero and empty strings, lists, sets, etc. are considered false. You may use bool()
to check whether a thing is considered to be true or false in Python. For instance, bool(0.0)
and bool([])
both return False
.
Bitwise Operators
Python operators for bitwise arithmetic are like those in the C language. They include & (bitwise and), | (bitwise or), ^ (exclusive or AKA xor), << (shift left), >> (shift right), and ~ (complement). Augmented assignment operators (AKA compound assignment operators) for the bitwise operations include &=, |=, ^=, <<=, and >>=. Bitwise operators apply to integers, even negative ones and very large ones; for the shift operators, the second operand must be non-negative. In the Python internal help, this is covered under the topics of EXPRESSIONS and BITWISE.
Examples:
- 0b1101 & 0b111 == 0b101
- Note: 0b starts a binary literal, just like 0x starts a hexadecimal literal.
- 0b1 | 0b100 == 0b101
- 0b111 ^ 0b101 == 0b10
- 1 << 4 == 16
- 7 >> 1 == 3
- 1 << 100 == 0x10000000000000000000000000
- Large results are supported.
- 1 << -1
- An error: the 2nd operand must be non-negative.
- -2 & 15 == 14
- For bitwise operations, negative integers are treated as if represented using two's complement with an infinite sequence of leading ones. Thus -2 is as if 0x...FFFFFFFFFE, which ANDed with 15 (0xF) yields 0xE, which is 14.
- format(-2 % (1 << 32), "032b")
- Determines a string showing the last 32 bits of the implied two's complement representation of -2.
- ~-2 == 1
- The above note about treatment of negative integers applies. For the complement (~), this treatment yields ~x == -1 * (x + 1). This can be verified: min((~x == -1 * (x + 1) for x in range(-10 ** 6, 10 ** 6))) == True.
- ~1 == -2
- The formula ~x == -1 * (x + 1) mentioned above applies. In the bitwise complement interpretation, all the imaginary leading leading zeros of 1 are toggled to leading ones, which is then interpreted as two's complement representation of -2, which is as if 0x...FFFFFFFFFE.
- x = 0b11110101; x &= ~(0xF << 1); x == 0b11100001
- A common idiom clears the least significant bits 5 to 2 using complement, showing the usefulness of the infinite-leading-ones two's complement implied representation of negative numbers for bitwise operations. Works for arbitrarily large x.
- ~2 & 0xFFFFFFFF == 0xFFFFFFFD
- We can emulate bitwise complement on 32-bit integers by ANDing the complement with the maximum unsigned 32-bit integer, 0xFFFFFFFF. We can proceed similarly for 8-bit complement, 16-bit complement, etc., by ANDing the complement with 0xFF, 0xFFFF, etc.
- 2 ^ 0xFFFFFFFF == 0xFFFFFFFD
- Another way to emulate fixed-size bitwise complement is by XORing with all Fs for the size.
- v = 2.0 & 2
- Yields an error: no automatic conversion of floats to ints, and no operation on the underlying representation of the float.
- int.bit_length(0x8000) == 16
- Determines how many bits are needed to represent the integer. Therefore, min((int.bit_length(1 << x) == x + 1 for x in range(100000))) == True.
Examples of augmented assignment operators:
- a = 0b1101; a &= 0b111; a == 0b101
- a = 0b1; a |= 0b100; a == 0b101
- a = 0b111; a ^= 0b101; a == 0b10
- a = 1; a <<= 4; a == 16
- a = 7; a >>= 1; a == 3
Class definitions can overload the operators for the instances of the class; thus, for instance, sets overload the pipe (|) operator to mean set union: {1,2} | {3,4} == {1,2,3,4}. The names of the override methods are __and__ for &, __or__ for |, __xor__ for ^, __invert__ for ~, __lshift__ for <<, __rshift__ for >>, __iand__ for &=, __ior_ for |=, __ixor__ for ^=, __ilshift__ for <<=, and __irshift__ for >>=.
Examples of use of bitwise operations include calculation of CRC and MD5. Admittedly, these would usually be implemented in C rather than Python for maximum speed; indeed, Python has libraries for these written in C. Nonetheless, implementations in Python are possible and are shown in the links to Rosetta Code below.
Links:
- BitwiseOperators, wiki.python.org
- BitManipulation, wiki.python.org
- Bitwise Operations on Integer Types in Library Reference, docs.python.org
- 2.5. Operators in The Python Language Reference, docs.python.org
- 6. Expressions in The Python Language Reference, docs.python.org
- 3. Data model in in The Python Language Reference, docs.python.org
- "Integers (int) [...] For the purpose of shift and mask operations, a binary representation is assumed, and negative numbers are represented in a variant of 2’s complement which gives the illusion of an infinite string of sign bits extending to the left."
- Bitwise operation, wikipedia.org
- Bitwise operations in C, wikipedia.org
- PEP 203 – Augmented Assignments, python.org
- Bitwise operation and usage, stackoverflow.com
- CRC-32 # Python, rosettacode.org
- MD5/Implementation # Python, rosettacode.org
- longobject.c at python/cpython, github.com - functions long_bitwise, long_rshift, long_lshift, and long_invert
References
External Links
- 3.1.1. Numbers in The Python Tutorial, docs.python.org
- 6. Expressions in The Python Language Reference, docs.python.org
Flow control
Python Programming/Flow control
Functions
Function Calls
A callable object is an object that can accept some arguments (also called parameters) and possibly return an object (often a tuple containing multiple objects).
A function is the simplest callable object in Python, but there are others, such as classes or certain class instances.
Defining Functions
A function is defined in Python by the following format:
def functionname(arg1, arg2, ...):
statement1
statement2
...
>>> def functionname(arg1,arg2):
... return arg1+arg2
...
>>> t = functionname(24,24) # Result: 48
If a function takes no arguments, it must still include the parentheses, but without anything in them:
def functionname():
statement1
statement2
...
The arguments in the function definition bind the arguments passed at function invocation (i.e. when the function is called), which are called actual parameters, to the names given when the function is defined, which are called formal parameters. The interior of the function has no knowledge of the names given to the actual parameters; the names of the actual parameters may not even be accessible (they could be inside another function).
A function can 'return' a value, for example:
def square(x):
return x*x
A function can define variables within the function body, which are considered 'local' to the function. The locals together with the arguments comprise all the variables within the scope of the function. Any names within the function are unbound when the function returns or reaches the end of the function body.
You can return multiple values as follows:
def first2items(list1):
return list1[0], list1[1]
a, b = first2items(["Hello", "world", "hi", "universe"])
print(a + " " + b)
Keywords: returning multiple values, multiple return values.
Declaring Arguments
When calling a function that takes some values for further processing, we need to send some values as Function Arguments. For example:
>>> def find_max(a,b):
if(a > b):
return str(a) + " is greater than " + str(b)
elif(b > a):
return str(b) + " is greater than " + str(a)
>>> find_max(30, 45) #Here (30, 45) are the arguments passing for finding max between this two numbers
The output will be: 45 is greater than 30
Default Argument Values
If any of the formal parameters in the function definition are declared with the format "arg = value," then you will have the option of not specifying a value for those arguments when calling the function. If you do not specify a value, then that parameter will have the default value given when the function executes.
>>> def display_message(message, truncate_after=4):
... print(message[:truncate_after])
...
>>> display_message("message")
mess
>>> display_message("message", 6)
messag
Links:
- 4.7.1. Default Argument Values, The Python Tutorial, docs.python.org
Variable-Length Argument Lists
Python allows you to declare two special arguments which allow you to create arbitrary-length argument lists. This means that each time you call the function, you can specify any number of arguments above a certain number.
def function(first,second,*remaining):
statement1
statement2
...
When calling the above function, you must provide value for each of the first two arguments. However, since the third parameter is marked with an asterisk, any actual parameters after the first two will be packed into a tuple and bound to "remaining."
>>> def print_tail(first,*tail):
... print(tail)
...
>>> print_tail(1, 5, 2, "omega")
(5, 2, 'omega')
If we declare a formal parameter prefixed with two asterisks, then it will be bound to a dictionary containing any keyword arguments in the actual parameters which do not correspond to any formal parameters. For example, consider the function:
def make_dictionary(max_length=10, **entries):
return dict([(key, entries[key]) for i, key in enumerate(entries.keys()) if i < max_length])
If we call this function with any keyword arguments other than max_length, they will be placed in the dictionary "entries." If we include the keyword argument of max_length, it will be bound to the formal parameter max_length, as usual.
>>> make_dictionary(max_length=2, key1=5, key2=7, key3=9)
{'key3': 9, 'key2': 7}
Links:
- 4.7.3. Arbitrary Argument Lists, The Python Tutorial, docs.python.org
By Value and by Reference
Objects passed as arguments to functions are passed by reference; they are not being copied around. Thus, passing a large list as an argument does not involve copying all its members to a new location in memory. Note that even integers are objects. However, the distinction of by value and by reference present in some other programming languages often serves to distinguish whether the passed arguments can be actually changed by the called function and whether the calling function can see the changes.
Passed objects of mutable types such as lists and dictionaries can be changed by the called function and the changes are visible to the calling function. Passed objects of immutable types such as integers and strings cannot be changed by the called function; the calling function can be certain that the called function will not change them. For mutability, see also Data Types chapter.
An example:
def appendItem(ilist, item):
ilist.append(item) # Modifies ilist in a way visible to the caller
def replaceItems(ilist, newcontentlist):
del ilist[:] # Modification visible to the caller
ilist.extend(newcontentlist) # Modification visible to the caller
ilist = [5, 6] # No outside effect; lets the local ilist point to a new list object,
# losing the reference to the list object passed as an argument
def clearSet(iset):
iset.clear()
def tryToTouchAnInteger(iint):
iint += 1 # No outside effect; lets the local iint to point to a new int object,
# losing the reference to the int object passed as an argument
print("iint inside:",iint) # 4 if iint was 3 on function entry
list1 = [1, 2]
appendItem(list1, 3)
print(list1) # [1, 2, 3]
replaceItems(list1, [3, 4])
print(list1) # [3, 4]
set1 = set([1, 2])
clearSet(set1 )
print(set1) # set([])
int1 = 3
tryToTouchAnInteger(int1)
print(int1) # 3
Preventing Argument Change
If an argument is of an immutable type any changes made to it will remain local to the called function. However, if the argument is of a mutable type, such as a list, changes made to it will update the corresponding value in the calling function. Thus, if the calling function wants to make sure its mutable value passed to some unknown function will not be changed by it must create and pass a copy of the value.
An example:
def evil_get_length(ilist):
length = len(ilist)
del ilist[:] # Muhaha: clear the list
return length
list1 = [1, 2]
print(evil_get_length(list1[:])) # Pass a copy of list1
print(list1) # list1 = [1, 2]
print(evil_get_length(list1)) # list1 gets cleared
print(list1) # list1 = []
Calling Functions
A function can be called by appending the arguments in parentheses to the function name or an empty pair of parentheses if the function takes no arguments.
foo()
square(3)
bar(5, x)
A function's return value can be used by assigning it to a variable, like so:
x = foo()
y = bar(5,x)
As shown above, when calling a function you can specify the parameters by name and you can do so in any order
def display_message(message, start=0, end=4):
print(message[start:end])
display_message("message", end=3)
This above is valid and start will have the default value of 0. A restriction placed on this is after the first named argument then all arguments after it must also be named. The following is not valid
display_message(end=5, start=1, "my message")
because the third argument ("my message") is an unnamed argument.
Nested functions
Nested functions are functions defined within other functions. Arbitrary level of nesting is possible.
Nested functions can read variables declared in the immediately outside function. For such variables that are mutable, nested functions can even modify them. For such variables that are immutable such as integers, attempt at modification in the nested function throws UnboundLocalError. In Python 3, an immutable immediately outside variable can be declared in the nested function to be nonlocal, in an analogy to global. Once this is done, the nested function can assign a new value to that variable and that modification is going to be seen outside of the nested function.
Nested functions can be used in #Closures, as shown below. Furthermore, they can be used to reduce repetion of code that pertains only to a single function, often with reduced argument list owing to seeing the immediately outside variables.
An example of a nested function that modifies an immediately outside variable that is a list and therefore mutable:
def outside():
outsideList = [1, 2]
def nested():
outsideList.append(3)
nested()
print(outsideList)
An example in which the outside variable is first accessed below the nested function definition and it still works:
def outside():
def nested():
outsideList.append(3)
outsideList = [1, 2]
nested()
print(outsideList)
Keywords: inner functions, internal functions, local functions.
Links:
- Nested Function in Python stackoverflow.com
- 7. Simple statements # 7.13. The nonlocal statement, docs.python.org/3
- Python nonlocal statement, stackoverflow.com
Closures
A closure is a nested function with an after-return access to the data of the outer function, where the nested function is returned by the outer function as a function object. Thus, even when the outer function has finished its execution after being called, the closure function returned by it can refer to the values of the variables that the outer function had when it defined the closure function.
An example:
def adder(outer_argument): # outer function
def adder_inner(inner_argument): # inner function, nested function
return outer_argument + inner_argument # Notice outer_argument
return adder_inner
add5 = adder(5) # a function that adds 5 to its argument
add7 = adder(7) # a function that adds 7 to its argument
print(add5(3)) # prints 8
print(add7(3)) # prints 10
Closures are possible in Python because functions are first-class objects. A function is merely an object of type function. Being an object means it is possible to pass a function object (an uncalled function) around as argument or as return value or to assign another name to the function object. A unique feature that makes closure useful is that the enclosed function may use the names defined in the parent function's scope.
Lambda Expressions
A lambda is an anonymous (unnamed) function. It is used primarily to write very short functions that are a hassle to define in the normal way. A function like this:
>>> def add(a, b):
... return a + b
...
>>> add(4, 3)
7
may also be defined using lambda
>>> print ((lambda a, b: a + b)(4, 3))
7
Lambda is often used as an argument to other functions that expects a function object, such as sorted()'s 'key' argument.
>>> sorted([[3, 4], [3, 5], [1, 2], [7, 3]], key=lambda x: x[1])
[[1, 2], [7, 3], [3, 4], [3, 5]]
The lambda form is often useful as a closure, such as illustrated in the following example:
>>> def attribution(name):
... return lambda x: x + ' -- ' + name
...
>>> pp = attribution('John')
>>> pp('Dinner is in the fridge')
'Dinner is in the fridge -- John'
Note that the lambda function can use the values of variables from the scope in which it was created similar to regular locally defined functions described above. In fact, exporting the precalculations embodiedby its constructor function is one of the essential utilities of closures.
Links:
- 4.7.5. Lambda Expressions, The Python Tutorial, docs.python.org
Generator Functions
When discussing loops, you came across the concept of an iterator. This yields in turn each element of some sequence, rather than the entire sequence at once, allowing you to deal with sequences much larger than might be able to fit in memory at once.
You can create your own iterators, by defining what is known as a generator function. To illustrate the usefulness of this, let us start by considering a simple function to return the concatenation of two lists:
def concat(a, b):
return a + b
print(concat([5, 4, 3], ["a", "b", "c"]))
# prints [5, 4, 3, 'a', 'b', 'c']
Imagine wanting to do something like
concat(list(range(0, 1000000)), list(range(1000000, 2000000)))
That would work, but it would consume a lot of memory.
Consider an alternative definition, which takes two iterators as arguments:
def concat(a, b):
for i in a:
yield i
for i in b:
yield i
Notice the use of the yield statement, instead of return. We can now use this something like
for i in concat(range(0, 1000000), range(1000000, 2000000)):
print(i)
and print out an awful lot of numbers, without using a lot of memory at all.
Note: You can still pass a list or other sequence type wherever Python expects an iterator (like to an argument of your concat
function); this will still work, and makes it easy not to have to worry about the difference where you don’t need to.
Links:
- Functional Programming HOWTO # Generators, docs.python.org
External Links
- 4.6. Defining Functions, The Python Tutorial, docs.python.org
Scoping
Variables
Variables in Python are automatically declared by assignment. Variables are always references to objects, and are never typed. Variables exist only in the current scope or global scope. When they go out of scope, the variables are destroyed, but the objects to which they refer are not (unless the number of references to the object drops to zero).
Scope is delineated by function and class blocks. Both functions and their scopes can be nested. So therefore
def foo():
def bar():
x = 5 # x is now in scope
return x + y # y is defined in the enclosing scope later
y = 10
return bar() # now that y is defined, bar's scope includes y
Now when this code is tested,
>>> foo()
15
>>> bar()
Traceback (most recent call last):
File "<pyshell#26>", line 1, in -toplevel-
bar()
NameError: name 'bar' is not defined
The name 'bar' is not found because a higher scope does not have access to the names lower in the hierarchy.
It is a common pitfall to fail to assign an object to a variable before use. In its most common form:
>>> for x in range(10):
y.append(x) # append is an attribute of lists
Traceback (most recent call last):
File "<pyshell#46>", line 2, in -toplevel-
y.append(x)
NameError: name 'y' is not defined
Here, to correct this problem, one must add y = [] before the for loop executes.
A loop does not create its own scope:
for x in [1, 2, 3]:
inner = x
print(inner) # 3 rather than an error
Keyword global
Global variables of a Python module are read-accessible from functions in that module. In fact, if they are mutable, they can be also modified via method call. However, they cannot be modified by a plain assignment unless they are declared global in the function.
An example to clarify:
count1 = 1
count2 = 1
list1 = []
list2 = []
def test1():
print(count1) # Read access is unproblematic, referring to the global
def test2():
try:
print(count1) # This try block is problematic because...
count1 += 1 # count1 += 1 causes count1 to be local but local version is undefined.
except UnboundLocalError as error:
print("Error caught:", error)
def test3():
list1 = [2] # No outside effect; this defines list1 to be a local variable
def test4():
global count2, list2
print(count1) # Read access is unproblematic, referring to the global
count2 += 1 # We can modify count2 via assignment
list1.append(1) # Impacts the global list1 even without global declaration since its a method call
list2 = [2] # We can modify list2 via assignment
test1()
test2()
test3()
test4()
print("count1:", count1) # 1
print("count2:", count2) # 2
print("list1:", list1) # [1]
print("list2:", list2) # [2]
Links:
- 6.13. The global statement, docs.python.org
- What are the rules for local and global variables in Python? in Programming FAQ, docs.python.org
Keyword nonlocal
Keyword nonlocal, available since Python 3.0, is an analogue of global for nested scopes. It enables a nested function to assign-modify even an immutable variable that is local to the outer function.
An example:
# Requires Python 3
def outer():
outerint = 0
outerint2 = 10
def inner():
nonlocal outerint
outerint = 1 # Impacts outer's outerint only because of the nonlocal declaration
outerint2 = 1 # No impact
inner()
print(outerint)
print(outerint2)
outer()
Simulation of nonlocal in Python 2 via a mutable object:
def outer():
outerint = [1] # Technique 1: Store int in a list
class outerNL: pass # Technique 2: Store int in a class
outerNL.outerint2 = 11
def inner():
outerint[0] = 2 # List members can be modified
outerNL.outerint2 = 12 # Class members can be modified
inner()
print(outerint[0])
print(outerNL.outerint2)
outer()
Links:
- 7.13. The nonlocal statement, docs.python.org
globals and locals
To find out which variables exist in the global and local scopes, you can use locals() and globals() functions, which return dictionaries:
int1 = 1
def test1():
int1 = 2
globals()["int1"] = 3 # Write access seems possible
print(locals()["int1"])# 2
test1()
print(int1) # 3
Write access to locals() dictionary is discouraged by the Python documentation.
Links:
- 2. Built-in Functions # globals, docs.python.org
- 2. Built-in Functions # locals, docs.python.org
External links
- 4. Execution model, docs.python.org
- 7.13. The nonlocal statement, docs.python.org
- PEP 3104 -- Access to Names in Outer Scopes, python.org
Exceptions
Python 2 handles all errors with exceptions.
An exception is a signal that an error or other unusual condition has occurred. There are a number of built-in exceptions, which indicate conditions like reading past the end of a file, or dividing by zero. You can also define your own exceptions.
Overview
Exceptions in Python at a glance:
import random
try:
ri = random.randint(0, 2)
if ri == 0:
infinity = 1/0
elif ri == 1:
raise ValueError("Message")
#raise ValueError, "Message" # Deprecated
elif ri == 2:
raise ValueError # Without message
except ZeroDivisionError:
pass
except ValueError as valerr:
# except ValueError, valerr: # Deprecated?
print(valerr)
raise # Raises the exception just caught
except: # Any other exception
pass
finally: # Optional
pass # Clean up
class CustomValueError(ValueError): pass # Custom exception
try:
raise CustomValueError
raise TypeError
except (ValueError, TypeError): # Value error catches custom, a derived class, as well
pass # A tuple catches multiple exception classes
Raising exceptions
Whenever your program attempts to do something erroneous or meaningless, Python raises exception to such conduct:
>>> 1 / 0
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ZeroDivisionError: integer division or modulo by zero
This traceback indicates that the ZeroDivisionError
exception is being raised. This is a built-in exception -- see below for a list of all the other ones.
Catching exceptions
In order to handle errors, you can set up exception handling blocks in your code. The keywords try
and except
are used to catch exceptions. When an error occurs within the try
block, Python looks for a matching except
block to handle it. If there is one, execution jumps there.
If you execute this code:
try:
print(1/0)
except ZeroDivisionError:
print("You can't divide by zero!")
Then Python will print this:
You can't divide by zero!
If you don't specify an exception type on the except
line, it will cheerfully catch all exceptions. This is generally a bad idea in production code, since it means your program will blissfully ignore unexpected errors as well as ones which the except
block is actually prepared to handle.
Exceptions can propagate up the call stack:
def f(x):
return g(x) + 1
def g(x):
if x < 0: raise ValueError, "I can't cope with a negative number here."
else: return 5
try:
print(f(-6))
except ValueError:
print("That value was invalid.")
In this code, the print
statement calls the function f
. That function calls the function g
, which will raise an exception of type ValueError. Neither f
nor g
has a try
/except
block to handle ValueError. So the exception raised propagates out to the main code, where there is an exception-handling block waiting for it. This code prints:
That value was invalid.
Sometimes it is useful to find out exactly what went wrong, or to print the python error text yourself. For example:
try:
the_file = open("the_parrot")
except IOError, (ErrorNumber, ErrorMessage):
if ErrorNumber == 2: # file not found
print("Sorry, 'the_parrot' has apparently joined the choir invisible.")
else:
print("Congratulation! you have managed to trip a #%d error" % ErrorNumber)
print(ErrorMessage)
Which will print:
Sorry, 'the_parrot' has apparently joined the choir invisible.
Custom Exceptions
Code similar to that seen above can be used to create custom exceptions and pass information along with them. This can be very useful when trying to debug complicated projects. Here is how that code would look; first creating the custom exception class:
class CustomException(Exception):
def __init__(self, value):
self.parameter = value
def __str__(self):
return repr(self.parameter)
And then using that exception:
try:
raise CustomException("My Useful Error Message")
except CustomException, (instance):
print("Caught: " + instance.parameter)
Trying over and over again
Recovering and continuing with finally
Exceptions could lead to a situation where, after raising an exception, the code block where the exception occurred might not be revisited. In some cases this might leave external resources used by the program in an unknown state.
finally
clause allows programmers to close such resources in case of an exception. Between 2.4 and 2.5 version of python there is change of syntax for finally
clause.
- Python 2.4
try:
result = None
try:
result = x/y
except ZeroDivisionError:
print("division by zero!")
print("result is ", result)
finally:
print("executing finally clause")
- Python 2.5
try:
result = x / y
except ZeroDivisionError:
print("division by zero!")
else:
print("result is", result)
finally:
print("executing finally clause")
Built-in exception classes
All built-in Python exceptions
Exotic uses of exceptions
Exceptions are good for more than just error handling. If you have a complicated piece of code to choose which of several courses of action to take, it can be useful to use exceptions to jump out of the code as soon as the decision can be made. The Python-based mailing list software Mailman does this in deciding how a message should be handled. Using exceptions like this may seem like it's a sort of GOTO -- and indeed it is, but a limited one called an escape continuation. Continuations are a powerful functional-programming tool and it can be useful to learn them.
Just as a simple example of how exceptions make programming easier, say you want to add items to a list but you don't want to use "if" statements to initialize the list we could replace this:
if hasattr(self, 'items'):
self.items.extend(new_items)
else:
self.items = list(new_items)
Using exceptions, we can emphasize the normal program flow—that usually we just extend the list—rather than emphasizing the unusual case:
try:
self.items.extend(new_items)
except AttributeError:
self.items = list(new_items)
External links
- 8. Errors and Exceptions in The Python Tutorial, docs.python.org
- 8. Errors and Exceptions in The Python Tutorial for Python 2.4, docs.python.org
- 6. Built-in Exceptions, docs.python.org
Input and output
Input
Python 3.x has one function for input from user, input()
. By contrast, legacy Python 2.x has two functions for input from user: input()
and raw_input()
.
There are also very simple ways of reading a file and, for stricter control over input, reading from stdin if necessary.
input() in Python 3.x
In Python 3.x, input() asks the user for a string of data (ended with a newline), and simply returns the string. It can also take an argument, which is displayed as a prompt before the user enters the data. E.g.
print(input('What is your name?'))
prints out
What is your name? <user input data here>
Example: to assign the user's name, i.e. string data, to a variable "x" you would type
x = input('What is your name?')
In legacy Python 2.x, the above applies to what was raw_input()
function, and there was also input()
function that behaved differently, automatically evaluating what the user entered; in Python 3, the same would be achieved via eval(input())
.
Links:
- input() in Built-in Functions in Library Reference for Python 3, docs.python.org
- raw_input() in Built-in Functions in Library Reference for Python 2, docs.python.org
input() in Python 2.x
In legacy Python 2.x, input() takes the input from the user as a string and evaluates it.
Therefore, if a script says:
x = input('What are the first 10 perfect squares? ')
it is possible for a user to input:
map(lambda x: x*x, range(10))
which yields the correct answer in list form. Note that no inputted statement can span more than one line.
input() should not be used for anything but the most trivial program, for security reasons. Turning the strings returned from raw_input() into Python types using an idiom such as:
x = None
while not x:
try:
x = int(raw_input())
except ValueError:
print('Invalid Number')
is preferable, as input() uses eval() to turn a literal into a Python type, which allows a malicious person to run arbitrary code from inside your program trivially.
Links:
- input() in Built-in Functions in Library Reference for Python 2, docs.python.org
File Input
File Objects
To read from a file, you can iterate over the lines of the file using open:
f = open('test.txt', 'r')
for line in f:
print(line[0])
f.close()
This will print the first character of each line. A newline is attached to the end of each line read this way. The second argument to open can be 'r', 'w', or 'rw', among some others.
The newer and better way to read from a file:
with open("test.txt", "r") as txt:
for line in txt:
print(line)
The advantage is that the opened file will close itself after finishing the part within the with statement, and will do so even if an exception is thrown.
Because files are automatically closed when the file object goes out of scope, there is no real need to close them explicitly. So, the loop in the previous code can also be written as:
for line in open('test.txt', 'r'):
print(line[0])
You can read a specific numbers of characters at a time:
c = f.read(1)
while len(c) > 0:
if len(c.strip()) > 0: print(c)
c = f.read(1)
This will read the characters from f one at a time, and then print them if they're not whitespace.
A file object implicitly contains a marker to represent the current position. If the file marker should be moved back to the beginning, one can either close the file object and reopen it or just move the marker back to the beginning with:
f.seek(0)
Standard File Objects
There are built-in file objects representing standard input, output, and error. These are in the sys module and are called stdin, stdout, and stderr. There are also immutable copies of these in __stdin__, __stdout__, and __stderr__. This is for IDLE and other tools in which the standard files have been changed.
You must import the sys module to use the special stdin, stdout, stderr I/O handles.
import sys
For finer control over input, use sys.stdin.read(). To implement the UNIX 'cat' program in Python, you could do something like this:
import sys
for line in sys.stdin:
print(line, end="")
Note that sys.stdin.read() will read from standard input till EOF. (which is usually Ctrl+D.)
Parsing command line
Command-line arguments passed to a Python program are stored in sys.argv list. The first item in the list is name of the Python program, which may or may not contain the full path depending on the manner of invocation. sys.argv list is modifiable.
Printing all passed arguments except for the program name itself:
import sys
for arg in sys.argv[1:]:
print(arg)
Parsing passed arguments for passed minus options:
import sys
option_f = False
option_p = False
option_p_argument = ""
i = 1
while i < len(sys.argv):
if sys.argv[i] == "-f":
option_f = True
sys.argv.pop(i)
elif sys.argv[i] == "-p":
option_p = True
sys.argv.pop(i)
option_p_argument = sys.argv.pop(i)
else:
i += 1
Above, the arguments at which options are found are removed so that sys.argv can be looped for all remaining arguments.
Parsing of command-line arguments is further supported by library modules optparse (deprecated), argparse (since Python 2.7) and getopt (to make life easy for C programmers).
A minimum parsing example for argparse:
import argparse
parser = argparse.ArgumentParser(description="Concatenates two strings")
addarg = parser.add_argument
addarg("s1", help="First string to concatenate")
addarg("s2", help="Second string to concatenate")
args = parser.parse_args()
result = args.s1 + args.s2
print(result)
Parse with argparse--specify the arg type as int:
import argparse
parser = argparse.ArgumentParser(description="Sum two ints")
addarg = parser.add_argument
addarg("i1", help="First int to add", type=int)
addarg("i2", help="Second int to add", type=int)
args = parser.parse_args()
result = args.i1 + args.i2
print(result)
Parse with argparse--add optional switch -m to yield multiplication instead of addition:
import argparse
parser = argparse.ArgumentParser(description="Sums or multiplies two ints.")
addarg = parser.add_argument
addarg("i1", help="First int", type=int)
addarg("i2", help="Second int", type=int)
addarg("-m", help="Multiplies rather than adds.", action="store_true")
args = parser.parse_args()
if args.m:
result = args.i1 * args.i2
else:
result = args.i1 + args.i2
print(result)
Parse with argparse--set an argument to consume one or more items:
import argparse
parser = argparse.ArgumentParser(description="Sums one or more ints.")
addarg = parser.add_argument
addarg("intlist", help="Ints", type=int, nargs="+")
args = parser.parse_args()
result = 0
for item in args.intlist:
result += item
print(result)
Usage example: python ArgparseTest.py 1 3 5
Parse with argparse--as above but with a help epilog to be output after parameter descriptions upon -h:
import argparse
parser = argparse.ArgumentParser(description="Sums one or more ints.",
epilog="Example: python ArgparseTest.py 1 3 5")
addarg = parser.add_argument
addarg("intlist", help="Ints", type=int, nargs="+")
args = parser.parse_args()
result = 0
for item in args.intlist:
result += item
print(result)
Parse with argparse--make second integer argument optional via nargs:
import argparse
parser = argparse.ArgumentParser(description="Sums one or two integers.",
epilog="Example: python ArgparseTest.py 3 4\n"
"Example: python ArgparseTest.py 3")
addarg = parser.add_argument
addarg("i1", help="First int", type=int)
addarg("i2", help="Second int, optional, defaulting to 1.", type=int, default=1, nargs="?")
args = parser.parse_args()
result = args.i1 + args.i2
print(result)
Links:
- The Python Standard Library - 28.1. sys, docs.python.org
- The Python Standard Library - 15.4. argparse, docs.python.org
- The Python Standard Library - 15.5. optparse, docs.python.org
- The Python Standard Library - 15.6. getopt, docs.python.org
- Argparse Tutorial, docs.python.org
Output
The basic way to do output is the print statement.
print('Hello, world')
To print multiple things on the same line separated by spaces, use commas between them:
print('Hello,', 'World')
This will print out the following:
Hello, World
While neither string contained a space, a space was added by the print statement because of the comma between the two objects. Arbitrary data types can be printed:
print(1, 2, 0xff, 0777, 10+5j, -0.999, map, sys)
This will output the following:
1 2 255 511 (10+5j) -0.999 <built-in function map> <module 'sys' (built-in)>
Objects can be printed on the same line without needing to be on the same line:
for i in range(10):
print(i, end=" ")
This will output the following:
0 1 2 3 4 5 6 7 8 9
To end the printed line with a newline, add a print statement without any objects.
for i in range(10):
print(i, end=" ")
print()
for i in range(10,20):
print(i, end=" ")
This will output the following:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
If the bare print statement were not present, the above output would look like:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
You can print to a file instead of to standard output:
print('Hello, world', file=f)
This will print to any object that implements write(), which includes file objects.
Note on legacy Python 2: in Python 2, print is a statement rather than a function and there is no need to put brackets around its arguments. Instead of print(i, end=" "), one would write print i,.
Omitting newlines
In Python 3.x, you can output without a newline by passing end="" to the print function or by using the method write:
import sys
print("Hello", end="")
sys.stdout.write("Hello") # Or stderr to write to standard error stream.
In Python 2.x, to avoid adding spaces and newlines between objects' output with subsequent print statements, you can do one of the following:
Concatenation: Concatenate the string representations of each object, then later print the whole thing at once.
print(str(1)+str(2)+str(0xff)+str(0777)+str(10+5j)+str(-0.999)+str(map)+str(sys))
This will output the following:
12255511(10+5j)-0.999<built-in function map><module 'sys' (built-in)>
Write function: You can make a shorthand for sys.stdout.write and use that for output.
import sys
write = sys.stdout.write
write('20')
write('05\n')
This will output the following:
2005
You may need sys.stdout.flush() to get that text on the screen quickly.
Examples
Examples of output with Python 3.x:
- from __future__ import print_function
- Ensures Python 2.6 and later Python 2.x can use Python 3.x print function.
- print("Hello", "world")
- Prints the two words separated with a space. Notice the surrounding brackets, ununsed in Python 2.x.
- print("Hello world", end="")
- Prints without the ending newline.
- print("Hello", "world", sep="-")
- Prints the two words separated with a dash.
- print("Hello", 34)
- Prints elements of various data types, separating them by a space.
- print("Hello " + 34)
- Throws an error as a result of trying to concatenate a string and an integer.
- print("Hello " + str(34))
- Uses "+" to concatenate strings, after converting a number to a string.
- sum=2+2; print "The sum: %i" % sum
- Prints a string that has been formatted with the use of an integer passed as an argument. See also #Formatting.
- print ("Error", file=sys.stderr)
- Outputs to a file handle, in this case standard error stream.
Examples of output with Python 2.x:
- print "Hello"
- print "Hello", "world"
- Separates the two words with a space.
- print "Hello", 34
- Prints elements of various data types, separating them by a space.
- print "Hello " + 34
- Throws an error as a result of trying to concatenate a string and an integer.
- print "Hello " + str(34)
- Uses "+" to concatenate strings, after converting a number to a string.
- print "Hello",
- Prints "Hello " without a newline, with a space at the end.
- sys.stdout.write("Hello")
- Prints "Hello" without a newline. Doing "import sys" is a prerequisite. Needs a subsequent "sys.stdout.flush()" in order to display immediately on the user's screen.
- sys.stdout.write("Hello\n")
- Prints "Hello" with a newline.
- print >> sys.stderr, "An error occurred."
- Prints to standard error stream.
- sys.stderr.write("Hello\n")
- Prints to standard error stream.
- sum=2+2; print "The sum: %i" % sum
- Prints a string that has been formatted with the use of an integer passed as an argument.
- formatted_string = "The sum: %i" % (2+2); print formatted_string
- Like the previous, just that the formatting happens outside of the print statement.
- print "Float: %6.3f" % 1.23456
- Outputs "Float: 1.234". The number 3 after the period specifies the number of decimal digits after the period to be displayed, while 6 before the period specifies the total number of characters the displayed number should take, to be padded with spaces if needed.
- print "%s is %i years old" % ("John", 23)
- Passes two arguments to the formatter.
File Output
Printing numbers from 1 to 10 to a file, one per line:
file1 = open("TestFile.txt","w")
for i in range(1,10+1):
print(i, file=file1)
file1.close()
With "w", the file is opened for writing. With "file=file1", print sends its output to a file rather than standard output.
Printing numbers from 1 to 10 to a file, separated with a dash:
file1 = open("TestFile.txt", "w")
for i in range(1, 10+1):
if i > 1:
file1.write("-")
file1.write(str(i))
file1.close()
Opening a file for appending rather than overwriting:
file1 = open("TestFile.txt", "a")
In Python 2.x, a redirect to a file is done like print >>file1, i.
See also Files chapter.
Formatting
Formatting numbers and other values as strings using the string percent operator:
v1 = "Int: %i" % 4 # 4
v2 = "Int zero padded: %03i" % 4 # 004
v3 = "Int space padded: %3i" % 4 # 4
v4 = "Hex: %x" % 31 # 1f
v5 = "Hex 2: %X" % 31 # 1F - capitalized F
v6 = "Oct: %o" % 8 # 10
v7 = "Float: %f" % 2.4 # 2.400000
v8 = "Float: %.2f" % 2.4 # 2.40
v9 = "Float in exp: %e" % 2.4 # 2.400000e+00
vA = "Float in exp: %E" % 2.4 # 2.400000E+00
vB = "List as string: %s" % [1, 2, 3]
vC = "Left padded str: %10s" % "cat"
vD = "Right padded str: %-10s" % "cat"
vE = "Truncated str: %.2s" % "cat"
vF = "Dict value str: %(age)s" % {"age": 20}
vG = "Char: %c" % 65 # A
vH = "Char: %c" % "A" # A
Formatting numbers and other values as strings using the format() string method, since Python 2.6:
v1 = "Arg 0: {0}".format(31) # 31
v2 = "Args 0 and 1: {0}, {1}".format(31, 65)
v3 = "Args 0 and 1: {}, {}".format(31, 65)
v4 = "Arg indexed: {0[0]}".format(["e1", "e2"])
v5 = "Arg named: {a}".format(a=31)
v6 = "Hex: {0:x}".format(31) # 1f
v7 = "Hex: {:x}".format(31) # 1f - arg 0 is implied
v8 = "Char: {0:c}".format(65) # A
v9 = "Hex: {:{h}}".format(31, h="x") # 1f - nested evaluation
Formatting numbers and other values as strings using literal string interpolation, since Python 3.6:
int1 = 31; int2 = 41; str1="aaa"; myhex = "x"
v1 = f"Two ints: {int1} {int2}"
v2 = f"Int plus 1: {int1+1}" # 32 - expression evaluation
v3 = f"Str len: {len(str1)}" # 3 - expression evaluation
v4 = f"Hex: {int1:x}" # 1f
v5 = f"Hex: {int1:{myhex}}" # 1f - nested evaluation
Links:
- 5.6.2. String Formatting Operations, docs.python.org
- 2. Built-in Functions # format, docs.python.org
- 7.1.2. Custom String Formatting, docs.python.org
- 7.1.3.1. Format Specification Mini-Language, docs.python.org
- 7.1.4. Template strings, docs.python.org
- PEP 3101 -- Advanced String Formatting, python.org
- PEP 498 -- Literal String Interpolation, python.org
External Links
- 7. Input and Output in The Python Tutorial, python.org
- 6.6. The print statement in The Python Language Reference, python.org
- 2. Built-in Functions #open in The Python Standard Library at Python Documentation, python.org
- 5. Built-in Types #file.write in The Python Standard Library at Python Documentation, python.org
- 27.1. sys — System-specific parameters and functions in Python Documentation, python org -- mentions sys.stdout, and sys.stderr
- 2.3.8 File Objects in Python Library Reference, python.org, for "flush"
- 5.6.2. String Formatting Operations in The Python Standard Library at Python Documentation, python.org -- for "%i", "%s" and similar string formatting
- 7.2.2. The string format operator, in Python 2.5 quick reference, nmt.edu, for "%i", "%s" and similar string formatting
Modules
Modules are a way to structure a program and create reusable libraries. A module is usually stored in and corresponds to a separate .py file. Many modules are available from the standard library. You can create your own modules. Python searches for modules in the current directory and other locations; the list of module search locations can be expanded by expanding PYTHONPATH environment variable and by other means.
Importing a Module
To use the functions and classes offered by a module, you have to import the module:
import math
print(math.sqrt(10))
The above imports the math standard module, making all of the functions in that module namespaced by the module name. It imports all functions and all classes, if any.
You can import the module under a different name:
import math as Mathematics
print(Mathematics.sqrt(10))
You can import a single function, making it available without the module name namespace:
from math import sqrt
print(sqrt(10))
You can import a single function and make it available under a different name:
from math import cos as cosine
print(cosine(10))
You can import multiple modules in a row:
import os, sys, re
You can make an import as late as in a function definition:
def sqrtTen():
import math
print(math.sqrt(10))
Such an import only takes place when the function is called.
You can import all functions from the module without the module namespace, using an asterisk notation:
from math import *
print(sqrt(10))
However, if you do this inside a function, you get a warning in Python 2 and error in Python 3:
def sqrtTen():
from math import *
print(sqrt(10))
You can guard for a module not found:
try:
import custommodule
except ImportError:
pass
Modules can be different kinds of things:
- Python files
- Shared Objects (under Unix and Linux) with the .so suffix
- DLL's (under Windows) with the .pyd suffix
- Directories
Modules are loaded in the order they're found, which is controlled by sys.path. The current directory is always on the path.
Directories should include a file in them called __init__.py, which should probably include the other files in the directory.
Creating a DLL that interfaces with Python is covered in another section.
Imported Check
You can check whether a module has been imported as follows:
if "re" in sys.modules:
print("Regular expression module is ready for use.")
Links:
- 28.1. sys # sys.modules, docs.python.org
Creating a Module
From a File
The easiest way to create a module is by having a file called mymod.py either in a directory recognized by the PYTHONPATH variable or (even easier) in the same directory where you are working. If you have the following file mymod.py
class Object1:
def __init__(self):
self.name = 'object 1'
you can already import this "module" and create instances of the object Object1.
import mymod
myobject = mymod.Object1()
from mymod import *
myobject = Object1()
From a Directory
It is not feasible for larger projects to keep all classes in a single file. It is often easier to store all files in directories and load all files with one command. Each directory needs to have a __init__.py
file which contains python commands that are executed upon loading the directory.
Suppose we have two more objects called Object2
and Object3
and we want to load all three objects with one command. We then create a directory called mymod and we store three files called Object1.py
, Object2.py
and Object3.py
in it. These files would then contain one object per file but this not required (although it adds clarity). We would then write the following __init__.py
file:
from Object1 import *
from Object2 import *
from Object3 import *
__all__ = ["Object1", "Object2", "Object3"]
The first three commands tell python what to do when somebody loads the module. The last statement defining __all__ tells python what to do when somebody executes from mymod import *. Usually we want to use parts of a module in other parts of a module, e.g. we want to use Object1 in Object2. We can do this easily with an from . import * command as the following file Object2.py shows:
from . import *
class Object2:
def __init__(self):
self.name = 'object 2'
self.otherObject = Object1()
We can now start python and import mymod as we have in the previous section.
Making a program usable as a module
In order to make a program usable both as a standalone program to be called from a command line and as a module, it is advisable that you place all code in functions and methods, designate one function as the main one, and call then main function when __name__ built-in equals '__main__'. The purpose of doing so is to make sure that the code you have placed in the main function is not called when your program is imported as a module; the code would be called upon import if it were placed outside of functions and methods.
Your program, stored in mymodule.py, can look as follows:
def reusable_function(x, y):
return x + y
def main():
pass
# Any code you like
if __name__ == '__main__':
main()
The uses of the above program can look as follows:
from mymodule import reusable_function
my_result = reusable_function(4, 5)
Links:
Extending Module Path
When import is requested, modules are searched in the directories (and zip files?) in the module path, accessible via sys.path, a Python list. The module path can be extended as follows:
import sys
sys.path.append("/My/Path/To/Module/Directory")
from ModuleFileName import my_function
Above, if ModuleFileName.py is located at /My/Path/To/Module/Directory and contains a definition of my_function, the 2nd line ensures the 3rd line actually works.
Links:
- The Module Search Path at Python doc
Module Names
Module names seem to be limited to alphanumeric characters and underscore; dash cannot be used. While my-module.py can be created and run, importing my-module fails. The name of a module is the name of the module file minus the .py suffix.
Module names are case sensitive. If the module file is called MyModule.py, doing "import mymodule" fails while "import MyModule" is fine.
PEP 0008 recommends module names to be in all lowercase, with possible use of underscores.
Examples of module names from the standard library include math, sys, io, re, urllib, difflib, and unicodedata.
Links:
- Package and Module Names at PEP 0008 -- Style Guide for Python Code
- The Python Standard Library at docs.python.org
Built-in Modules
For a module to be built-in is not the same as to be part of the standard library. For instance, re is not a built-in module but rather a module written in Python. By contrast, _sre is a built-in module.
Obtaining a list of built-in module names:
print(sys.builtin_module_names) print("_sre" in sys.builtin_module_names) # True print("math" in sys.builtin_module_names) # True
Links:
- 28.1. sys # sys.builtin_module_names, docs.python.org
- The Python Standard Library, docs.python.org
External links
- 6. Modules, The Python Tutorial, python.org
- Python Module Index, python.org
- 31. Importing Modules, python.org
- Installing Python Modules, python.org
- Idioms and Anti-Idioms in Python, python.org
- Python: Why should 'from <module> import *' be prohibited?, stackoverflow.com
- Error handling when importing modules, stackoverflow.com
Classes
Classes are a way of aggregating similar data and functions. A class is basically a scope inside which various code (especially function definitions) is executed, and the locals to this scope become attributes of the class, and of any objects constructed by this class. An object constructed by a class is called an instance of that class.
Overview
Classes in Python at a glance:
import math
class MyComplex:
"""A complex number""" # Class documentation
classvar = 0.0 # A class attribute, not an instance one
def phase(self): # A method
return math.atan2(self.imaginary, self.real)
def __init__(self): # A constructor
"""A constructor"""
self.real = 0.0 # An instance attribute
self.imaginary = 0.0
c1 = MyComplex()
c1.real = 3.14 # No access protection
c1.imaginary = 2.71
phase = c1.phase() # Method call
c1.undeclared = 9.99 # Add an instance attribute
del c1.undeclared # Delete an instance attribute
print(vars(c1)) # Attributes as a dictionary
vars(c1)["undeclared2"] = 7.77 # Write access to an attribute
print(c1.undeclared2) # 7.77, indeed
MyComplex.classvar = 1 # Class attribute access
print(c1.classvar == 1) # True; class attribute access, not an instance one
print("classvar" in vars(c1)) # False
c1.classvar = -1 # An instance attribute overshadowing the class one
MyComplex.classvar = 2 # Class attribute access
print(c1.classvar == -1) # True; instance attribute access
print("classvar" in vars(c1)) # True
class MyComplex2(MyComplex): # Class derivation or inheritance
def __init__(self, re = 0, im = 0):
self.real = re # A constructor with multiple arguments with defaults
self.imaginary = im
def phase(self):
print("Derived phase")
return MyComplex.phase(self) # Call to a base class; "super"
c3 = MyComplex2()
c4 = MyComplex2(1, 1)
c4.phase() # Call to the method in the derived class
class Record: pass # Class as a record/struct with arbitrary attributes
record = Record()
record.name = "Joe"
record.surname = "Hoe"
Defining a Class
To define a class, use the following format:
class ClassName:
"Here is an explanation about your class"
pass
The capitalization in this class definition is the convention, but is not required by the language. It's usually good to add at least a short explanation of what your class is supposed to do. The pass statement in the code above is just to say to the python interpreter just go on and do nothing. You can remove it as soon as you are adding your first statement.
Instance Construction
The class is a callable object that constructs an instance of the class when called. Let's say we create a class Foo.
class Foo:
"Foo is our new toy."
pass
To construct an instance of the class, Foo, "call" the class object:
f = Foo()
This constructs an instance of class Foo and creates a reference to it in f.
Class Members
In order to access the member of an instance of a class, use the syntax <class instance>.<member>
. It is also possible to access the members of the class definition with <class name>.<member>
.
Methods
A method is a function within a class. The first argument (methods must always take at least one argument) is always the instance of the class on which the function is invoked. For example
>>> class Foo:
... def setx(self, x):
... self.x = x
... def bar(self):
... print(self.x)
If this code were executed, nothing would happen, at least until an instance of Foo were constructed, and then bar were called on that instance.
Why a mandatory argument?
In a normal function, if you were to set a variable, such as test = 23
, you could not access the test variable. Typing test
would say it is not defined. This is true in class functions unless they use the self
variable.
Basically, in the previous example, if we were to remove self.x, function bar could not do anything because it could not access x. The x in setx() would disappear. The self argument saves the variable into the class's "shared variables" database.
Why self?
You do not need to use self. However, it is a norm to use self.
Invoking Methods
Calling a method is much like calling a function, but instead of passing the instance as the first parameter like the list of formal parameters suggests, use the function as an attribute of the instance.
>>> f = Foo()
>>> f.setx(5)
>>> f.bar()
This will output
5
It is possible to call the method on an arbitrary object, by using it as an attribute of the defining class instead of an instance of that class, like so:
>>> Foo.setx(f,5)
>>> Foo.bar(f)
This will have the same output.
Dynamic Class Structure
As shown by the method setx above, the members of a Python class can change during runtime, not just their values, unlike classes in languages like C++ or Java. We can even delete f.x after running the code above.
>>> del f.x
>>> f.bar()
Traceback (most recent call last): File "<stdin>", line 1, in ? File "<stdin>", line 5, in bar AttributeError: Foo instance has no attribute 'x'
Another effect of this is that we can change the definition of the Foo class during program execution. In the code below, we create a member of the Foo class definition named y. If we then create a new instance of Foo, it will now have this new member.
>>> Foo.y = 10
>>> g = Foo()
>>> g.y
10
Viewing Class Dictionaries
At the heart of all this is a dictionary that can be accessed by "vars(ClassName)"
>>> vars(g)
{}
At first, this output makes no sense. We just saw that g had the member y, so why isn't it in the member dictionary? If you remember, though, we put y in the class definition, Foo, not g.
>>> vars(Foo)
{'y': 10, 'bar': <function bar at 0x4d6a3c>, '__module__': '__main__',
'setx': <function setx at 0x4d6a04>, '__doc__': None}
And there we have all the members of the Foo class definition. When Python checks for g.member, it first checks g's vars dictionary for "member," then Foo. If we create a new member of g, it will be added to g's dictionary, but not Foo's.
>>> g.setx(5)
>>> vars(g)
{'x': 5}
Note that if we now assign a value to g.y, we are not assigning that value to Foo.y. Foo.y will still be 10, but g.y will now override Foo.y
>>> g.y = 9
>>> vars(g)
{'y': 9, 'x': 5}
>>> vars(Foo)
{'y': 10, 'bar': <function bar at 0x4d6a3c>, '__module__': '__main__',
'setx': <function setx at 0x4d6a04>, '__doc__': None}
Sure enough, if we check the values:
>>> g.y
9
>>> Foo.y
10
Note that f.y will also be 10, as Python won't find 'y' in vars(f), so it will get the value of 'y' from vars(Foo).
Some may have also noticed that the methods in Foo appear in the class dictionary along with the x and y. If you remember from the section on lambda functions, we can treat functions just like variables. This means that we can assign methods to a class during runtime in the same way we assigned variables. If you do this, though, remember that if we call a method of a class instance, the first parameter passed to the method will always be the class instance itself.
Changing Class Dictionaries
We can also access the members dictionary of a class using the __dict__ member of the class.
>>> g.__dict__
{'y': 9, 'x': 5}
If we add, remove, or change key-value pairs from g.__dict__, this has the same effect as if we had made those changes to the members of g.
>>> g.__dict__['z'] = -4
>>> g.z
-4
Why use classes?
Classes are special due to the fact once an instance is made, the instance is independent of all other instances. I could have two instances, each with a different x value, and they will not affect the other's x.
f = Foo()
f.setx(324)
f.boo()
g = Foo()
g.setx(100)
g.boo()
f.boo()
and g.boo()
will print different values.
New Style Classes
New style classes were introduced in python 2.2. A new-style class is a class that has a built-in as its base, most commonly object. At a low level, a major difference between old and new classes is their type. Old class instances were all of type instance
. New style class instances will return the same thing as x.__class__ for their type. This puts user defined classes on a level playing field with built-ins. Old/Classic classes are slated to disappear in Python 3. With this in mind all development should use new style classes. New Style classes also add constructs like properties and static methods familiar to Java programmers.
Old/Classic Class
>>> class ClassicFoo:
... def __init__(self):
... pass
New Style Class
>>> class NewStyleFoo(object):
... def __init__(self):
... pass
Properties
Properties are attributes with getter and setter methods.
>>> class SpamWithProperties(object):
... def __init__(self):
... self.__egg = "MyEgg"
... def get_egg(self):
... return self.__egg
... def set_egg(self, egg):
... self.__egg = egg
... egg = property(get_egg, set_egg)
>>> sp = SpamWithProperties()
>>> sp.egg
'MyEgg'
>>> sp.egg = "Eggs With Spam"
>>> sp.egg
'Eggs With Spam'
>>>
and since Python 2.6, with @property decorator
>>> class SpamWithProperties(object):
... def __init__(self):
... self.__egg = "MyEgg"
... @property
... def egg(self):
... return self.__egg
... @egg.setter
... def egg(self, egg):
... self.__egg = egg
Static Methods
Static methods in Python are just like their counterparts in C++ or Java. Static methods have no "self" argument and don't require you to instantiate the class before using them. They can be defined using staticmethod()
>>> class StaticSpam(object):
... def StaticNoSpam():
... print("You can't have have the spam, spam, eggs and spam without any spam... that's disgusting")
... NoSpam = staticmethod(StaticNoSpam)
>>> StaticSpam.NoSpam()
You can't have have the spam, spam, eggs and spam without any spam... that's disgusting
They can also be defined using the function decorator @staticmethod.
>>> class StaticSpam(object):
... @staticmethod
... def StaticNoSpam():
... print("You can't have have the spam, spam, eggs and spam without any spam... that's disgusting")
Inheritance
Like all object oriented languages, Python provides support for inheritance. Inheritance is a simple concept by which a class can extend the facilities of another class, or in Python's case, multiple other classes. Use the following format for this:
class ClassName(BaseClass1, BaseClass2, BaseClass3,...):
...
ClassName is what is known as the derived class, that is, derived from the base classes. The derived class will then have all the members of its base classes. If a method is defined in the derived class and in the base class, the member in the derived class will override the one in the base class. In order to use the method defined in the base class, it is necessary to call the method as an attribute on the defining class, as in Foo.setx(f,5) above:
>>> class Foo:
... def bar(self):
... print("I'm doing Foo.bar()")
... x = 10
...
>>> class Bar(Foo):
... def bar(self):
... print("I'm doing Bar.bar()")
... Foo.bar(self)
... y = 9
...
>>> g = Bar()
>>> Bar.bar(g)
I'm doing Bar.bar()
I'm doing Foo.bar()
>>> g.y
9
>>> g.x
10
Once again, we can see what's going on under the hood by looking at the class dictionaries.
>>> vars(g)
{}
>>> vars(Bar)
{'y': 9, '__module__': '__main__', 'bar': <function bar at 0x4d6a04>,
'__doc__': None}
>>> vars(Foo)
{'x': 10, '__module__': '__main__', 'bar': <function bar at 0x4d6994>,
'__doc__': None}
When we call g.x, it first looks in the vars(g) dictionary, as usual. Also as above, it checks vars(Bar) next, since g is an instance of Bar. However, thanks to inheritance, Python will check vars(Foo) if it doesn't find x in vars(Bar).
Multiple inheritance
As shown in section #Inheritance, a class can be derived from multiple classes:
class ClassName(BaseClass1, BaseClass2, BaseClass3):
pass
A tricky part about multiple inheritance is method resolution: upon a method call, if the method name is available from multiple base classes or their base classes, which base class method should be called.
The method resolution order depends on whether the class is an old-style class or a new-style class. For old-style classes, derived classes are considered from left to right, and base classes of base classes are considered before moving to the right. Thus, above, BaseClass1 is considered first, and if method is not found there, the base classes of BaseClass1 are considered. If that fails, BaseClass2 is considered, then its base classes, and so on. For new-style classes, see the Python documentation online.
Links:
- 9.5.1. Multiple Inheritance, docs.python.org
- The Python 2.3 Method Resolution Order, python.org
Special Methods
There are a number of methods which have reserved names which are used for special purposes like mimicking numerical or container operations, among other things. All of these names begin and end with two underscores. It is convention that methods beginning with a single underscore are 'private' to the scope they are introduced within.
Initialization and Deletion
__init__
One of these purposes is constructing an instance, and the special name for this is '__init__'. __init__() is called before an instance is returned (it is not necessary to return the instance manually). As an example,
class A:
def __init__(self):
print('A.__init__()')
a = A()
outputs
A.__init__()
__init__() can take arguments, in which case it is necessary to pass arguments to the class in order to create an instance. For example,
class Foo:
def __init__ (self, printme):
print(printme)
foo = Foo('Hi!')
outputs
Hi!
Here is an example showing the difference between using __init__() and not using __init__():
class Foo:
def __init__ (self, x):
print(x)
foo = Foo('Hi!')
class Foo2:
def setx(self, x):
print(x)
f = Foo2()
Foo2.setx(f,'Hi!')
outputs
Hi! Hi!
__del__
Similarly, '__del__' is called when an instance is destroyed; e.g. when it is no longer referenced.
__enter__ and __exit__
These methods are also a constructor and a destructor but they're only executed when the class is instantiated with with
. Example:
class ConstructorsDestructors:
def __init__(self):
print('init')
def __del__(self):
print('del')
def __enter__(self):
print('enter')
def __exit__(self, exc_type, exc_value, traceback):
print('exit')
with ConstructorsDestructors():
pass
init enter exit del
__new__
Metaclass constructor.
Representation
__str__Converting an object to a string, as with the print statement or with the str() conversion function, can be overridden by overriding __str__. Usually, __str__ returns a formatted version of the objects content. This will NOT usually be something that can be executed. For example: class Bar:
def __init__ (self, iamthis):
self.iamthis = iamthis
def __str__ (self):
return self.iamthis
bar = Bar('apple')
print(bar)
outputs apple __repr__This function is much like __str__(). If __str__ is not present but this one is, this function's output is used instead for printing. __repr__ is used to return a representation of the object in string form. In general, it can be executed to get back the original object. For example: class Bar:
def __init__ (self, iamthis):
self.iamthis = iamthis
def __repr__(self):
return "Bar('%s')" % self.iamthis
bar = Bar('apple')
bar
outputs (note the difference: it may not be necessary to put it inside a print, however in Python 2.7 it does) Bar('apple') |
|
Attributes
__setattr__This is the function which is in charge of setting attributes of a class. It is provided with the name and value of the variables being assigned. Each class, of course, comes with a default __setattr__ which simply sets the value of the variable, but we can override it. >>> class Unchangable:
... def __setattr__(self, name, value):
... print("Nice try")
...
>>> u = Unchangable()
>>> u.x = 9
Nice try
>>> u.x
Traceback (most recent call last): File "<stdin>", line 1, in ? AttributeError: Unchangable instance has no attribute 'x' __getattr___Similar to __setattr__, except this function is called when we try to access a class member, and the default simply returns the value. >>> class HiddenMembers:
... def __getattr__(self, name):
... return "You don't get to see " + name
...
>>> h = HiddenMembers()
>>> h.anything
"You don't get to see anything"
__delattr__This function is called to delete an attribute. >>> class Permanent:
... def __delattr__(self, name):
... print(name, "cannot be deleted")
...
>>> p = Permanent()
>>> p.x = 9
>>> del p.x
x cannot be deleted
>>> p.x
9
|
|
Operator Overloading
Operator overloading allows us to use the built-in Python syntax and operators to call functions which we define.
Binary Operators
If a class has the __add__ function, we can use the '+' operator to add instances of the class. This will call __add__ with the two instances of the class passed as parameters, and the return value will be the result of the addition. >>> class FakeNumber:
... n = 5
... def __add__(A,B):
... return A.n + B.n
...
>>> c = FakeNumber()
>>> d = FakeNumber()
>>> d.n = 7
>>> c + d
12
To override the augmented assignment operators, merely add 'i' in front of the normal binary operator, i.e. for '+=' use '__iadd__' instead of '__add__'. The function will be given one argument, which will be the object on the right side of the augmented assignment operator. The returned value of the function will then be assigned to the object on the left of the operator. >>> c.__imul__ = lambda B: B.n - 6
>>> c *= d
>>> c
1
It is important to note that the augmented assignment operators will also use the normal operator functions if the augmented operator function hasn't been set directly. This will work as expected, with "__add__" being called for "+=" and so on. >>> c = FakeNumber()
>>> c += d
>>> c
12
|
|
Unary Operators
Unary operators will be passed simply the instance of the class that they are called on. >>> FakeNumber.__neg__ = lambda A : A.n + 6
>>> -d
13
|
|
Item Operators
It is also possible in Python to override the indexing and slicing operators. This allows us to use the class[i] and class[a:b] syntax on our own objects. The simplest form of item operator is __getitem__. This takes as a parameter the instance of the class, then the value of the index. >>> class FakeList:
... def __getitem__(self,index):
... return index * 2
...
>>> f = FakeList()
>>> f['a']
'aa'
We can also define a function for the syntax associated with assigning a value to an item. The parameters for this function include the value being assigned, in addition to the parameters from __getitem__ >>> class FakeList:
... def __setitem__(self,index,value):
... self.string = index + " is now " + value
...
>>> f = FakeList()
>>> f['a'] = 'gone'
>>> f.string
'a is now gone'
We can do the same thing with slices. Once again, each syntax has a different parameter list associated with it. >>> class FakeList:
... def __getslice___(self,start,end):
... return str(start) + " to " + str(end)
...
>>> f = FakeList()
>>> f[1:4]
'1 to 4'
Keep in mind that one or both of the start and end parameters can be blank in slice syntax. Here, Python has default value for both the start and the end, as show below. >> f[:]
'0 to 2147483647'
Note that the default value for the end of the slice shown here is simply the largest possible signed integer on a 32-bit system, and may vary depending on your system and C compiler.
We also have operators for deleting items and slices.
Note that these are the same as __getitem__ and __getslice__. |
|
Other Overrides
|
Programming Practices
The flexibility of python classes means that classes can adopt a varied set of behaviors. For the sake of understandability, however, it's best to use many of Python's tools sparingly. Try to declare all methods in the class definition, and always use the <class>.<member> syntax instead of __dict__ whenever possible. Look at classes in C++ and Java to see what most programmers will expect from a class.
Encapsulation
Since all python members of a python class are accessible by functions/methods outside the class, there is no way to enforce encapsulation short of overriding __getattr__, __setattr__ and __delattr__. General practice, however, is for the creator of a class or module to simply trust that users will use only the intended interface and avoid limiting access to the workings of the module for the sake of users who do need to access it. When using parts of a class or module other than the intended interface, keep in mind that the those parts may change in later versions of the module, and you may even cause errors or undefined behaviors in the module, since encapsulation is private.
Doc Strings
When defining a class, it is convention to document the class using a string literal at the start of the class definition. This string will then be placed in the __doc__ attribute of the class definition.
>>> class Documented:
... """This is a docstring"""
... def explode(self):
... """
... This method is documented, too! The coder is really serious about
... making this class usable by others who don't know the code as well
... as he does.
...
... """
... print("boom")
>>> d = Documented()
>>> d.__doc__
'This is a docstring'
Docstrings are a very useful way to document your code. Even if you never write a single piece of separate documentation (and let's admit it, doing so is the lowest priority for many coders), including informative docstrings in your classes will go a long way toward making them usable.
Several tools exist for turning the docstrings in Python code into readable API documentation, e.g., EpyDoc.
Don't just stop at documenting the class definition, either. Each method in the class should have its own docstring as well. Note that the docstring for the method explode in the example class Documented above has a fairly lengthy docstring that spans several lines. Its formatting is in accordance with the style suggestions of Python's creator, Guido van Rossum in PEP 8.
Adding methods at runtime
To a class
It is fairly easy to add methods to a class at runtime. Lets assume that we have a class called Spam and a function cook. We want to be able to use the function cook on all instances of the class Spam:
class Spam:
def __init__(self):
self.myeggs = 5
def cook(self):
print("cooking %s eggs" % self.myeggs)
Spam.cook = cook #add the function to the class Spam
eggs = Spam() #NOW create a new instance of Spam
eggs.cook() #and we are ready to cook!
This will output
cooking 5 eggs
To an instance of a class
It is a bit more tricky to add methods to an instance of a class that has already been created. Lets assume again that we have a class called Spam and we have already created eggs. But then we notice that we wanted to cook those eggs, but we do not want to create a new instance but rather use the already created one:
class Spam:
def __init__(self):
self.myeggs = 5
eggs = Spam()
def cook(self):
print("cooking %s eggs" % self.myeggs)
import types
f = types.MethodType(cook, eggs, Spam)
eggs.cook = f
eggs.cook()
Now we can cook our eggs and the last statement will output:
cooking 5 eggs
Using a function
We can also write a function that will make the process of adding methods to an instance of a class easier.
def attach_method(fxn, instance, myclass):
f = types.MethodType(fxn, instance, myclass)
setattr(instance, fxn.__name__, f)
All we now need to do is call the attach_method with the arguments of the function we want to attach, the instance we want to attach it to and the class the instance is derived from. Thus our function call might look like this:
attach_method(cook, eggs, Spam)
Note that in the function add_method we cannot write instance.fxn = f since this would add a function called fxn to the instance.
External links
- 9. Classes, docs.python.org
- 2. Built-in Functions # vars, docs.python.org
Metaclasses
In Python, classes are themselves objects. Just as other objects are instances of a particular class, classes themselves are instances of a metaclass.
Python3
The Pep 3115 defines the changes to python 3 metaclasses. In python3 you have a method __prepare__ that is called in the metaclass to create a dictionary or other class to store the class members.[1] Then there is the __new__ method that is called to create new instances of that class. [2]
The type Metaclass
The metaclass for all standard Python types is the "type" object.
>>> type(object)
<type 'type'>
>>> type(int)
<type 'type'>
>>> type(list)
<type 'type'>
Just like list, int and object, "type" is itself a normal Python object, and is itself an instance of a class. In this case, it is in fact an instance of itself.
>>> type(type)
<type 'type'>
It can be instantiated to create new class objects similarly to the class factory example above by passing the name of the new class, the base classes to inherit from, and a dictionary defining the namespace to use.
For instance, the code:
>>> class MyClass(BaseClass):
... attribute = 42
Could also be written as:
>>> MyClass = type("MyClass", (BaseClass,), {'attribute' : 42})
Metaclasses
It is possible to create a class with a different metaclass than type by setting the metaclass keyword argument when defining the class. When this is done, the class, and its subclass will be created using your custom metaclass. For example
class CustomMetaclass(type):
def __init__(cls, name, bases, dct):
print("Creating class %s using CustomMetaclass" % name)
super(CustomMetaclass, cls).__init__(name, bases, dct)
class BaseClass(metaclass=CustomMetaclass):
pass
class Subclass1(BaseClass):
pass
This will print
Creating class BaseClass using CustomMetaclass Creating class Subclass1 using CustomMetaclass
By creating a custom metaclass in this way, it is possible to change how the class is constructed. This allows you to add or remove attributes and methods, register creation of classes and subclasses creation and various other manipulations when the class is created.
More resources
- Wikipedia article on Aspect Oriented Programming
- Unifying types and classes in Python 2.2
- O'Reilly Article on Python Metaclasses
References
Reflection
A Python script can find out about the type, class, attributes and methods of an object. This is referred to as reflection or introspection. See also Metaclasses.
Reflection-enabling functions include type(), isinstance(), callable(), dir() and getattr().
Type
The type method enables to find out about the type of an object. The following tests return True:
- type(3) is int
- type(3.0) is float
- type(10**10) is long # Python 2
- type(1 + 1j) is complex
- type('Hello') is str
- type([1, 2]) is list
- type([1, [2, 'Hello']]) is list
- type({'city': 'Paris'}) is dict
- type((1,2)) is tuple
- type(set()) is set
- type(frozenset()) is frozenset
- ----
- type(3).__name__ == "int"
- type('Hello').__name__ == "str"
- ----
- import types, re, Tkinter # For the following examples
- type(re) is types.ModuleType
- type(re.sub) is types.FunctionType
- type(Tkinter.Frame) is types.ClassType
- type(Tkinter.Frame).__name__ == "classobj"
- type(Tkinter.Frame()).__name__ == "instance"
- type(re.compile('myregex')).__name__ == "SRE_Pattern"
- type(type(3)) is types.TypeType
The type function disregards class inheritance: "type(3) is object" yields False while "isinstance(3, object)" yields True.
Links:
- 2. Built-in Functions # type, python.org
- 8.15. types — Names for built-in types, python.org
Isinstance
Determines whether an object is an instance of a type or class.
The following tests return True:
- isinstance(3, int)
- isinstance([1, 2], list)
- isinstance(3, object)
- isinstance([1, 2], object)
- import Tkinter; isinstance(Tkinter.Frame(), Tkinter.Frame)
- import Tkinter; Tkinter.Frame().__class__.__name__ == "Frame"
Note that isinstance provides a weaker condition than a comparison using #Type.
Function isinstance and a user-defined class:
class Plant: pass # Dummy class
class Tree(Plant): pass # Dummy class derived from Plant
tree = Tree() # A new instance of Tree class
print(isinstance(tree, Tree)) # True
print(isinstance(tree, Plant)) # True
print(isinstance(tree, object)) # True
print(type(tree) is Tree) # True
print(type(tree).__name__ == "instance") # False
print(tree.__class__.__name__ == "Tree") # True
Links:
- Built-in Functions # isinstance, python.org
- isinstance() considered harmful, canonical.org
Issubclass
Determines whether a class is a subclass of another class. Pertains to classes, not their instances.
class Plant: pass # Dummy class
class Tree(Plant): pass # Dummy class derived from Plant
tree = Tree() # A new instance of Tree class
print(issubclass(Tree, Plant)) # True
print(issubclass(Tree, object)) # False in Python 2
print(issubclass(int, object)) # True
print(issubclass(bool, int)) # True
print(issubclass(int, int)) # True
print(issubclass(tree, Plant)) # Error - tree is not a class
Links:
- Built-in Functions # issubclass, python.org
Duck typing
Duck typing provides an indirect means of reflection. It is a technique consisting in using an object as if it was of the requested type, while catching exceptions resulting from the object not supporting some of the features of the class or type.
Links:
- Glossary # duck-typing, python.org
Callable
For an object, determines whether it can be called. A class can be made callable by providing a __call__() method.
Examples:
- callable(2)
- Returns False. Ditto for callable("Hello") and callable([1, 2]).
- callable([1,2].pop)
- Returns True, as pop without "()" returns a function object.
- callable([1,2].pop())
- Returns False, as [1,2].pop() returns 2 rather than a function object.
Links:
- Built-in Functions # callable, python.org
Dir
Returns the list of names of attributes of an object, which includes methods. Is somewhat heuristic and possibly incomplete, as per python.org.
Examples:
- dir(3)
- dir("Hello")
- dir([1, 2])
- import re; dir(re)
- Lists names of functions and other objects available in the re module for regular expressions.
Links:
- Built-in Functions # dir, python.org
Getattr
Returns the value of an attribute of an object, given the attribute name passed as a string.
An example:
- getattr(3, "imag")
The list of attributes of an object can be obtained using #Dir.
Links:
- Built-in Functions # getattr, python.org
Keywords
A list of Python keywords can be obtained from Python:
import keyword
pykeywords = keyword.kwlist
print(keyword.iskeyword("if")) # True
print(keyword.iskeyword("True")) # False
Links:
- 32.6. keyword — Testing for Python keywords, python.org
Built-ins
A list of Python built-in objects and functions can be obtained from Python:
print(dir(__builtins__)) # Output the list
print(type(__builtins__.list)) # = <type 'type'>
print(type(__builtins__.open)) # = <type 'builtin_function_or_method'>
print(list is __builtins__.list) # True
print(open is __builtins__.open) # True
Links:
- 28.3. __builtin__ — Built-in objects, python.org
- Built-in Functions # dir, python.org
External links
- 2. Built-in Functions, docs.python.org
- How to determine the variable type in Python?, stackoverflow.com
- Differences between isinstance() and type() in python, stackoverflow.com
- W:Reflection (computer_programming)#Python, Wikipedia
- W:Type introspection#Python, Wikipedia
Regular Expression
Python includes a module for working with regular expressions on strings. For more information about writing regular expressions and syntax not specific to Python, see the regular expressions wikibook. Python's regular expression syntax is similar to Perl's
To start using regular expressions in your Python scripts, import the "re" module:
import re
Overview
Regular expression functions in Python at a glance:
import re
if re.search("l+","Hello"): print(1) # Substring match suffices
if not re.match("ell.","Hello"): print(2) # The beginning of the string has to match
if re.match(".el","Hello"): print(3)
if re.match("he..o","Hello",re.I): print(4) # Case-insensitive match
print(re.sub("l+", "l", "Hello")) # Prints "Helo"; replacement AKA substitution
print(re.sub(r"(.*)\1", r"\1", "HeyHey")) # Prints "Hey"; backreference
print(re.sub("EY", "ey", "HEy", flags=re.I))# Prints "Hey"; case-insensitive sub
print(re.sub(r"(?i)EY", r"ey", "HEy")) # Prints "Hey"; case-insensitive sub
for match in re.findall("l+.", "Hello Dolly"):
print(match) # Prints "llo" and then "lly"
for match in re.findall("e(l+.)", "Hello Dolly"):
print(match) # Prints "llo"; match picks group 1
for match in re.findall("(l+)(.)", "Hello Dolly"):
print(match[0], match[1]) # The groups end up as items in a tuple
match = re.match("(Hello|Hi) (Tom|Thom)","Hello Tom Bombadil")
if match: # Equivalent to if match is not None
print(match.group(0)) # Prints the whole match disregarding groups
print(match.group(1) + match.group(2)) # Prints "HelloTom"
Matching and searching
One of the most common uses for regular expressions is extracting a part of a string or testing for the existence of a pattern in a string. Python offers several functions to do this.
The match and search functions do mostly the same thing, except that the match function will only return a result if the pattern matches at the beginning of the string being searched, while search will find a match anywhere in the string.
>>> import re
>>> foo = re.compile(r'foo(.{,5})bar', re.I+re.S)
>>> st1 = 'Foo, Bar, Baz'
>>> st2 = '2. foo is bar'
>>> search1 = foo.search(st1)
>>> search2 = foo.search(st2)
>>> match1 = foo.match(st1)
>>> match2 = foo.match(st2)
In this example, match2 will be None
, because the string st2
does not start with the given pattern. The other 3 results will be Match objects (see below).
You can also match and search without compiling a regexp:
>>> search3 = re.search('oo.*ba', st1, re.I)
Here we use the search function of the re module, rather than of the pattern object. For most cases, its best to compile the expression first. Not all of the re module functions support the flags argument and if the expression is used more than once, compiling first is more efficient and leads to cleaner looking code.
The compiled pattern object functions also have parameters for starting and ending the search, to search in a substring of the given string. In the first example in this section, match2
returns no result because the pattern does not start at the beginning of the string, but if we do:
>>> match3 = foo.match(st2, 3)
it works, because we tell it to start searching at character number 3 in the string.
What if we want to search for multiple instances of the pattern? Then we have two options. We can use the start and end position parameters of the search and match function in a loop, getting the position to start at from the previous match object (see below) or we can use the findall and finditer functions. The findall function returns a list of matching strings, useful for simple searching. For anything slightly complex, the finditer function should be used. This returns an iterator object, that when used in a loop, yields Match objects. For example:
>>> str3 = 'foo, Bar Foo. BAR FoO: bar'
>>> foo.findall(str3)
[', ', '. ', ': ']
>>> for match in foo.finditer(str3):
... match.group(1)
...
', '
'. '
': '
If you're going to be iterating over the results of the search, using the finditer function is almost always a better choice.
Match objects
Match objects are returned by the search and match functions, and include information about the pattern match.
The group function returns a string corresponding to a capture group (part of a regexp wrapped in ()
) of the expression, or if no group number is given, the entire match.
Using the search1
variable we defined above:
>>> search1.group()
'Foo, Bar'
>>> search1.group(1)
', '
Capture groups can also be given string names using a special syntax and referred to by matchobj.group('name')
. For simple expressions this is unnecessary, but for more complex expressions it can be very useful.
You can also get the position of a match or a group in a string, using the start and end functions:
>>> search1.start()
0
>>> search1.end()
8
>>> search1.start(1)
3
>>> search1.end(1)
5
This returns the start and end locations of the entire match, and the start and end of the first (and in this case only) capture group, respectively.
Replacing
Another use for regular expressions is replacing text in a string. To do this in Python, use the sub function.
sub takes up to 3 arguments: The text to replace with, the text to replace in, and, optionally, the maximum number of substitutions to make. Unlike the matching and searching functions, sub returns a string, consisting of the given text with the substitution(s) made.
>>> import re
>>> mystring = 'This string has a q in it'
>>> pattern = re.compile(r'(a[n]? )(\w) ')
>>> newstring = pattern.sub(r"\1'\2' ", mystring)
>>> newstring
"This string has a 'q' in it"
This takes any single alphanumeric character (\w in regular expression syntax) preceded by "a" or "an" and wraps in in single quotes. The \1
and \2
in the replacement string are backreferences to the 2 capture groups in the expression; these would be group(1) and group(2) on a Match object from a search.
The subn function is similar to sub, except it returns a tuple, consisting of the result string and the number of replacements made. Using the string and expression from before:
>>> subresult = pattern.subn(r"\1'\2' ", mystring)
>>> subresult
("This string has a 'q' in it", 1)
Replacing without constructing and compiling a pattern object:
>>> result = re.sub(r"b.*d","z","abccde")
>>> result
'aze'
Splitting
The split function splits a string based on a given regular expression:
>>> import re
>>> mystring = '1. First part 2. Second part 3. Third part'
>>> re.split(r'\d\.', mystring)
['', ' First part ', ' Second part ', ' Third part']
Escaping
The escape function escapes all non-alphanumeric characters in a string. This is useful if you need to take an unknown string that may contain regexp metacharacters like (
and .
and create a regular expression from it.
>>> re.escape(r'This text (and this) must be escaped with a "\" to use in a regexp.')
'This\\ text\\ \\(and\\ this\\)\\ must\\ be\\ escaped\\ with\\ a\\ \\"\\\\\\"\\ to\\ use\\ in\\ a\\ regexp\\.'
Flags
The different flags use with regular expressions:
Abbreviation | Full name | Description |
---|---|---|
re.I |
re.IGNORECASE |
Makes the regexp case-insensitive |
re.L |
re.LOCALE |
Makes the behavior of some special sequences (\w, \W, \b, \B, \s, \S ) dependent on the current locale
|
re.M |
re.MULTILINE |
Makes the ^ and $ characters match at the beginning and end of each line, rather than just the beginning and end of the string
|
re.S |
re.DOTALL |
Makes the . character match every character including newlines.
|
re.U |
re.UNICODE |
Makes \w, \W, \b, \B, \d, \D, \s, \S dependent on Unicode character properties
|
re.X |
re.VERBOSE |
Ignores whitespace except when in a character class or preceded by an non-escaped backslash, and ignores # (except when in a character class or preceded by an non-escaped backslash) and everything after it to the end of a line, so it can be used as a comment. This allows for cleaner-looking regexps.
|
Pattern objects
If you're going to be using the same regexp more than once in a program, or if you just want to keep the regexps separated somehow, you should create a pattern object, and refer to it later when searching/replacing.
To create a pattern object, use the compile function.
import re
foo = re.compile(r'foo(.{,5})bar', re.I+re.S)
The first argument is the pattern, which matches the string "foo", followed by up to 5 of any character, then the string "bar", storing the middle characters to a group, which will be discussed later. The second, optional, argument is the flag or flags to modify the regexp's behavior. The flags themselves are simply variables referring to an integer used by the regular expression engine. In other languages, these would be constants, but Python does not have constants. Some of the regular expression functions do not support adding flags as a parameter when defining the pattern directly in the function, if you need any of the flags, it is best to use the compile function to create a pattern object.
The r
preceding the expression string indicates that it should be treated as a raw string. This should normally be used when writing regexps, so that backslashes are interpreted literally rather than having to be escaped.
External links
- re — Regular expression operations, docs.python.org
- Python Programming/RegEx, wikiversity.org
GUI Programming
There are various GUI toolkits usable from Python.
Very productive are true GUI-builders, where the programmer can arrange the GUI window and other components such as database by using the mouse only in an intuitive fashion like in Windows Delphi 2.0. Very little typing is required. For python, only Boa Constructor follows this paradigm. WXglade and Qt-designer, monkey studio etc. come somewhat near but remain incomplete.
Disadvantages with the following kits described below are:
- Difficult deployment - the apps won't run on a particular GNU-Linux installation without major additional work
- breakage - apps won't work due to bit-rot.
Tkinter
Tkinter is a Python wrapper for Tcl/Tk providing a cross-platform GUI toolkit. On Windows, it comes bundled with Python; on other operating systems, it can be installed. The set of available widgets is smaller than in some other toolkits, but since Tkinter widgets are extensible, many of the missing compound widgets can be created using the extensibility, such as combo box and scrolling pane.
A minimal example:
from Tkinter import *
root = Tk()
frame = Frame(root)
frame.pack()
label = Label(frame, text="Hey there.")
label.pack()
quitButton = Button(frame, text="Quit", command=frame.quit)
quitButton.pack()
root.mainloop()
Main chapter: Tkinter.
Links:
- 24.1. Tkinter — Python interface to Tcl/Tk, python.org
- Tkinter 8.5 reference: a GUI for Python, infohost.nmt.edu; the same as pdf
- An Introduction to Tkinter, effbot.org
PyGTK
See also book PyGTK For GUI Programming
PyGTK provides a convenient wrapper for the GTK+ library for use in Python programs, taking care of many of the boring details such as managing memory and type casting. The bare GTK+ toolkit runs on Linux, Windows, and Mac OS X (port in progress), but the more extensive features — when combined with PyORBit and gnome-python — require a GNOME install, and can be used to write full featured GNOME applications.
PyQt
PyQt is a wrapper around the cross-platform Qt C++ toolkit. It has many widgets and support classes supporting SQL, OpenGL, SVG, XML, and advanced graphics capabilities. A PyQt hello world example:
from PyQt4.QtCore import *
from PyQt4.QtGui import *
class App(QApplication):
def __init__(self, argv):
super(App, self).__init__(argv)
self.msg = QLabel("Hello, World!")
self.msg.show()
if __name__ == "__main__":
import sys
app = App(sys.argv)
sys.exit(app.exec_())
PyQt is a set of bindings for the cross-platform Qt application framework. PyQt v4 supports Qt4 and PyQt v3 supports Qt3 and earlier.
wxPython
Bindings for the cross platform toolkit wxWidgets. WxWidgets is available on Windows, Macintosh, and Unix/Linux.
import wx
class test(wx.App):
def __init__(self):
wx.App.__init__(self, redirect=False)
def OnInit(self):
frame = wx.Frame(None, -1,
"Test",
pos=(50,50), size=(100,40),
style=wx.DEFAULT_FRAME_STYLE)
button = wx.Button(frame, -1, "Hello World!", (20, 20))
self.frame = frame
self.frame.Show()
return True
if __name__ == '__main__':
app = test()
app.MainLoop()
Dabo
Dabo is a full 3-tier application framework. Its UI layer wraps wxPython, and greatly simplifies the syntax.
import dabo
dabo.ui.loadUI("wx")
class TestForm(dabo.ui.dForm):
def afterInit(self):
self.Caption = "Test"
self.Position = (50, 50)
self.Size = (100, 40)
self.btn = dabo.ui.dButton(self, Caption="Hello World",
OnHit=self.onButtonClick)
self.Sizer.append(self.btn, halign="center", border=20)
def onButtonClick(self, evt):
dabo.ui.info("Hello World!")
if __name__ == '__main__':
app = dabo.ui.dApp()
app.MainFormClass = TestForm
app.start()
pyFltk
pyFltk is a Python wrapper for the FLTK, a lightweight cross-platform GUI toolkit. It is very simple to learn and allows for compact user interfaces.
The "Hello World" example in pyFltk looks like:
from fltk import *
window = Fl_Window(100, 100, 200, 90)
button = Fl_Button(9,20,180,50)
button.label("Hello World")
window.end()
window.show()
Fl.run()
Other Toolkits
- PyKDE - Part of the kdebindings package, it provides a python wrapper for the KDE libraries.
- PyXPCOM provides a wrapper around the Mozilla XPCOM component architecture, thereby enabling the use of standalone XUL applications in Python. The XUL toolkit has traditionally been wrapped up in various other parts of XPCOM, but with the advent of libxul and XULRunner this should become more feasible. These days, nobody uses PyXPCOM for very good reasons: PyXPCOM gives one dead links and outdated incompatible firefox extensions.
External links
- Graphic User Interface FAQ, python.org
- An excerpt from Chapter 20: GUI Development, from Python Programming on Win32, onlamp.com, by Mark Hammond, Andy Robinson; covers Tkinter, PythonWin and wxPython
- Google Ngram Viewer: Tkinter, wxPython, wxWidgets, PyGTK, PyQt
Authors
Authors of Python textbook
Game Programming in Python
3D Game Programming
3D Game Engine with a Python binding
ctypes python module for Irrlicht Engine SDK. | |
PyPi Link | https://pypi.python.org/pypi/pyirrlicht |
---|---|
Pip command | pip install pyirrlicht |
Both are very good free open source C++ 3D game Engine with a Python binding.
- CrystalSpace is a free cross-platform software development kit for real-time 3D graphics, with particular focus on games. Crystal Space is accessible from Python in two ways: (1) as a Crystal Space plugin module in which C++ code can call upon Python code, and in which Python code can call upon Crystal Space; (2) as a pure Python module named ‘cspace’ which one can ‘import’ from within Python programs. To use the first option, load the ‘cspython’ plugin as you would load any other Crystal Space plugin, and interact with it via the SCF ‘iScript’ interface .The second approach allows you to write Crystal Space applications entirely in Python, without any C++ coding. CS Wiki
3D Game Engines written for Python
Engines designed for Python from scratch.
Open Source 3D creation. Free to use for any purpose, forever. | |
Download link | https://www.blender.org/download/ |
---|
- Blender is an impressive 3D tool with a fully integrated 3D graphics creation suite allowing modeling, animation, rendering, post-production, real-time interactive 3D and game creation and playback with cross-platform compatibility. The 3D game engine uses an embedded python interpreter to make 3D games.
Panda3D is a game engine, a framework for 3D rendering and game development for Python and C++ programs | |
Download link | http://www.panda3d.org/download.php |
---|
- Panda3D is a 3D game engine. It's a library written in C++ with Python bindings. Panda3D is designed in order to support a short learning curve and rapid development. This software is available for free download with source code under the BSD License. The development was started by [Disney]. Now there are many projects made with Panda3D, such as Disney's Pirate's of the Caribbean Online, ToonTown, Building Virtual World, Shell Games and many others. Panda3D supports several features: Procedural Geometry, Animated Texture, Render to texture, Track motion, fog, particle system, and many others.
Crystal Space is a mature, full-featured Software Development Kit (SDK) providing real-time 3D graphics for applications such as games and virtual reality | |
Download link | http://www.crystalspace3d.org/main/Download |
---|
- Crystal-space Is a 3D game engine, with a Python bindings, named *Crystal, view Wikipedia page of *CrystalSpace.
2D Game Programming
Python Game Development | |
PyPi Link | https://pypi.python.org/pypi/Pygame |
---|---|
Pip command | pip install Pygame |
- Pygame is a cross platform Python library which wraps SDL. It provides many features like Sprite groups and sound/image loading and easy changing of an objects position. It also provides the programmer access to key and mouse events. A full tutorial can be found in the free book "Making Games with Python & Pygame".
Python Game Utilities | |
Download link | https://code.google.com/archive/p/pgu/downloads |
---|---|
Dependencies | PyGame |
- Phil's Pygame Utilities (PGU) is a collection of tools and libraries that enhance Pygame. Tools include a tile editor and a level editor (tile, isometric, hexagonal). GUI enhancements include full featured GUI, HTML rendering, document layout, and text rendering. The libraries include a sprite and tile engine (tile, isometric, hexagonal), a state engine, a timer, and a high score system. (Beta with last update March, 2007. APIs to be deprecated and isometric and hexagonal support is currently Alpha and subject to change.) [Update 27/02/08 Author indicates he is not currently actively developing this library and anyone that is willing to develop their own scrolling isometric library offering can use the existing code in PGU to get them started.]
Cross-platform windowing and multimedia library | |
PyPi Link | https://pypi.python.org/pypi/pyglet |
---|---|
Pip command | pip install pyglet |
- Pyglet is a cross-platform windowing and multimedia library for Python with no external dependencies or installation requirements. Pyglet provides an object-oriented programming interface for developing games and other visually-rich applications for Windows, Mac OS X and Linux. Pyglet allows programs to open multiple windows on multiple screens, draw in those windows with OpenGL, and play back audio and video in most formats. Unlike similar libraries available, pyglet has no external dependencies (such as SDL) and is written entirely in Python. Pyglet is available under a BSD-Style license.
A software library for rapid development of hardware-accelerated multitouch applications. | |
PyPi Link | https://pypi.python.org/pypi/kivy |
---|---|
Pip command | pip install kivy |
Dependencies | docutils; pygments (auto-installed with kivy)
kivy.deps.sdl2; kivy.deps.glew (will not auto-install, run pip install kivy_examples |
- Kivy Kivy is a library for developing multi-touch applications. It is completely cross-platform (Linux/OSX/Win & Android with OpenGL ES2). It comes with native support for many multi-touch input devices, a growing library of multi-touch aware widgets and hardware accelerated OpenGL drawing. Kivy is designed to let you focus on building custom and highly interactive applications as quickly and easily as possible.
A fast 2D sprite engine using OpenGL | |
PyPi Link | https://pypi.python.org/pypi/Rabbyt |
---|---|
Pip command | pip install Rabbyt |
- Rabbyt A fast Sprite library for Python with game development in mind. With Rabbyt Anims, even old graphics cards can produce very fast animations of 2,400 or more sprites handling position, rotation, scaling, and color simultaneously.
See Also
- 10 Lessons Learned - How To Build a Game In A Week From Scratch With No Budget
Sockets
![]() |
This page or section is an undeveloped draft or outline. You can help to develop the work, or you can ask for assistance in the project room. |
HTTP Client
Make a very simple HTTP client
import socket
s = socket.socket()
s.connect(('localhost', 80))
s.send('GET / HTTP/1.1\nHost:localhost\n\n')
s.recv(40000) # receive 40000 bytes
NTP/Sockets
Connecting to and reading an NTP time server, returning the time as follows
ntpps picoseconds portion of time ntps seconds portion of time ntpms milliseconds portion of time ntpt 64-bit ntp time, seconds in upper 32-bits, picoseconds in lower 32-bits
Files
File I/O
Read entire file:
inputFileText = open("testit.txt", "r").read()
print(inputFileText)
In this case the "r" parameter means the file will be opened in read-only mode.
Read certain amount of bytes from a file:
inputFileText = open("testit.txt", "r").read(123)
print(inputFileText)
When opening a file, one starts reading at the beginning of the file, if one would want more random access to the file, it is possible to use seek()
to change the current position in a file and tell()
to get to know the current position in the file. This is illustrated in the following example:
>>> f=open("/proc/cpuinfo","r")
>>> f.tell()
0L
>>> f.read(10)
'processor\t'
>>> f.read(10)
': 0\nvendor'
>>> f.tell()
20L
>>> f.seek(10)
>>> f.tell()
10L
>>> f.read(10)
': 0\nvendor'
>>> f.close()
>>> f
<closed file '/proc/cpuinfo', mode 'r' at 0xb7d79770>
Here a file is opened, twice ten bytes are read, tell()
shows that the current offset is at position 20, now seek()
is used to go back to position 10 (the same position where the second read was started) and ten bytes are read and printed again. And when no more operations on a file are needed the close()
function is used to close the file we opened.
Read one line at a time:
for line in open("testit.txt", "r"):
print(line)
In this case readlines()
will return an array containing the individual lines of the file as array entries. Reading a single line can be done using the readline()
function which returns the current line as a string. This example will output an additional newline between the individual lines of the file, this is because one is read from the file and print introduces another newline.
Write to a file requires the second parameter of open()
to be "w", this will overwrite the existing contents of the file if it already exists when opening the file:
outputFileText = "Here's some text to save in a file"
open("testit.txt", "w").write(outputFileText)
Append to a file requires the second parameter of open()
to be "a" (from append):
outputFileText = "Here's some text to add to the existing file."
open("testit.txt", "a").write(outputFileText)
Note that this does not add a line break between the existing file content and the string to be added.
Since Python 2.5, you can use with keyword to ensure the file handle is released as soon as possible and to make it exception-safe:
with open("input.txt") as file1:
data = file1.read()
# process the data
Or one line at a time:
with open("input.txt") as file1:
for line in file1:
print(line)
Related to the with keywords is Context Managers chapter.
Links:
- 7.5. The with statement, python.org
- PEP 343 -- The "with" Statement, python.org
Testing Files
Determine whether path exists:
import os
os.path.exists('<path string>')
When working on systems such as Microsoft Windows™, the directory separators will conflict with the path string. To get around this, do the following:
import os
os.path.exists('C:\\windows\\example\\path')
A better way however is to use "raw", or r
:
import os
os.path.exists(r'C:\windows\example\path')
But there are some other convenient functions in os.path
, where os.path.exists()
only confirms whether or not path exists, there are functions which let you know if the path is a file, a directory, a mount point or a symlink. There is even a function os.path.realpath()
which reveals the true destination of a symlink:
>>> import os
>>> os.path.isfile("/")
False
>>> os.path.isfile("/proc/cpuinfo")
True
>>> os.path.isdir("/")
True
>>> os.path.isdir("/proc/cpuinfo")
False
>>> os.path.ismount("/")
True
>>> os.path.islink("/")
False
>>> os.path.islink("/vmlinuz")
True
>>> os.path.realpath("/vmlinuz")
'/boot/vmlinuz-2.6.24-21-generic'
Common File Operations
To copy or move a file, use the shutil library.
import shutil
shutil.move("originallocation.txt","newlocation.txt")
shutil.copy("original.txt","copy.txt")
To perform a recursive copy it is possible to use copytree()
, to perform a recursive remove it is possible to use rmtree()
import shutil
shutil.copytree("dir1","dir2")
shutil.rmtree("dir1")
To remove an individual file there exists the remove()
function in the os module:
import os
os.remove("file.txt")
Finding Files
Files can be found using glob:
glob.glob('*.txt') # Finds files in the current directory ending in dot txt
glob.glob('*\\*.txt') # Finds files in any of the direct subdirectories
# of the currect directory ending in dot txt
glob.glob('C:\\Windows\\*.exe')
for fileName in glob.glob('C:\\Windows\\*.exe'):
print(fileName)
glob.glob('C:\\Windows\\**.exe', recursive=True) # Py 3.5: ** allows recursive nesting
The content of a directory can be listed using listdir:
filesAndDirectories=os.listdir('.')
for item in filesAndDirectories:
if os.path.isfile(item) and item.endswith('.txt'):
print("Text file: " + item)
if os.path.isdir(item):
print("Directory: " + item)
Getting a list of all items in a directory, including the nested ones:
for root, directories, files in os.walk('/user/Joe Hoe'):
print("Root: " + root) # e.g. /user/Joe Hoe/Docs
for dir1 in directories:
print("Dir.: " + dir1) # e.g. Fin
print("Dir. 2: " + os.path.join(root, dir1)) # e.g. /user/Joe Hoe/Docs/Fin
for file1 in files:
print("File: " + file1) # e.g. MyFile.txt
print("File 2: " + os.path.join(root, file1))# e.g. /user/Joe Hoe/Docs/MyFile.txt
Above, root takes value of each directory in /user/Joe Hoe including /user/Joe Hoe itself, and directories and files are only those directly present in each root.
Getting a list of all files in a directory, including the nested ones, ending in .txt, using list comprehension:
files = [os.path.join(r, f) for r, d, fs in os.walk(".") for f in fs
if f.endswith(".txt")]
# As iterator
files = (os.path.join(r, f) for r, d, fs in os.walk(".") for f in fs
if f.endswith(".txt"))
Links:
- glob, python.org
- glob, Py 3, python.org
- os.listdir, python.org
- os.walk, python.org
- os.path.join, python.org
Current Directory
Getting current working directory:
os.getcwd()
Changing current working directory:
os.chdir('C:\\')
External Links
- os — Miscellaneous operating system interfaces in Python documentation
- glob — Unix style pathname pattern expansion in Python documentation
- shutil — High-level file operations in Python documentation
- Brief Tour of the Standard Library in The Python Tutorial
Database Programming
Python has support for working with databases via a simple API. Modules included with Python include modules for SQLite and Berkeley DB. Modules for MySQL , PostgreSQL , FirebirdSQL and others are available as third-party modules. The latter have to be downloaded and installed before use.
The package MySQLdb can be installed, for example, using the debian package "python-mysqldb".
DBMS Specifics
MySQL
An Example with MySQL would look like this:
import MySQLdb
db = MySQLdb.connect("host machine", "dbuser", "password", "dbname")
cursor = db.cursor()
query = """SELECT * FROM sampletable"""
lines = cursor.execute(query)
data = cursor.fetchall()
db.close()
On the first line, the Module MySQLdb is imported. Then a connection to the database is set up and on line 4, we save the actual SQL statement to be executed in the variable query. On line 5 we execute the query and on line 6 we fetch all the data. After the execution of this piece of code, lines contains the number of lines fetched (e.g. the number of rows in the table sampletable). The variable data contains all the actual data, e.g. the content of sampletable. In the end, the connection to the database would be closed again. If the number of lines are large, it is better to use row = cursor.fetchone() and process the rows individually:
#first 5 lines are the same as above
while True:
row = cursor.fetchone()
if row == None: break
#do something with this row of data
db.close()
Obviously, some kind of data processing has to be used on row, otherwise the data will not be stored. The result of the fetchone() command is a Tuple.
In order to make the initialization of the connection easier, a configuration file can be used:
import MySQLdb
db = MySQLdb.connect(read_default_file="~/.my.cnf")
...
Here, the file .my.cnf in the home directory contains the necessary configuration information for MySQL.
Sqlite
An example with sqlite is very similar to the one above and the cursor provides many of the same functionalities.
import sqlite3
db = sqlite3.connect("/path/to/file")
cursor = db.cursor()
query = """SELECT * FROM sampletable"""
lines = cursor.execute(query)
data = cursor.fetchall()
db.close()
When writing to the db, one has to remember to call db.commit(), otherwise the changes are not saved:
import sqlite3
db = sqlite3.connect("/path/to/file")
cursor = db.cursor()
query = """INSERT INTO sampletable (value1, value2) VALUES (1,'test')"""
cursor.execute(query)
db.commit()
db.close()
Postgres
import psycopg2
conn = psycopg2.connect("dbname=test")
cursor = conn.cursor()
cursor.execute("select * from test");
for i in cursor.next():
print(i)
conn.close()
Firebird
import firebirdsql
conn = firebirdsql.connect(dsn='localhost/3050:/var/lib/firebird/2.5/test.fdb', user='alice', password='wonderland')
cur = conn.cursor()
cur.execute("select * from baz")
for c in cur.fetchall():
print(c)
conn.close()
General Principles
Parameter Quoting
You will frequently need to substitute dynamic data into a query string. It is important to ensure this is done correctly.
# Do not do this!
result = db.execute("SELECT name FROM employees WHERE location = '" + location + "'")
This example is wrong, because it doesn’t correctly deal with special characters, like apostrophes, in the string being substituted. If your code has to deal with potentially hostile users (like on a public-facing Web server), this could leave you open to an SQL injection attack.
For simple cases, use the automatic parameter substitution provided by the execute
method, e.g.
result = db.execute("SELECT name FROM employees WHERE location = ?", [location])
The DBMS interface itself will automatically convert the values you pass into the correct SQL syntax.
For more complex cases, the DBMS module should provide a quoting function that you can explicitly call. For example, MySQLdb provides the escape_string
method, while APSW (for SQLite3) provides format_sql_value
. This is necessary where the query structure takes a more dynamic form:
criteria = [("company", company)] # list of tuples (fieldname, value)
if department != None :
criteria.append(("department", department))
# ... append other optional criteria as appropriate ...
result = db.execute(
"SELECT name FROM employees WHERE "
+
" and ".join(
"%s = %s" % (criterion[0], MySQLdb.escape_string(criterion[1]))
for criterion in criteria
)
)
This will dynamically construct queries like “select name from employees where company = 'some company'” or “select name from employees where company = 'some company' and department = 'some department'”, depending on which fields have been filled in by the user.
Use Iterators
Python iterators are a natural fit for the problem of iterating over lots of database records. Here is an example of a function that performs a database query and returns an iterator for the results, instead of returning them all at once. It relies on the fact that, in APSW (the Python 3 interface library for SQLite), the cursor.execute
method itself returns an iterator for the result records. The result is that you can write very concise code for doing complex database queries in Python.
def db_iter(db, cmd, mapfn = lambda x : x) :
"executes cmd on a new cursor from connection db and yields the results in turn."
cu = db.cursor()
result = cu.execute(cmd)
while True:
yield mapfn(next(result))
Example uses of this function:
for artist, publisher in db_iter(
db = db,
cmd =
"SELECT artist, publisher FROM artists WHERE location = %s"
%
apsw.format_sql_value(location)
):
print(artist, publisher)
and
for location in db_iter(
db = db,
cmd = "SELECT DISTINCT location FROM artists",
mapfn = lambda x : x[0]
):
print(location)
In the first example, since db_iter
returns a tuple for each record, this can be directly assigned to individual variables for the record fields. In the second example, the tuple has only one element, so a custom mapfn
is used to extract this element and return it instead of the tuple.
Never Use “SELECT *” in a Script
Database table definitions are frequently subject to change. As application requirements evolve, fields and even entire tables are often added, or sometimes removed. Consider a statement like
result = db.execute("select * from employees")
You may happen to know that the employees table currently contains, say, 4 fields. But tomorrow someone may add a fifth field. Did you remember to update your code to deal with this? If not, it’s liable to crash. Or even worse, produce an incorrect result!
Better to always list the specific fields you’re interested in, no matter how many there are:
result = db.execute("select name, address, department, location from employees")
That way, any extra fields added will simply be ignored. And if any of the named fields are removed, the code will at least fail with a runtime error, which is a good reminder that you forgot to update it!
Looping on Field Breaks
Consider the following scenario: your sales company database has a table of employees, and also a table of sales made by each employee. You want to loop over these sale entries, and produce some per-employee statistics. A naïve approach might be:
- Query the database to get a list of employees
- For each employee, do a database query to get the list of sales for each employee.
If you have a lot of employees, then the first query may produce a large list, and the second step will involve a correspondingly large number of database queries.
In fact, the entire processing loop can run off a single database query, using the standard SQL construct called a join
.
Here is what an example of such a loop could look like:
rows = db_iter \
(
db = db,
cmd =
"select employees.name, sales.amount, sales.date from"
" employees left join sales on employees.id = sales.employee_id"
" order by employees.name, sales.date"
)
prev_employee_name = None
while True:
row = next(rows, None)
if row != None :
employee_name, amount, date = row
if row == None or employee_name != prev_employee_name :
if prev_employee_name != None :
# done stats for this employee
report(prev_employee_name, employee_stats)
if row == None :
break
# start stats for a new employee
prev_employee_name = employee_name
employee_stats = {"total_sales" : 0, "number_of_sales" : 0}
if date != None :
employee_stats["earliest_sale"] = date
# another row of stats for this employee
if amount != None :
employee_stats["total_sales"] += amount
employee_stats["number_of_sales"] += 1
if date != None :
employee_stats["latest_sale"] = date
Here the statistics are quite simple: earliest and latest sale, and number and total amount of sales, and could be computed directly within the SQL query. But the same loop could compute more complex statistics (like standard deviation) that cannot be represented directly within a simple SQL query.
Note how the statistics for each employee are written out under either of two conditions:
- The employee name of the next record is different from the previous one
- The end of the query results has been reached.
Both conditions are tested with row == None or employee_name != prev_employee_name
; after writing out the employee statistics, a separate check for the second condition row == None
is used to terminate the loop. If the loop doesn’t terminate, then processing is initialized for the new employee.
Note also the use of a left join
in this case: if an employee has had no sales, then the join will return a single row for that employee, with SQL null
values (represented by None
in Python) for the fields from the sales table. This is why we need checks for such None
values before processing those fields.
Alternatively, we could have used an inner join
, which would have returned no results for an employee with no sales. Whether you want to omit such an employee from your report, or include them with totals of zero, is really up to your application.
See Also
External links
- APSW module, code.google.com — SQLite3 for Python 2.x and 3.x
- SQLite documentation
- Psycopg2 (PostgreSQL module - newer), initd.org
- PyGreSQL (PostgreSQL module - older), pygresql.org
- MySQL module, sourceforge.net
- FirebirdSQL module, github.com
[[Category:Subject:|Yasondinalt]]
Web Page Harvesting
The urllib module which is bundled with python can be used for web interaction. This module provides a file-like interface for web urls.
Getting page text as a string
An example of reading the contents of a webpage
import urllib.request as urllib
pageText = urllib.urlopen("http://www.spam.org/eggs.html").read()
print(pageText)
Processing page text line by line:
import urllib.request as urllib
for line in urllib.urlopen("https://en.wikibooks.org/wiki/Python_Programming/Internet"):
print(line)
Get and post methods can be used, too.
import urllib.request as urllib
params = urllib.urlencode({"plato":1, "socrates":10, "sophokles":4, "arkhimedes":11})
# Using GET method
pageText = urllib.urlopen("http://international-philosophy.com/greece?%s" % params).read()
print(pageText)
# Using POST method
pageText = urllib.urlopen("http://international-philosophy.com/greece", params).read()
print(pageText)
Downloading files
To save the content of a page on the internet directly to a file, you can read() it and save it as a string to a file object
import urllib2
data = urllib2.urlopen("http://upload.wikimedia.org/wikibooks/en/9/91/Python_Programming.pdf", "pythonbook.pdf").read() # not recommended as if you are downloading 1gb+ file, will store all data in ram.
file = open('Python_Programming.pdf','wb')
file.write(data)
file.close()
This will download the file from here and save it to a file "pythonbook.pdf" on your hard drive.
Other functions
The urllib module includes other functions that may be helpful when writing programs that use the internet:
>>> plain_text = "This isn't suitable for putting in a URL"
>>> print(urllib.quote(plain_text))
This%20isn%27t%20suitable%20for%20putting%20in%20a%20URL
>>> print(urllib.quote_plus(plain_text))
This+isn%27t+suitable+for+putting+in+a+URL
The urlencode function, described above converts a dictionary of key-value pairs into a query string to pass to a URL, the quote and quote_plus functions encode normal strings. The quote_plus function uses plus signs for spaces, for use in submitting data for form fields. The unquote and unquote_plus functions do the reverse, converting urlencoded text to plain text.
With Python, MIME compatible emails can be sent. This requires an installed SMTP server.
import smtplib
from email.mime.text import MIMEText
msg = MIMEText(
"""Hi there,
This is a test email message.
Greetings""")
me = 'sender@example.com'
you = 'receiver@example.com'
msg['Subject'] = 'Hello!'
msg['From'] = me
msg['To'] = you
s = smtplib.SMTP()
s.connect()
s.sendmail(me, [you], msg.as_string())
s.quit()
This sends the sample message from 'sender@example.com' to 'receiver@example.com'.
External links
- urllib.request, docs.python.org
- HOWTO Fetch Internet Resources Using The urllib Package, docs.python.org
- urllib2 for Python 2, docs.python.org
- HOWTO Fetch Internet Resources Using urllib2 — Python 2.7, docs.python.org
Threading
Threading in python is used to run multiple threads (tasks, function calls) at the same time. Note that this does not mean that they are executed on different CPUs. Python threads will NOT make your program faster if it already uses 100 % CPU time. In that case, you probably want to look into parallel programming. If you are interested in parallel programming with python, please see here.
Python threads are used in cases where the execution of a task involves some waiting. One example would be interaction with a service hosted on another computer, such as a webserver. Threading allows python to execute other code while waiting; this is easily simulated with the sleep function.
Examples
A Minimal Example with Function Call
Make a thread that prints numbers from 1-10 and waits a second between each print:
import threading
import time
def loop1_10():
for i in range(1, 11):
time.sleep(1)
print(i)
threading.Thread(target=loop1_10).start()
A Minimal Example with Object
#!/usr/bin/env python
import threading
import time
class MyThread(threading.Thread):
def run(self): # Default called function with mythread.start()
print("{} started!".format(self.getName())) # "Thread-x started!"
time.sleep(1) # Pretend to work for a second
print("{} finished!".format(self.getName())) # "Thread-x finished!"
def main():
for x in range(4): # Four times...
mythread = MyThread(name = "Thread-{}".format(x)) # ...Instantiate a thread and pass a unique ID to it
mythread.start() # ...Start the thread, run method will be invoked
time.sleep(.9) # ...Wait 0.9 seconds before starting another
if __name__ == '__main__':
main()
The output looks like this:
Thread-0 started! Thread-1 started! Thread-0 finished! Thread-2 started! Thread-1 finished! Thread-3 started! Thread-2 finished! Thread-3 finished!
Extending with C
Python modules can be written in pure Python but they can also be written in the C language. The following shows how to extend Python with C.
Using the Python/C API
A minimal example
To illustrate the mechanics, we will create a minimal extension module containing a single function that outputs "Hello" followed by the name passed in as the first parameter.
We will first create the C source code, placing it to hellomodule.c:
#include <Python.h>
static PyObject*
say_hello(PyObject* self, PyObject* args)
{
const char* name;
if (!PyArg_ParseTuple(args, "s", &name))
return NULL;
printf("Hello %s!\n", name);
Py_RETURN_NONE;
}
static PyMethodDef HelloMethods[] =
{
{"say_hello", say_hello, METH_VARARGS, "Greet somebody."},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC
inithello(void)
{
(void) Py_InitModule("hello", HelloMethods);
}
Then we will need a setup file, setup.py:
from distutils.core import setup, Extension
module1 = Extension('hello', sources = ['hellomodule.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
Then we can build the module using a procedure whose details depends on the operating system and the compiler suite.
Building with GCC for Linux
Before our module can be compiled, you must install the Python development headers if you have not already. On Debian and Debian-based systems such as Ubuntu, these can be installed with the following command:
$ sudo apt install python-dev
On openSUSE, the required package is called python-devel
and can be installed with zypper
:
$ sudo zypper install python-devel
Now that Python.h
is available, we can compile the module source code we created in the previous section as follows:
$ python setup.py build
The will compile the module to a file called hello.so in build/lib.linux-i686-x.y.
Building with GCC for Microsoft Windows
Microsoft Windows users can use MinGW to compile the extension module from the command line. Assuming gcc is in the path, you can build the extension as follows:
python setup.py build -cmingw32
The above will produce file hello.pyd, a Python Dynamic Module, similar to a DLL. The file will land in build\lib.win32-x.y.
An alternate way of building the module in Windows is to build a DLL. (This method does not need an extension module file). From cmd.exe, type:
gcc -c hellomodule.c -I/PythonXY/include gcc -shared hellomodule.o -L/PythonXY/libs -lpythonXY -o hello.dll
where XY represents the version of Python, such as "24" for version 2.4.
Building using Microsoft Visual C++
With VC8, distutils is broken. Therefore, we will use cl.exe from a command prompt instead:
cl /LD hellomodule.c /Ic:\Python24\include c:\Python24\libs\python24.lib /link/out:hello.dll
Using the extension module
Change to the subdirectory where the file hello.so resides. In an interactive Python session you can use the module as follows.
>>> import hello >>> hello.say_hello("World") Hello World!
A module for calculating Fibonacci numbers
In this section, we present a module for Fibonacci numbers, thereby expanding on the minimal example above. Compared to the minimal example, what is worth noting is the use of "i" in PyArg_ParseTuple() and Py_BuildValue().
The C source code in (fibmodule.c):
#include <Python.h>
int
_fib(int n)
{
if (n < 2)
return n;
else
return _fib(n-1) + _fib(n-2);
}
static PyObject*
fib(PyObject* self, PyObject* args)
{
int n;
if (!PyArg_ParseTuple(args, "i", &n))
return NULL;
return Py_BuildValue("i", _fib(n));
}
static PyMethodDef FibMethods[] = {
{"fib", fib, METH_VARARGS, "Calculate the Fibonacci numbers."},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC
initfib(void)
{
(void) Py_InitModule("fib", FibMethods);
}
The build script (setup.py):
from distutils.core import setup, Extension
module1 = Extension('fib', sources = ['fibmodule.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
Usage:
>>> import fib >>> fib.fib(10) 55
Using SWIG
SWIG is a tool that helps a variety of scripting and programming languages call C and C++ code. SWIG makes creation of C language modules much more straightforward.
To use SWIG, you need to get it up and running first.
You can install it on an Ubuntu system as follows:
$ sudo apt-get install swig $ sudo apt-get install python-dev
To get SWIG for Windows, you can use binaries available from the SWIG download page.
Once you have SWIG, you need to create the module source file and the module interface file:
hellomodule.c:
#include <stdio.h>
void say_hello(const char* name) {
printf("Hello %s!\n", name);
}
hello.i:
%module hello
extern void say_hello(const char* name);
Then we let SWIG do its work:
swig -python hello.i
The above produces files hello.py and hello_wrap.c.
The next step is compiling; substitute /usr/include/python2.4/ with the correct path to Python.h for your setup:
gcc -fpic -c hellomodule.c hello_wrap.c -I/usr/include/python2.4/
As the last step, we do the linking:
gcc -shared hellomodule.o hello_wrap.o -o _hello.so -lpython
The module is used as follows:
>>> import hello >>> hello.say_hello("World") Hello World!
External links
- Extending and Embedding the Python Interpreter, python.org
- Python/C API Reference Manual, python.org
- SWIG, swig.org
- Download SWIG, swig.org
Extending with C++
There are different ways to extend Python with C and C++ code:
- In plain C, using Python.h
- Using Swig
- Using Boost.Python, optionally with Py++ preprocessing
- Using pybind11
- Using Cython.
This page describes Boost.Python. Before the emergence of Cython, it was the most comfortable way of writing C++ extension modules.
Boost.Python comes bundled with the Boost C++ Libraries. To install it on an Ubuntu system, you might need to run the following commands
$ sudo apt-get install libboost-python-dev $ sudo apt-get install python-dev
A Hello World Example
The C++ source code (hellomodule.cpp)
#include <iostream>
using namespace std;
void say_hello(const char* name) {
cout << "Hello " << name << "!\n";
}
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
using namespace boost::python;
BOOST_PYTHON_MODULE(hello)
{
def("say_hello", say_hello);
}
setup.py
#!/usr/bin/env python
from distutils.core import setup
from distutils.extension import Extension
setup(name="PackageName",
ext_modules=[
Extension("hello", ["hellomodule.cpp"],
libraries = ["boost_python"])
])
Now we can build our module with
python setup.py build
The module `hello.so` will end up in e.g `build/lib.linux-i686-2.4`.
Using the extension module
Change to the subdirectory where the file `hello.so` resides. In an interactive python session you can use the module as follows.
>>> import hello >>> hello.say_hello("World") Hello World!
An example with CGAL
Some, but not all, functions of the CGAL library already have Python bindings. Here an example is provided for a case without such a binding and how it might be implemented. The example is taken from the CGAL Documentation.
// test.cpp
using namespace std;
/* PYTHON */
#include <boost/python.hpp>
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
namespace python = boost::python;
/* CGAL */
#include <CGAL/Cartesian.h>
#include <CGAL/Range_segment_tree_traits.h>
#include <CGAL/Range_tree_k.h>
typedef CGAL::Cartesian<double> K;
typedef CGAL::Range_tree_map_traits_2<K, char> Traits;
typedef CGAL::Range_tree_2<Traits> Range_tree_2_type;
typedef Traits::Key Key;
typedef Traits::Interval Interval;
Range_tree_2_type *Range_tree_2 = new Range_tree_2_type;
void create_tree() {
typedef Traits::Key Key;
typedef Traits::Interval Interval;
std::vector<Key> InputList, OutputList;
InputList.push_back(Key(K::Point_2(8,5.1), 'a'));
InputList.push_back(Key(K::Point_2(1.0,1.1), 'b'));
InputList.push_back(Key(K::Point_2(3,2.1), 'c'));
Range_tree_2->make_tree(InputList.begin(),InputList.end());
Interval win(Interval(K::Point_2(1,2.1),K::Point_2(8.1,8.2)));
std::cout << "\n Window Query:\n";
Range_tree_2->window_query(win, std::back_inserter(OutputList));
std::vector<Key>::iterator current=OutputList.begin();
while(current!=OutputList.end()){
std::cout << " " << (*current).first.x() << "," << (*current).first.y()
<< ":" << (*current).second << std::endl;
current++;
}
std::cout << "\n Done\n";
}
void initcreate_tree() {;}
using namespace boost::python;
BOOST_PYTHON_MODULE(test)
{
def("create_tree", create_tree, "");
}
// setup.py
#!/usr/bin/env python
from distutils.core import setup
from distutils.extension import Extension
setup(name="PackageName",
ext_modules=[
Extension("test", ["test.cpp"],
libraries = ["boost_python"])
])
We then compile and run the module as follows:
$ python setup.py build $ cd build/lib* $ python >>> import test >>> test.create_tree() Window Query: 3,2.1:c 8,5.1:a Done >>>
Handling Python objects and errors
One can also handle more complex data, e.g. Python objects like lists. The attributes are accessed with the extract function executed on the objects "attr" function output. We can also throw errors by telling the library that an error has occurred and returning. In the following case, we have written a C++ function called "afunction" which we want to call. The function takes an integer N and a vector of length N as input, we have to convert the python list to a vector of strings before calling the function.
#include <vector>
using namespace std;
void _afunction_wrapper(int N, boost::python::list mapping) {
int mapping_length = boost::python::extract<int>(mapping.attr("__len__")());
//Do Error checking, the mapping needs to be at least as long as N
if (mapping_length < N) {
PyErr_SetString(PyExc_ValueError,
"The string mapping must be at least of length N");
boost::python::throw_error_already_set();
return;
}
vector<string> mystrings(mapping_length);
for (int i=0; i<mapping_length; i++) {
mystrings[i] = boost::python::extract<char const *>(mapping[i]);
}
//now call our C++ function
_afunction(N, mystrings);
}
using namespace boost::python;
BOOST_PYTHON_MODULE(c_afunction)
{
def("afunction", _afunction_wrapper);
}
External links
- Boost.Python, boost.org
- pybind11, pypi.org
Extending with ctypes
ctypes[3] is a foreign function interface module for Python (included with Python 2.5 and above), which allows you to load in dynamic libraries and call C functions. This is not technically extending Python, but it serves one of the primary reasons for extending Python: to interface with external C code.
Basics
A library is loaded using the ctypes.CDLL
function. After you load the library, the functions inside the library are already usable as regular Python calls. For example, if we wanted to forego the standard Python print statement and use the standard C library function, printf
, you would use this:
from ctypes import *
libName = 'libc.so' # If you're on a UNIX-based system
libName = 'msvcrt.dll' # If you're on Windows
libc = CDLL(libName)
libc.printf("Hello, World!\n")
Of course, you must use the libName line that matches your operating system, and delete the other. If all goes well, you should see the infamous Hello World string at your console.
Getting Return Values
ctypes assumes, by default, that any given function's return type is a signed integer of native size. Sometimes you don't want the function to return anything, and other times, you want the function to return other types. Every ctypes function has an attribute called restype
. When you assign a ctypes class to restype
, it automatically casts the function's return value to that type.
Common Types
ctypes name | C type | Python type | Notes |
---|---|---|---|
None | void | None | the None object |
c_bool | C99 _Bool | bool | |
c_byte | signed char | int | |
c_char | signed char | str | length of one |
c_char_p | char * | str | |
c_double | double | float | |
c_float | float | float | |
c_int | signed int | int | |
c_long | signed long | long | |
c_longlong | signed long long | long | |
c_short | signed short | long | |
c_ubyte | unsigned char | int | |
c_uint | unsigned int | int | |
c_ulong | unsigned long | long | |
c_ulonglong | unsigned long long | long | |
c_ushort | unsigned short | int | |
c_void_p | void * | int | |
c_wchar | wchar_t | unicode | length of one |
c_wchar_p | wchar_t * | unicode |
WSGI web programming
![]() |
This page or section is an undeveloped draft or outline. You can help to develop the work, or you can ask for assistance in the project room. |
WSGI Web Programming
External Resources
http://docs.python.org/library/wsgiref.html
References
Language reference
The latest documentation for the standard python libraries and modules can always be found at The Python.org documents section
License
GNU Free Documentation License
![]() | As of July 15, 2009 Wikibooks has moved to a dual-licensing system that supersedes the previous GFDL only licensing. In short, this means that text licensed under the GFDL only can no longer be imported to Wikibooks, retroactive to 1 November 2008. Additionally, Wikibooks text might or might not now be exportable under the GFDL depending on whether or not any content was added and not removed since July 15. |
Version 1.3, 3 November 2008 Copyright (C) 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
0. PREAMBLE
The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.
This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.
We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.
1. APPLICABILITY AND DEFINITIONS
This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.
A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.
A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.
The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.
The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.
A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque".
Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.
The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.
The "publisher" means any person or entity that distributes copies of the Document to the public.
A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition.
The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.
2. VERBATIM COPYING
You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.
You may also lend copies, under the same conditions stated above, and you may publicly display copies.
3. COPYING IN QUANTITY
If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.
If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.
If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.
It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.
4. MODIFICATIONS
You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:
- Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.
- List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.
- State on the Title page the name of the publisher of the Modified Version, as the publisher.
- Preserve all the copyright notices of the Document.
- Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.
- Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.
- Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.
- Include an unaltered copy of this License.
- Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.
- Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.
- For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.
- Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.
- Delete any section Entitled "Endorsements". Such a section may not be included in the Modified version.
- Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section.
- Preserve any Warranty Disclaimers.
If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.
You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties—for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.
You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.
The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.
5. COMBINING DOCUMENTS
You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.
The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.
In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements".
6. COLLECTIONS OF DOCUMENTS
You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.
You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.
7. AGGREGATION WITH INDEPENDENT WORKS
A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.
If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.
8. TRANSLATION
Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.
If a section in the Document is Entitled "Acknowledgements", "Dedications", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.
9. TERMINATION
You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.
10. FUTURE REVISIONS OF THIS LICENSE
The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.
Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Document.
11. RELICENSING
"Massive Multiauthor Collaboration Site" (or "MMC Site") means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A "Massive Multiauthor Collaboration" (or "MMC") contained in the site means any set of copyrightable works thus published on the MMC site.
"CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.
"Incorporate" means to publish or republish a Document, in whole or in part, as part of another Document.
An MMC is "eligible for relicensing" if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.
The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.
How to use this License for your documents
To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:
- Copyright (c) YEAR YOUR NAME.
- Permission is granted to copy, distribute and/or modify this document
- under the terms of the GNU Free Documentation License, Version 1.3
- or any later version published by the Free Software Foundation;
- with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
- A copy of the license is included in the section entitled "GNU
- Free Documentation License".
If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the "with...Texts." line with this:
- with the Invariant Sections being LIST THEIR TITLES, with the
- Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.
If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.
If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.
External links
- Python books available for free download
- Non-programmers python tutorial donated to this project. Wiki version
- Dive into Python
- How to think Like a Computer Scientist: Learning with Python
- A Byte of Python
- ActiveState Python Cookbook
- Text Processing in Python
- Dev Shed's Python Tutorials
- MakeBot - Simple Python IDE designed for teaching game programming to kids.
- SPE - Stani's Python Editor