Debugging Tips

Let There Be Python: Downloading and Installing

Tip

Debugging Windows Installation

The only problem you are likely to encounter doing a Windows installation is a lack of administrative privileges on your computer. In this case, you will need help from your support department to either do the installation for you, or give you administrative privileges.

Instant Gratification : The Simplest Possible Conversation

Tip

Debugging Windows Command Prompt

In the unlikely event that you can’t use Python from the Command Prompt, you have an issue with your Windows “path”. Your path tells the Command Prompt where to find the various commands. The word python becomes a command when the python.exe file is on the system’s path.

Generally, you should reinstall Python to give the Python installer a chance to set the path correctly for you. If, for some reason, that doesn’t work, here’s how you can set the system path in Windows.

Setting the Windows Path

  1. Open the Control Panel.

    Use the Start menu, Settings sub menu to locate your Control Panel.

  2. Open the System Control Panel

    Double-click the System Control Panel. This opens the System Properties panel.

  3. Open the Advanced Tab of the System Control Panel

    Click the Advanced tab on the System Control Panel.

    There are three areas: Performance, Environment Variables and Startup and Recovery. We’ll be setting the environment variables.

  4. Open the Environment Variables of the Advanced Tab of the System Control Panel

    Click the Environment Variables... button.

    This dialog box has a title of Environment Variables. It shows two areas: user variables and System variables. We’ll be updating one of the system variables.

  5. Edit the Path variable

    This dialog box has a title of Environment Variables. Scroll through the list of System variables, looking for Path. Click on the Path to highlight it.

    Click the Edit... button.

    This dialog box has a title of Edit System Variable. It has two sections to show the variable name of Path and the variable value.

  6. Add Python’s location to the Path value

    This dialog box has a title of Edit System Variable. It has two sections to show the variable name of Path and the variable value.

    Click on the value and use the right arrow key to scroll through the value you find. At the end, add the following ;C:\python26. Don’t forget the ; to separate this search location from other search locations on the path.

    Click OK to save this change. It is now a permanent part of your Windows setup on this computer. You’ll never have to change this again.

  7. Finish Changing Your System Properties

    The current dialog box has a title of Environment Variables. Click OK to save your changes.

    The current dialog box has a title of System Properties. Click OK to save your changes.

Tip

Debugging Typing a Python Statement

When you see the ... prompt from Python, it means that your statement is incomplete. Are you missing a ) to make the () pairings complete? Did you accidentally use the \? Hit Return twice and you’ll get a nice syntax error and you’ll be back at the >>> where you can try again.

Also, you’ll get unexpected errors if you try to use [], and {} the way mathematicians do. Python only uses () to group expressions. If you try to use [], you’ll get a TypeError: unsupported operand type(s) for [] : 'list' and 'int'. If you try to use {}, you get a SyntaxError: invalid syntax.

IDLE Time : Using Tools To Be More Productive

Tip

Debugging A Script

If your idle script file doesn’t work, there are some common things to confirm:

  • Your file is in the same directory that the Terminal starts in. If you are unsure, you can use the pwd to print the working directory. In my case it is /home/slott. That’s where I put my idle startup file.
  • Your file is plain text. A word processor won’t save files as plain text automatically, so you should use something like gedit to assure that you’re creating a plain text file.

Tip

Debugging An Alias

If your alias doesn’t work, there are some common things to confirm:

  • Your .profile works correctly. You can type sh -v .profile or bash -v .bash_profile to test it. If you see error messages, likely you missed an apostrophe or messed up the spaces.

Simple Arithmetic : Numbers and Operators

Tip

Debugging Octal Numbers (Leading Zero Alert)

A number that begins with a zero is supposed to be in base 8. If you are copying numbers from another source, and that other uses leading zeros, you may be surprised by what Python does. If the number has digits of 8 or 9, it’s illegal. Otherwise, the number isn’t decimal.

I spent a few hours debugging a program where I had done exactly this. I was converting a very ancient piece of software, and some of the numbers had zeroed slapped on the front to make them all line up nicely. I typed them into Python without thinking that the leading zero meant it was really base 8 not base 10.

Better Arithmetic Through Functions

Tip

Debugging Function Expressions

If you look back at Syntax Rule 6, you’ll note that the ()s need to be complete. If you accidentally type something like round(2.45 with no closing ), you’ll see the following kind of exchange.

>>> round(2.45
...
...
... )
2.0

The ... is Python’s hint that the statement is incomplete. You’ll need to finish the ()s so that the statement is complete.

Extra Functions: math and random

Tip

Debugging Math Functions

The most common problem when using math functions is leaving off the math to qualify the various functions imported from this module. If you get a “NameError: name 'cos' is not defined” error message, it means you haven’t included the math qualifier.

The next most common problem is forgetting to import math each time you run IDLE or Python. You’ll get a “NameError: name 'math' is not defined” error if you forgot to import math.

The other common problem is failing to convert angles from degrees to radians.

Special Ops : Binary Data and Operators

Tip

Debugging Special Operators

The most common problems with the bit-fiddling operators is confusion about the relative priority of the operations. For conventional arithmetic operators, ** is the highest priority, * and / are lower priority and + and - are the lowest priority. However, among &, ^ and |, << and >> it isn’t obvious what the priorities are or should be.

When in doubt, add parenthesis to force the order you want.

Peeking Under the Hood

Tip

Debugging the from __future__ statement

There are two common spelling mistakes: omitting the double underscore from before and after __future__, and misspelling division.

  • If you get ImportError: No module named _future_, you misspelled __future__.
  • If you get SyntaxError: future feature :replaceable:`divinizing is not defined`, you misspelled division.

Seeing Results : The print Statement

Tip

Debugging the print Statement

One obvious mistake you will make is misspelling print. You’ll see NameError: name 'primpt' is not defined as the error message. I’ve spelled it “primpt” so often, I’ve been tempted to rewrite the Python language to add this as an alternative.

The other common mistake that is less obvious is omitting a comma between the values you are printing. When you do this, you’ll see a SyntaxError: invalid syntax message.

If the result of a print statement doesn’t look right, remember that you can always enter the various expressions directly into IDLEs Python shell to examine the processing one step at a time.

Expressions, Constants and Variables

Tip

Debugging the Assignment Statement

There are two common mistakes in the assignment statement. The first is to choose an illegal variable name. If you get a SyntaxError: can't assign to literal or SyntaxError: invalid syntax, the most likely cause is an illegal variable name.

The other mistake is to have an invalid expression on the right side of the =. If the result of an assignment statement doesn’t look right, remember that you can always enter the various expressions directly into IDLE‘s Python Shell window to examine the processing one step at a time.

Tip

Debugging the Augmented Assignment Statement

There are two common mistakes in the augmented assignment statement. The first is to choose an illegal variable name. If you get a SyntaxError: can't assign to literal or SyntaxError: invalid syntax the most likely cause is an illegal variable name.

The other mistake is to have an invalid expression on the right side of the assignment operator. If the result of an assignment statement doesn’t look right, remember that you can always enter the various expressions directly into IDLE‘s Python shell to examine the processing one step at a time.

Assignment Bonus Features

Tip

Debugging Multiple Assignment Statements

There are three common mistakes in the augmented assignment statement. The first is to choose an illegal variable name. If you get a SyntaxError: can't assign to literal or SyntaxError: invalid syntax the most likely cause is an illegal variable name.

One other mistake is to have an invalid expression on the right side of the assignment operator. If the result of an assignment statement doesn’t look right, remember that you can always enter the various expressions directly into IDLE‘s Python shell to examine the processing one step at a time.

The third mistake is to have a mismatch between the number of variables on the left side of the = and the number of expressions on the right side.

Can We Get Your Input?

Tip

Debugging the raw_input() Function

There are two kinds of mistakes that occur. The first kind of mistake are basic syntax errors in the raw_input() function call itself.

The other mistake, which is more difficult to prevent, is to provide invalid input to the script when it runs. Currently, we don’t quite have all of the language facilities necessary to recover from improper input values. When we cover the try statement and exception processing in The Unexpected : The try and except statements we’ll see how we can handle invalid input.

Tip

Debugging the del statement

If we misspell a variable name, or attempt to delete a variable that doesn’t exist, we’ll get an error like NameError: name 'hack' is not defined.

Truth and Logic : Boolean Data and Operators

Tip

Debugging Logic Operators

The most common problem people have with the logic operators is to mistake the priority rules. The lowest priority operator is or. and is higher priority and not is the highest priority. If there is any confusion, extra parentheses will help.

Making Decisions : The Comparison Operators

Tip

Debugging Comparison Operators

The most common problem people have with the comparison operators is to attempt to compare things which cannot meaningfully be compared. They ask, in essence, “which is larger, the Empire State Building or the color green?” An expression like 123 < 'a' doesn’t really make a lot of sense, even though it is legal Python.

Python copes with senseless comparisons using it’s coercion rules. In this case, the number 123 is coerced to a string '123', and the two strings are compared using the ordinary alphabetical order rules. In this case, digits come before letters and any number will be less than any word.

Sorting out the rules for coercion and comparison can be very confusing. Consequently, it should be avoided by exercising reasonable care in writing sensible programs. You should inspect your program to be sure you are comparing things sensibly. You can also put in explicit conversions using the various factory functions in Functions are Factories (really!).

Processing Only When Necessary : The if Statement

Tip

Debugging the if statement.

If you are typing an if statement, and you get a SyntaxError: invalid syntax, you omitted the :.

A common problem with if statements is an improper condition. You can put any expression in the if or elif statement. If the expression doesn’t have a boolean value, Python will use the bool() function to determine if the expression amounts to True or False. It’s far better to have a clear boolean expression rather than trust the rules used by the bool() function.

One of the more subtle problems with the if statement is being absolutely sure of the implicit condition that controls the else clause. By relying on an implicit condition, it is easy to overlook gaps in your logic.

Consider the following complete if statement that checks for a winner on a field bet. A field bet wins on 2, 3, 4, 9, 10, 11 or 12. The payout odds are different on 2 and 12.

outcome= 0
if d1+d2 == 2 or d1+d2 == 12:
    outcome= 2
    print("field pays 2:1")
elif d1+d2==4 or d1+d2==9 or d1+d2==10 or d1+d2==11:
    outcome= 1
    print("field pays even money")
else:
    outcome= -1
    print("field loses")

Here’s the subtle bug in this example. We test for 2 and 12 in the first clause; we test for 4, 9, 10 and 11 in the second. It’s not obvious that a roll of 3 is missing from the “field pays even money” condition. This fragment incorrectly treats 3, 5, 6, 7 and 8 alike in the else:.

While the else: clause is used commonly as a catch-all, a more proper use for else: is to raise an exception because a condition was found that did not match by any of the if or elif clauses.

While We Have More To Do : The for and while Statements

Tip

Debugging the for Statement

If you are typing a for statement, and you get a SyntaxError: invalid syntax, you omitted the :.

The most common problem is setting up the sequence properly. Very often, this is because of the complex rules for the range() function, and we have one too many or one too few values.

A less common problem is to misspell the variable in the for statement or the suite. If the variable names don’t match, the for statement will set a variable not used properly by the suite. An error like NameError: name 'j' is not defined means that your suite suite expected j, but that was not the variable on your for statement.

Another problem that we can’t really address completely is writing a for statement where the suite doesn’t do the right thing in the first place. In this case, it helps to be sure that the suite works in the first place. An execution trace (see Where Exactly Did We Expect To Be?) can help. Also, you can enter the statements from the suite separately to the Python shell to see what they do.

Tip

Debugging the while Statement

If you are typing a while statement, and you get a SyntaxError: invalid syntax, you omitted the :.

There are several problems that can be caused by an incorrectly designed while statement.

The while loop never stops! The first time you see this happen, you’ll probably shut off your computer. There’s no need to panic however, there are some better things to do when your computer appears “hung” and doesn’t do anything useful.

When your loop doesn’t terminate, you can use Ctrl-C to break out of the loop and regain control of your computer. Once you’re back at the >>> you can determine what was wrong with your loop. In the case of a loop that doesn’t terminate, the while expression is always True. There are two culprits.

  • You didn’t initialize the variables properly. The while expression must eventually become False for the loop to work. If your initialization isn’t correct, you may have created a situation where it will never become False.
  • You didn’t change the variables properly during the loop. If the variables in the while expression don’t change values, then the expression will never change, and the loop will either never iterate or it will never stop iterating.

If your loop never operates at all, then the while expression is always False. This means that your initialization isn’t right. A few print statements can show the values of your variables so you can see precisely what is going wrong.

One rare situation is a loop that isn’t supposed to operate. For example, if we are computing the average of 100 dice rolls, we’ll iterate 100 times. Sometimes, however, we have the “degenerate case”, where we are trying to average zero dice rolls. In this case, the while expression may start out False for a good reason. We can get into trouble with this if some of the other variables are not be set properly. This can happen when you’ve made the mistake of creating a new variable inside the loop body. To be sure that a loop is designed correctly, all variables should be initialized correctly, and no new variables should be created within the loop body; they should only be updated.

If your loop is inconsistent – it works for some input values, but doesn’t work for others – then the body of the loop is the source of the inconsistency. Every if statement alternative in the suite of statements within the loop has establish a consistent state at the end of the suite of statements.

Loop construction can be a difficult design problem. It’s easier to design the loop properly than to debug a loop which isn’t working. We’ll cover this in A Digression On Design.

Becoming More Controlling

Tip

Debugging the break Statement

The most common problem with the break statement is an incorrect condition on the surrounding if statement, or an incorrect condition on the while statement. Designing these repetitive statements can be a relatively difficult problem, so we have some additional material on just that subject in A Digression On Design.

The most important debugging tool is the print() function. You can print the values of relevant variables in various positions in your loop body to be sure that things work the way you expect.

Tip

Debugging the continue Statement

The most common problem with the continue statement is an incorrect condition on the surrounding if statement, or an incorrect condition on the while statement. Designing these repetitive statements can be a relatively difficult problem, so we have some additional material on just that subject in A Digression On Design.

Tip

Debugging the assert Statement

The assert statement is an important tool for debugging other problems in your program. It is rare to have a problem with the assert statement itself. The only thing you have to provide is the condition which must be true. If you can’t formulate the condition in the first place, it means you may have a larger problem in describing what is supposed to be happening in the program in general. If so, it helps to take a step back from Python and try to write an English-language description of what the program does and how it works.

Clear assert statements show a tidy, complete, trustworthy, reliable, clean, honest, thrifty program. Seriously. If you can make a clear statement of what must be true, then you have a very tight grip on what should be happening and how to prove that it really is happening. This is the very heart of programming: translating the program’s purpose into a condition, creating the statements that make the conditions true, and being able to back this design up with a proof and a formal assertion.

Turning Python Loose with More Sophisticated Scripts

Tip

Debugging Explicit Python Scripts

There are two things which can go awry with a script: the operating system can’t find Python or the operating system can’t find your script file. Here’s a debugging procedure which may help.

  1. Start with the simplest possible operating system command: python. If this doesn’t work, Python isn’t installed correctly: reinstall Python. If this does work, then the shell or command prompt can find Python, and that is not the problem.
  2. Use your operating system’s directory and file commands (Windows: CD and DIR; GNU/Linux or MacOS: pwd and ls) to find your script file. If your file is not in the current directory, then you can either change directories, or include the directory name in your file.
    • Change your current working directory to the correct location of your files. For Windows: use CD; for GNU/Linux and MacOS: use cd. For example, if your files are in an exercises directory, you can do cd exercises.
    • Include the directory name on your file. For example, if your files are in an exercises directory, you can run the script1.py script with python exercises/script1.py.
  3. If you can find Python, and you appear to be in the correct directory, the remaining problem is misspelling the filename for your script. This is relatively common, actually. First time GNU/Linux and MacOS users will find that the shell is sensitive to the case of the letters, that some letters look alike, it is possible to embed non-printing characters in a file name, and it is unwise to use letters which confuse the shell. We have the following advice.
    • File names in GNU/Linux should be one word, all lower case letters and digits. These are the standard Python expectations for module names. While there are ways around this by using the shells quoting and escaping rules, Python programs avoid this.
    • File names should avoid punctuation marks. There are only a few safe punctuation marks: -, . and _. Even these safe characters should not be the first character of the file name.
    • Some Windows programs will tack an extra .txt on your file. You may have to manually rename the file to get rid of this.
    • In GNU/Linux, you can sometimes embed a space or non-printing character in a file name. To find this, use the ls -s to see the non-printing characters. You’ll have to resort to fairly complex shell tricks to rename a badly named file to something more useful. The % character is a wild-card which matches any single character. If you have a file named script^M1.py, you can rename this with mv script%1.py script1.py. The % will match he unprintable ^M in the file name.

Tip

Debugging Implicit Python Scripts

If an implicit script doesn’t work, you have two problems to resolve. First, would it work as an explicit command? If not, then fix that before doing anything else. Say you have a script implicit.py that won’t run. The first thing to test is python implicit.py. If this doesn’t work, see Let Python Run It.

The most common problem with implicit scripts is the hidden hand-shake between the shell and the first line of the script file. Most GNU/Linux variants will use #!/usr/bin/env python.

You may not have the env program in your UNIX. Try typing just env at the shell prompt. If this returns an error, you will need to know where Python installed. Enter which python. The shell will search its path for Python and report the directory in which it was found. This might be /usr/local/bin/python. Use this in the #! line. For example, #!/usr/local/bin/python.

Adding New Verbs : The def Statement

Tip

Debugging Function Definition

There are three areas for mistakes in function definition: the def statement itself, the return statement and using the function in an expression.

The syntax of the def statement contains three parts. If you have syntax errors on the definition, you’ve got one of these three wrong, or you’re misspelling “def”.

  • The name, which is a Python name, following the name rules in Python Name Rules.
  • The parameters, which is list of names, separated by commas. The ()s around the parameter list is required, even if there are no parameters. The , to separate parameters is required.
  • The indented suite, a block of Python statements.

The return statement is how the return value is defined. If you omit this, your function always returns None. The return statement also ends execution of the function’s body; if you have this statement out of place, your function may not fully execute.

When you use the function, you have to pass actual argument values for each parameter variable. The matching is done by position: the first argument value is assigned to the first positional parameter.

Tip

Direct Python and Help()

When executing help() while using Python directly (not using IDLE), you’ll be interacting with a help viewer that allows you to scroll forward and back through the text.

For more information on the help viewer, see Getting Help.

On the Mac OS or GNU/Linux, you’ll see an (END) prompt telling you that you’ve reached the the document; hit q to exit from viewing help.

Tip

Debugging Optional Parameters

There are three areas for the common mistakes in function definition: the def statement itself, the return statement and using the function in an expression.

The syntax of the def statement contains three parts. If you have syntax errors on the definition, you’ve got one of these three wrong, or you’re misspelling “def”.

  • The name, which is a Python name, following the name rules in Python Name Rules.
  • The parameters, which is list of names, separated by commas. The ()s around the parameter list is required, even if there are no parameters. The , to separate parameters is required. If we use initializers, the = to separate the variable and the initial value is required punctuation. Also note that the initializer is evaluated when the def statement is executed, not when the function is evaluated.
  • The indented suite, a block of Python statements.

In our syntax summaries, we use ... to show things that can be repeated, we don’t actually enter the ....

The required parameters (which have no initial values) must precede the optional parameters (which have initial values).

The return statement is how the return value is defined. If you omit this, your function always returns None. The return statement also ends execution of the function’s body; if you have this statement out of place, your function may not fully execute.

When you use the function, you have to provide actual argument values for each parameter that doesn’t have an initializer. Since positional parameters are simply matched up in order, the arguments you present when you use of the function must match parameters of the definition.

Tip

Debugging Keyword Parameters

When you use a function, you have to provide actual argument values for each parameter that doesn’t have an initializer. Two things can go wrong here: the syntax of the function call is incorrect in the first place, or you haven’t provided values to all parameters.

You may have fundamental syntax errors, including mis-matched (), or a misspelled function name.

You can provide argument values by position or by using the parameter name or a mixture of both techniques. Python will first extract all of the keyword arguments and set the parameter values. After that, it will match up positional parameters in order. Finally, default values will be applied. There are several circumstances where things can go wrong.

  • A parameter is not set by keyword, position or default value
  • There are too many positional values.
  • A keyword is used that is not a parameter name in the function definition.

Common List Design Patterns

Tip

Debugging List Comprehensions

List comprehensions have rather complex syntax. There are a number of ways to get SyntaxError messages. The expression, the for-clause and the overall structure of the expression, including balancing the [] are all sources of problems. Debugging a list comprehension is most easily done by building up the list comprehension one clause at a time.

A simple comprehension has two clauses: the expression clause and the for-clause. You can try each part out in IDLE individually.

  • If the for-clause is wrong, the original sequence will not be correct.
  • If the expression clause is not correct, the resulting sequence will be incorrect.

The for-clause of a list comprehension can be seen by entering just the for-clause as a separate statement. The expression clause can be evaluated for specific values to be sure that it works correctly.

For example, [ 2*i+1 for i in range(5) ] can be debugged in two parts. First, assure that range(5) produces the source sequence you expected. Second, assure that 2*i+1 works for values of i from 0 to 4.

Tip

Debugging List Sorting

There are three kinds of problems that can prevent a customized sort operation from working correctly.

  • Our key function doesn’t have the right form. It must be a function that extracts the key from an item of the sequence being sorted.

    def key( item ):
        return something based on the item
    
  • The data in your list isn’t regular enough to be sorted. For example, if we have dates that are represented as strings like '1/10/56', '11/19/85', '3/8/87', these strings are irregular and won’t sort very nicely. As humans, we know that they should be sorted into year-month-date order, but the strings that Python sees begin with '1/', '11' and '3/', with an alphabetic order that may not be what you expected.

    To get this data into a usable form, we have to normalize it. Normalizing is a computer science term for getting data into a regular, consistent, usable form. In our example of sorting dates, we’ll need to use the time or datetime modules to parse these strings into proper Python objects that can be compared.

The Unexpected : The try and except statements

Tip

Debugging Exception Design

The try statement has at least two suites: the try suite and at least one except suite. Each of these can have :s missing, or be indented incorrectly. Since these are large, composite statements, there are a lot of places where problems can occur.

One other problem is that we may have put the wrong statements in the try suite. If we evaluate a statement that raises an exception, but that statement is not in a try suite, the exception won’t get handled. If our try statement doesn’t seem to catch the exception, one possibility is that we didn’t enclose correct statement in the try statement.

Since Python reports the line number where the exception was raised, we can see where the exception originated and adjust the location of the try or except clauses to include the proper statements.

Another problem is that the exception is raised and the exceptions on our except statements don’t match. We’ll address this in Debugging Exception Handling.

Including too many statements in the try suite is just as bad as having too few statements. Including statements which cannot raise an exception in the first place can lead to confusion when reading the program. When we look at a program we wrote two weeks ago, we don’t want to struggle to understand what it means. We’d like to be reasonably clear. To this end, a try suite should be as small as possible to handle the exception.

Second, we may be raising an exception for the wrong reason. Since a raise statement is always associated with an if, elif or else suites, the conditions on the if statement define the exceptional condition. We should be able to clearly articulate the condition that leads to the raise statement. Problems in the if statement will surface as errors in exception processing.

Tip

Debugging Exception Handling

First, we may have the wrong exceptions named in the except clauses. If we evaluate a statement that raises an exception, but that exception is not named in an except clause, the exception won’t get handled.

Since Python reports the name of the exception, we can use this information to add another except clause, or add the exception name to an existing except clause. We have to be sure we understand why we’re getting the exception and we have to be sure that our handler is doing something useful. Exceptions like RuntimeError, for example, shouldn’t be handled: they indicate that something is corrupt in our Python installation.

You won’t know you spelled an exception name wrong until an exception is actually raised and the except clauses are matched against the exception. The except clauses are merely potential statements. Once an exception is raised, they are actually evaluated, and any misspelled exception names will cause problems.

Second, we may be raising the wrong exception. If we attempt to raise an exception, but spelled the exception’s name wrong, we’ll get a strange-looking NameError, not the exception we expected.

As with the except clause, the exception name in a raise clause is not examined until the exceptional condition occurs and the raise statement is executed. Since raise statements almost always occur inside if, elif or else suites, the condition has to be met before the raise statement is executed.

Looping Back : Iterators, the for statement, and the yield statement

Tip

Debugging Iterators

There are several common problems with using an explicit iterator.

  • Skipping items without processing them.
  • Processing the same item twice
  • Getting a StopIteration exception raised when trying to skip the first item.

Generally, the best way to debug a generator is to use it in a very simple iteration statement that prints the result of the iteration. Printing the items will show us precisely what is happening. We can always change the print statement into a comment, but putting a # in front of print.

Here’s a good design pattern for skipping the first item in a sequence.

i = iter( someSequence )
next(i)         Skips an item on purpose
while True:
    a= next(i)
    some processing
    print(a)

Skipping items happens when we ask for the next() method of the iterator one too many times.

Processing an item twice happens when we forget to ask for the next() method of the iterator. We see it happen when a program picks off the header items, but fails to advance to the next item before processing the body.

Another common problem is getting a StopIteration exception raised when trying to skip the header item from a list or the header line from a file. In this case, the file or list was empty, and there was no header. Often, our programs need the following kind of try block to handle an empty file gracefully.

i = iter( someSequence )
try:
    next(i)  Skips an item on purpse
except StopIteration:
    No Items – this is a valid situation, not an error

Tip

Debugging Generator Functions

One of the most common problems people have with writing a generator is omitting the yield statement. Without the yield statement, you have written a simple function: the for statement can’t initialize it and get individual values from it.

If you get a TypeError: iteration over non-sequence error, you have omitted the yield statement from your generator function.

Additionally, all of the iterator problems are applicable when creating a generator function. We could have problems that cause us to skip items, process items twice or get an unexpected StopIteration exception.

Collecting Items : The set

Tip

Debugging set()

A common mistake is to do something like set( 1, 2, 3 ), which passes three values to the set() function. If you get a TypeError: set expected at most 1 arguments, got n, you didn’t provide proper tuple to the set factory function.

Another interesting problem is the difference between set( ("word",) ) and set( "word" ).

  • The first example provides a 1-element sequence, ("word,"), to set(), which becomes a 1-element set.
  • The second example passes a 4-character string, "word", which becomes a 4-element set.

In the case of creating sets from strings, there’s no error message. The question is really “what did you mean?” Did you intend to put the entire string into the set? Or did you intend to break the string down to individual characters, and put each character into the set?

External Data and Files

Tip

Constructing File Names

When using Windows-specific punctuation for filenames, you’ll have problems because Python interprets the \ as an escape character. To create a string with a Windows filename, you’ll either need to use \ in the string, or use an r" " string literal. For example, you can use any of the following: r"E:\writing\technical\pythonbook\python.html" or "E:\\writing\\technical\\pythonbook\\python.html".

Note that you can often use "E:/writing/technical/pythonbook/python.html". This uses the POSIX standard punctuation for files paths, /, and is the most portable. Python generally translates standard file names to Windows file names for you.

Generally, you should either use standard names (using /) or use the os.path module to construct filenames. This module eliminates the need to use any specific punctuation. The os.path.join() function makes properly punctuated filenames from sequences of strings

Tip

Debugging Files

There are a number of things that can go wrong in attempting to create a file object.

If the file name is invalid, you will get operating system errors. Usually they will look like this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'wakawaka'

It is very important to get the file’s path completely correct. You’ll notice that each time you start IDLE, it thinks the current working directory is something like C:\Python26. You’re probably doing your work in a different default directory.

When you open a module file in IDLE, you’ll notice that IDLE changes the current working directory is the directory that contains your module. If you have your .py files and your data files all in one directory, you’ll find that things work out well.

The next most common error is to have the wrong permissions. This usually means trying to writing to a file you don’t own, or attempting to create a file in a directory where you don’t have write permission. If you are using a server, or a computer owned by a corporation, this may require some work with your system administrators to sort out what you want to do and how you can accomplish it without compromising security.

The [Errno 2] note in the error message is a reference to the internal operating system error numbers. There are over 100 of these error numbers, all collected into the module named errno. There are a lot of different things that can go wrong, many of which are very, very obscure situations.

Files II : Some Examples and Some Modules

Tip

Debugging CSV Input

One problem with file processing is that our Python data structure isn’t a giant string of characters. However, the file is simply a giant string. Essentially, reading a file is a way of translating the characters into a useful Python structure.

The most common thing that can go wrong is not creating the expected structure in our Python program. In the Reading and Sorting example, we might not create our list of tuples correctly.

It is helpful to print the value of the data variable to get a good look at the data structure which is produced. Here we show the beginning of our “list of tuples”. We’ve adjusted the Python output to make it a little more readable.

[('"^DJI"', '10452.15', '"9/26/2005"', '"1:50pm"', '+32.56',
  '10420.22', '10509.23', '10420.22', '137206720'),
 ('"^IXIC"', '2121.07', '"9/26/2005"', '"1:50pm"', '+4.23',
  '2127.90', '2132.60', '2119.17', '0'), ...

Looking at the intermediate results helps us be sure that we are reading the file properly.

Files III : The Grand Unification

Tip

Debugging File Formats

When we talk about how data appears in files, we are talking about “data representation.” This is a difficult and sometimes subtle design decision. A common question is “How do I know what the data is?” . There are two important points of view.

  • The program you are designing will save data in a file for processing later. Since you are designing the file, you get to choose the representation. You can pick something that is easy for your Python program to write. Or, you can look at other programs and pick something that is easy for the other programs to read. This can be a difficult balancing act.
  • The program you are designing must read data prepared by another program. Since someone else designed the file, you will interpret the data they provide. If their format is something that Python can easily interpret, your program will be very simple. However, the more common situation is that their format is not something Python can interpret, and you must write this interpretation yourself.

Defining New Objects

Tip

Debugging a Class Definition

When we get syntax errors on a class definition, it can be in the class line or one of the internal method function definitions.

If we get a simple SyntaxError on the first line, we have misspelled class, left off a ( or ), or omitted the : that begins the suite of statements that defines the class.

If we get a syntax error further in the class definition, then our method functions aren’t defined correctly. Be sure to indent the def once (so it nests inside the class). Be sure to indent the suite of statements inside the def twice.

Tip

Debugging Object Construction

Assuming we’ve defined a class correctly, there are a three of things that can go wrong when attempting to construct an object of that class.

  • The class name is spelled incorrectly.
  • You’ve omitted the () after the class name. If we say d= Die, we’ve assigned the class object, Die, to the variable d. We have to say d= Die() to use the class name as a factory and create an instance of a class.
  • You’ve got incorrect argument values for the parameters of the __init__().

If we get a NameError: name 'Hack' is not defined, then the class (Hack, in this example) is not actually defined. This could mean one of three things: our class definition had errors in the first place, our definition class name isn’t spelled the same as our object creation (either we spelled it wrong when defining the class, or spelled it wrong when using the class to create an object.) The third possible error is that we have defined the class in a module, imported it, but forgot to quality the class name with the module name.

If our class wasn’t defined, it means we either forgot to define the class, or overlooked the SyntaxError when defining it. If our class has one name and our object constructor has another name, that’s just carelessness; pick a name and stick to it. If we are trying to import our definitions, we can either qualify the names properly, or use from  module import * as the import statement.

Another common problem is using the class name without ()s. If we say d= Die, we’ve assigned the class object (Die) to the variable d. We have to say d= Die() to create an instance of a class.

If we’ve defined our class properly, we can get a message like TypeError: __init__() takes exactly 2 arguments (1 given) when we attempt to construct an object. This means that our __init__() method function doesn’t match the object construction call that we made.

The __init__() function must have a self parameter name, and it must be first. When we construct an object, we don’t provide an argument value for the self parameter, but we must provide values for all of the other parameters after self.

If your initialization function, __init__(), doesn’t seem to work, the most likely cause is that you have misspelled the name. There are two _ before and two _ after the init.

Tip

Debugging Class vs. Object Issues

Perhaps the biggest mistake newbies make is attempting to exercise the method functions of a class instead of a specific object. You can’t easily say Die.roll(), you’ll get the cryptic TypeError: unbound method error message. The phrase “unbound method” means that no instance was being used.

When you say d1= Die(), you are creating an instance. When you see d1.roll(), then you are asking that specific object to do its roll() operation.

Module Definitions – Adding New Concepts

Tip

Debugging Imports

There are four things that can go wrong with an import: (1) the module can’t be found; (2) the module isn’t valid Python; (3) the module doesn’t define what you thought it should define; (4) the module name isn’t unique and some other module with the same name is being found.

Be sure the module’s .py file name is correct, and it’s located on the sys.path. Module filenames are traditionally all lowercase, with minimal punctuation. Some operating systems (like GNU/Linux) are case-sensitive and a seemingly insignificant difference between Random.py and random.py can make your module impossible to find.

The two most visible places to put module files are the current working directory and the Python Lib/site-packages directory. For Windows, this directory is usually under C:\python26\. For GNU/Linux, this is often under the /usr/lib/python2.6/ directory. For MacOS users, this will be in the /System/Library/Frameworks/Python.framework/Versions/Current/ directory tree.

If your module isn’t valid Python, you’ll get syntax errors when you try to import it. You can discover the exact errors by trying to execute the module file using the F5 key in IDLE.

If the module doesn’t define what you thought, there are two likely causes: the Python definitions are incorrect, or you’ve omitted a necessary module-name qualifier. For example, when we do import math everything in that module requires the math qualifier. Within a module, however, we don’t need to qualify names of other things defined in the same module file.

If your Python class or function definitions aren’t correct, it has nothing to do with the modularization. The problem is more fundamental. Starting from something simple and adding features is generally the best way to learn.

The sys.path is a list, which is searched in order. Your working directory is searched first. When your module has the same name as some extension module, your module will conceal that extension module. I’ve spent hours discovering that my module named Image was concealing PIL’s Image module.

Fixed-Point Numbers : Doing High Finance with decimal

Tip

Debugging decimal

There are a number of things that can go wrong with using decimal numbers. If we forget to import decimal (note that the module has a lower-case name), then we’ll get NameError messages.

If we use an import decimal, we must say decimal.Decimal to make a new number. If we use from decimal import *, we can say Decimal to make a new number.

We can convert integers or strings to Decimal numbers. If we accidentally provide a floating-point value instead of a String, we get TypeError: Cannot convert float to Decimal. First convert the float to a string.

Tip

Debugging decimal Rounding

The quantization method appears strange at first because we provide a Decimal object rather than the number of decimal positions. The built-in round() function rounds to a number of positions. The quantize() method of a decimal number, however, uses a decimal number that defines the position to which to round.

If you get AttributeError: 'int' object has no attribute '_is_special', from the quantize() function, this means you tried something like aDecimal.quantize(3). You should use something like aDecimal.quantize(Decimal('0.001')).

Time and Date Processing : The time and datetime Modules

Tip

Debugging time

Because of the various conversions, it’s easy to get confused by having a floating-point time and a time_struct time. When you get TypeError exceptions, you are missing a conversion between the two representations. You can use the help() function and the Python Library Reference (chapter 6.10) to sort this out.

Text Processing and Pattern Matching : The re Module

Tip

Debugging Regular Expressions

If you forget to import the module, then you get NameError on every class, function or variable reference.

If you spell the name wrong on your import statement, or the module isn’t on your Python Path, you’ll get an ImportError. First, be sure you’ve spelled the module name correctly. If you import sys and then look at sys.path, you can see all the places Python look for the module. You can look in each of those directories to see that the files are named.

There are two large problems that can cause problems with regular expressions: getting the regular expression wrong and getting the processing wrong.

The regular expression language, with it’s special characters, escapes, and heavy use of \ is rather difficult to learn. If you get error exceptions from re.compile(), then your RE pattern is improper. For example error: multiple repeat means that your RE is misusing "*" characters. There are a number of these errors which indicate that you are likely missing a \ to escape the special meaning of one or more characters in your pattern.

If you get TypeError errors from match() or search(), then you have not used a candidate string with your pattern. Once you’ve compiled a pattern with pat= re.compile("some pattern"), you use that pattern object with candidate strings: matching= pat.match("candidate"). If you try pat.match(23), 23 isn’t a string and you get a TypeError.

Beyond these very visible problems are the more subtle problem with a pattern that doesn’t match what you think it should match. We’ll look at this separately, in More Debugging Hints.

Wrapping and Packaging Our Solution

Tip

Debugging the Main Program Switch

There are two sides to the main program switch. When a module is executed from the command line, you want it to do useful things. When a module is imported by another module, you want it to provide definitions, but not actually do anything.

Command-Line Behavior. If you get a NameError, you misspelled __name__. If, on the other hand, nothing seems to happen, then you may have misspelled "__main__".

Another common problem is providing all of the class and function definitions, but omitting the main script entirely. The class and def statements all execute silently. If there’s no main script to create the objects and call the functions, then nothing will happen.

Import Behavior. If things happen when you import a module, it’s missing the main program switch. When a module is evolving from main program to library that is used by a new main program, we sometimes leave the old main program in place.

The best way to handle the change from main program to library is to put the old main program into a function with a name like main(), and then put it the simple main program switch that calls this main() function when the module name is "__main__".

Table Of Contents

Previous topic

Professionalism : Additional Tips and Hints

Next topic

Bibliography

This Page