Professionalism : Additional Tips and Hints

A common question – that’s really more of a complaint – is “I get the language, I get the data structures, I get the library, I just don’t know how to get started on the program I want to write.”

This chapter addresses some of those “how do I get started?” and “how do I finish what I’ve started?” questions.

In The Software Life-Cycle we’ll talk about the whole life-cycle of a piece of software, from concept through use to replacement. We’ll look at the initial efforts in Inception – Getting the Characters Right and Elaboration – Overcoming Obstacles. Most of this book has been about programming.

Finally, we’ll talk about additional ways to stay organized. Quality Assurance – Does it Work? includes some notes on quality assurance: how do you know your program works? Configuration Management – Pieces and Parts inclues some notes on a topic called configuration management: what do you have? We’ll wrap up with Transition – Installing the Final Product.

The Software Life-Cycle

In What is Programming? we noted that the life of software has four acts. In that section, we glossed over the first two acts in order to get to the third, which was the programming part.

Let’s go back and look at all four acts in a little bit more depth.

Inception. Software begins as a concept. You start with an idea for solving a problem that involves data and processing. The solution seems to involve information resources that are on a computer, or can be put on a computer.

This idea is the inception for a software development project.

One of the harder parts of getting started is defining the problem. It is not easy to clearly articulate problems because we often jump right past defining the problem into solving it. The most significant distraction to careful problem definition is having focus on the technology failing to recongize who are the actors and what their goals are.

The inception of a project is both the definition of the problem and defining what constitutes a “successful solution”. You have a bunch of questions to answer: “Who uses it?” “What will they do?” “How will it work?” “When and where will they use it?” and most important “Why will they use my software?”

We’ll look at this in some more depth in Inception – Getting the Characters Right.

Elaboration. As we proceed from concept to implementation, we have to elaborate a number of details. We look at details of the problem, our proposed solution, the selected technology and of our software design.s

One very complex activity is applying technology to solve the problem. When you look at Architectural Patterns – A Family Tree and all the choices available, it’s very hard to pick one that fits the problem, locate appropriate libraries and frameworks, and determine what you need to do to apply all that technology to your problem. We can’t really look at this in too much depth becuase there are simply so many choices.

We can, however, talk about the initial steps of transforming an idea into software. Some folks call this “turning the corner” from analyzing the problem to creating a solution. Things work out well when this is a gradual shift in focus and not an abrupt change in technology and terminology.

We’ll look at this in Elaboration – Overcoming Obstacles.

Construction. The bulk of this book was about construction of software. There are some more things we can – and will – say about construction. There are two specific areas that are important aspects of the work, but aren’t specifically related to Python or writing programs in the Python language.

Transition. The final step in software development is the transition from the developer’s hands to the user’s hands. The reason we raised the curtain on this four-act play was to create software that someone can use to be happier and more productive.

Software that you’ve built for your own use can still benefit from a formal move into a “finished” area. It’s good to close the curtain on development and call a project complete.

We’ll look at some ways of doing this in Transition – Installing the Final Product.

Inception – Getting the Characters Right

In Why Read This Book? and What is a Program? we emphasized that programs were about data and processing. Also, in Getting Our Bearings we stated that this book covered processing first and data second.

That overview really only tells part of the story.

Our overview ignores what is – perhaps – the most important part of the concept underlying any piece of software: Who are the actors that use the application?

Actors and Goals. Really, the core part of inception is asking yourself the following questions.

  • Who are the actors? Who interacts with my application? How can we classify the actors?

    It’s important that we classify all the individuals into roles based on what they’re permitted or required to do. Some pieces of software have a single class of actor (often called “the user”). Other pieces of software may have several roles that different people fulfill. Sometimes a single person will act in different roles.

  • For each class of actor, what is their goal? How does my software help them meet that goal and be happier or more productive?

    Actors have goals. They don’t do some task randomly; they have a purpose. It’s very important to recognize that purpose and assure that the software we’re writing fits that purpose.

Use Cases. When an actor interacts with your software, the sequence of interactions forms a use case. Most use cases are pretty simple, and have a goal and a sequence of interactions that the actor engages in to reach their goal.

Some use cases are more complicated and may have a number of variant scenarios to reach the same goal.

It’s important to note that every program, not matter how trivially small, has at least one use case. Someone interacts with your program to achieve a goal.

Even if your program is a simple script, run from the command-line, there’s still a use case. The interaction is generally really simple: the actor runs the program, the system responds with the results. Even though it’s simple, it’s still an interaction, and it must be considered as part of your program’s use cases.

During the opening acts of software development, we merely want to identify the use cases in some general way. We want to give them titles, perhaps define which actors will engage in them. We might want to state the goals for each use case.

As we move into Elaboration, we’ll provide more extensive information.

Elaboration – Overcoming Obstacles

We elaborate several things as our software moves from concept to implementation.

The first things we usually do is expand on our use cases. The most important part of this is to clearly understand the actor’s goals and the problem they have to overcome to achieve their goals. The use cases are the interactions that the actors need to have in order to meet their goals.

Writing use cases well is beyond the scope of this book. We can only suggest that well-written use cases clearly define the actors, their goals, and how they interact with a system to achieve those goals. Once you have the use cases in hand, it’s much easier to design and write your Python programs.

Once we’ve organized the use cases around the actors goals and their interactions with our software, we can move on to picking some technologies and designing our software.

Core Skills. We’ve mentioned the abstraction principle many times. The elaboration task is really about providing the proper level of abstraction. A program is a complex thing. The user cases, the computer, the chosen technologies, the modules and classes, even the context in which the program is used are all important at one time or another.

In order to manage all these details, we need to make careful use of abstraction.

Abstraction is a two-way street. Sometimes we add details to a general concept, sometimes we summarize a lot of details into a general concept.

The most important part of the abstraction skill is determining how best to summarize a lot of details so the relevant information is apparent, and the irrelevant information is concealed. Doing this well means that we often have to rearrange our summaries, concepts and abstractions so that they properly reflect the relevant features.

Technology Choice. We often have too many technology choices. When we look at the brief overview in Architectural Patterns – A Family Tree we can see a dismaying variety of ways to attack any given problem. All of them are good, and all of them can be made to work.

Without extensive experience it’s hard to know which involve a lot of complex programming, and which are pretty easy. As tools and platforms change, the complexities move around, making it difficult to make any blanket statements about technology choices.

Often, we solve problems using the technologies we’re most familiar with. We call this the “Hammer Syndrome”. If the only tool you have is a hammer, every problem will look like a nail.

The only way to overvome the hammer syndrome is to learn how to use a wide variety of programming tools. You need to write programs with a variety of architectures: Command Line, GUI and Web in particular. Once you’ve written programs in these three important architectures, you are better able to choose an architecture that solves your problem effectively.

Design Details. Once we’ve got use cases and a technology, we can start to design a solution. As noted in Script or Library? The Main Program Switch, we’re going to have modules that define the essential “model” of the real-world objects, and we’re going to have a main program module.

Often, these two modules start out as a single module with everything. Later – as our program matures – we may refactor to split it into pieces.

Often, we start with class definitions. To write our class definitions, it sometimes helps to rough out a design as a simple file of definitions with document strings. It helps to put a crisp responsibility statement and a list of collaborators in the docstring to help focus on what a class does.

class Bin( object ):
    """A bin on a roulette wheel.

    Responsibilities:

    - Number of bin
    - Color
    - Even/Odd
    - High/Low
    - Red/Black
    - Column number
    - etc.

    Collaborates with:

    - Wheel
    """
    pass

class Wheel( list ):
    """A collection of 38 Bins.

    Responsibilities:

    - Holds all the bins
    - Picks a bin at random

    Collaborators:

    - Bin
    - Some simulation
    """
    pass

class Simulation( object ):
    """TODO: don't know quite what this does...

    Reponsibilities:

    Collaborators:

    - Wheel
    """
    pass

We can use a file like this for “thinking out loud” about our design. What classes do we think we’ll need? What will they do? What real-world thing do they parallel?

Quality Assurance – Does it Work?

We could talk a lot about how Python and the practices of quality assurance fit together.

Instead, we’ll leave you with a few hints and suggestions.

Simpler Is Better. First, we have to paraphrase Albert Einstein, and suggest that software must be “made as simple as possible, but no simpler.”

We have to add E. W. Dijkstra’s observation that “simplicity and elegance are unpopular because they require hard work and discipline to achieve”.

To this, I’ll add “If it’s really hard, you’re doing it wrong.” Most Python libraries are simple and solve some prople really well. If you pick the wrong library, you may have to really struggle to get it to do what you want.

If you find that you’re really struggling, it may mean that you’ve picked the wrong tool for the job. You might want to stop, take a step back, and look around for alternative tools, or an alternative approach.

Building the Right Thing. Quality Assurance starts during inception when we ask ourselves if we’re really solving the right problem for the right people. This fundamental question – “What problem does this solve?” – must be carried through every step of building software.

It helps to expand this question slightly:

  • What is the problem?
  • Who has this problem?
  • Why do they have this problem?
  • When and Where do they have this problem?

Often, this fundamental question is ignored. New technology is an attractive nuisance, and too many programmers are seduced by technology that isn’t really helping them solve any problems.

This aspect of quality assurance requires some reflection and consideration. Sometimes it requires wisdom or insight. Other times it requires someone to ask the “dumb question” of “why are we building this?” or “why are we building it this way”?

Build the Thing Right. Quality Assurance contains a more technical consideration, also. This comes into play during Construction and Transition. This fundamental question – “Does this actually solve the problem?” – must also be asked.

We often expand on this slightly:

  • Are we using using Python (and the various libraries) the right way?

  • Does our program actually work?

    This question should always be followed by “What evidence do you have?” Which leads us to the final question.

  • Does it produce correct answers for test cases?

One technique for determining if we’re building something the right way is to write tests for all of the packages, modules, classes and functions we created. Besides providing evidence of correct behavior, writing tests is a very popular way to gain experience with the Python language, the libraries, and the modules and classes we’re trying to design.

This sense of quality assurance requires technical tools. We’ll look at two Python modules that help with using tests as part of quality assurance.

  • doctest. This module examines the docstrings for classes and functions to locate test scenarios.
  • unittest. This module executes unit test scripts. A unit test script is a special-purpose module designed to test other modules.

A Fixture That Needs Testing. Let’s look at a really simple module that defines a single function. We’ll show examples of how to write tests for this module.

Here’s our initial version of this module, without giving miuch thought to how we would test this module for correct behavior.

wheel.py

#!/usr/bin/env python
"""A really simple module."""

def even( spin ):
    """even( spin ) -> true if the spin is even and not zero.
    """
    return spin % 2 == 0

Note carefully that our docstring for the even() function doesn’t match the actual definition. We have a bug, and we’ll show how testing reveals this bug.

Doctest Example. To use doctest, we need to write docstrings that includes actual examples of using the function. We make those examples look like they were copied and pasted from Python in interactive mode.

Let’s put this function in a module called wheel.py.

wheel.py

#!/usr/bin/env python
"""A really simple module.

>>> import wheel
>>> wheel.even( 2 )
True
"""

def even( spin ):
    """even( spin ) -> true if the spin is even and not zero."

    >>> even( 1 )
    False
    >>> even( 2 )
    True
    >>> even( 0 )
    False
    """
    return spin % 2 == 0
  1. In the module docstring, we show how the module as a whole should be used. We created an interactive Python log showing how we expect this module to behave in general.
  2. In the even() docstring, we also show how the function should be used. In this case, we wrote the log we expected to see. This docstring shows what would happen if the function was written correctly.

Here’s a separate script that will examine the docstrings, locate the test sequences, execute them, and compare actual results against the expected results in the docstrings.

import wheel
import doctest
doctest.testmod(wheel)
  1. We import the module we’re going to test.
  2. We import doctest.
  3. We evaulate the doctest.testmod() function against the given module.

Here’s the output from running this.

MacBook-5:notes slott$ python test1_doctest.py
**********************************************************************
File "/Users/slott/Documents/demo/roulette/wheel.py", line 16, in wheel.even
Failed example:
    even( 0 )
Expected:
    False
Got:
    True
**********************************************************************
1 items had failures:
   1 of   3 in wheel.even
***Test Failed*** 1 failures.

Interestingly, our even() function has a bug. In Roulette, the numbers 0 and 00 are neither even nor odd. Our even() function doesn’t handle 0 at all.

Unittest Example. To use unittest, we need to write a module that includes some tests cases and a main script. The test cases are formalized as subclasses of unittest.TestCase. Each of these class definitions embodies a series of test methods applied to a test fixture.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
#!/usr/bin/env python
import unittest

import wheel

class TestEven( unittest.TestCase ):
    def test_0( self ):
        self.assertFalse( wheel.even(0) )
    def test_1( self ):
        self.assertFalse( wheel.even(1) )
    def test_2( self ):
        self.assertTrue( wheel.even(2) )

if __name__ == "__main__":
    unittest.main()
  1. We import unittest.
  1. We import the module we’re going to test.
  1. We define a subclass of unittest.TestCase.
    • Each method function that begin with test... is a different test of our “fixture” – in this case, the function wheel.even().
    • Each method function is based on an “assert” or “fail” method function of unittest.TestCase. There are dozens of these method functions to help us specify the behavior of our fixture. In this case, we used the assertTrue() and assertFalse() functions.
  1. We use the main program switch to execute unittest.main() only when this module is the main module.

    The unittest.main() function will locate all of the subclasses of TestCase. It locate all method functions with names that start with test. It will then create the object, execute the methods, and count the number of tests that pass, fail or have errors.

Here’s the output.

MacBook-5:notes slott$ python test1_unittest.py
F..
======================================================================
FAIL: test_0 (__main__.TestEven)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "notes/test1_unittest.py", line 8, in test_0
    self.assertFalse( wheel.even( 0 ) )
AssertionError

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

The first part (F..) is a summary of the tests being run. It shows a test failure followed by two successes. This is followed by the details of each failure.

If everything works, you’ll see a string like ... and nothing more.

This shows the bug in our even(). In Roulette, the numbers 0 and 00 are neither even nor odd. Our even() function doesn’t handle 0 correctly.

Configuration Management – Pieces and Parts

It’s common to build applications from a number of separate cmoponents. We might, for example, have a simple application that has a library module with the essential classes and functions, plus a main application module, plus some unittest scripts.

Further, our application might depend on other libraries that we downloaded from the internet. For example, we might be using PIL or pyAudio to process images or audio samples.

How do we manage the configuration of our various components?

Accounting 101. Configuration management is really an accounting practice. The heart of configuration management is to tracking the various assets that are part of our overall application program.

Unlike real-world accounting, however, we don’t have tangible physical objects that we can put an asset tag on. Instead we have files on our computer that we need to keep track of.

Further, software components have version numbers which are a very important thing. We have to be very careful to distinguish between two different versions of the same component or library that we downlaoded.

Naming. Half the battle in tracking our software assets is to have directory names that include version numbers. For example, when we install Python under Windows, we need to put it in a directory named C:\python2.5. That way, we can add Python 3.0 without breaking our existing installation.

Completeness. The other half of the battle in tracking our assets is having a complete list of everything we’re using. It’s important to keep a file with the following information.

  • The web sites from which your downloaded components.

  • The version number you downloaded.

  • The kind of license under which you’re using the component.

    Generally, the license will be one of the following: Academic Free License (AFL), Attribution Assurance License, BSD License, GNU General Public License (GPL), GNU Library or Lesser General Public License (LGPL), Mozilla Public License 1.1 (MPL 1.1), Python Software Foundation License.

Transition – Installing the Final Product

Software is infinitely changeable. You can always tweak and adjust your Python programs.

However, at some point, you want to declare the project as “done”. Usually in the sense of “done for now.” Or “done until I think of something to add.”

We need to transition our software to some reasonably final form. When we’re provising software to someone else, this packaging step is essential. When we’re using software for our own purposes, this step is optional but important.

Organizing Your Workbench. We need to make a distinction between a “working copy” of some module and the official “library copy” of the module.

When we’re learning, everything’s a working copy. Further, everything starts out jumbled into a single directory.

As we move toward more professional level of craftsmanship, we start to organize our projects into separate directories. Further, we’ll need to install our modules in the Python site-packages directory.

The Python Library. When we looked at the import statement in Thinking In Modules, and the Declaration of Dependence, we touched on the search path. The search path is the list of places that Python searches for a module. If you import the sys module, you’ll see that sys.path is the list of locations that are searched.

What we in the section on modules was the first locations searched was '' (an empty string). This is Python’s short-hand for the current working directory. Because of this, we can simply pile our modules into a single directory, and everything works nicely.

As soon as we start moving things around, we need to do one of two things:

  • Add the locations of our modules to the PYTHONPATH environment variable.
  • Put our modules into Python’s site-packages directory.

Installed Packages. Installing modules in the official site-packages directory is done with a set of Python tools called distutils. We can only touch on the essence of distutils here.

To make our package easily installed in site-packages, we need to write a setup.py file. This file contains the information required to package and install our module.

Here’s a typical distutils setup.py module.

#!/usr/bin/env python
from distutils.core import setup

from distutils.core import setup
setup(name='My Roulette Module',
      version='1.0',
      py_modules=['roulette'],
      )

This will allow you to do the following to install your module in the site-packages library.

python setup.py install

Once you’ve done this, you can work in any directory on your computer and have access to your new module. This allows you to create new applications based on modules you’ve already written.

Table Of Contents

Previous topic

Architectural Patterns – A Family Tree

Next topic

Debugging Tips

This Page