D.E.K.

Background

TeX (→ LaTeX) was written by Don Knuth.

TeX and Metafont have been application tests for Web; Knuth's programming system for literate programming.

Thus, the Web system passed its most crucial test when it was first used to produce the programs for TeX and for Metafont.

—[0, p. 185]

>>>> TeX.date
Traceback (most recent call last):
  File "<brain>", line 1, in <module>
KnowledgeBaseError: I am too young for this
>>>>

Timeline of Literate programming in python

Literate programming

• Structured programming is not yet entirely satisfactory.
• Structured programs is more readable than machine code.
• Literature is more readable than programs.
• Consider programs to be works of literature.

—[0, p. 99]

Literate programming

1. We understand a complicated system by understanding its simple parts.
2. Programming is not creating a hierarchical structure. It's neither bottom-up nor top-down.
3. If we express a program as a web of ideas, we can emphasize its structural properties in a natural and satisfying way.
4. It's rather a web of ideas which results from your mental representation and gets expressed in source code.
5. Definitions in programs cannot be put anywhere. Therefore the order of the instructions is merely given.
6. This order is a problem for us. The order of consciousness is the semantically intelligent order understandable by programmers and gets violated.
7. We have to provide tools for reordering, formatting and restructuring text in a LP system.

—[0, p. 99]

Literate programming

A literate programming system is provided iff

• One document contains documentation and source code
• Two programs extract source code and documentation separately
• Extracted text will not be modified
• Section references make up the whole program

The problem of software documentation

Visual representation of handling the Knowledge abstraction conflict

Software documentation

What are the variables?

• Abstraction layer
  Do we want to talk about the Observer pattern in ASM?
• Software paradigm
  Is a real-world object an object, a class, a record or a list with associated functions?
• Target audience
  High-skilled programmers will know more concepts and therefore understand faster. But we might also have several audiences {programmer, end user}
• Readability
  A brainf*ck program might require more documentation

Software documentation

What are the variables?

• Conventions & idioms
  Every programmer has its own conventions. Believe it or not!
• Application domain
  Different domain. Different metaphors, objects, assumption about the world…
• Formatted output
  Everybody has his own special needs
• Represent Web & Order of consciousness in document

Example: Abstraction layer

  0 SETUP_LOOP              33 (to 36)
  3 LOAD_FAST                1 (b)
  6 LOAD_CONST               1 (0)
  9 COMPARE_OP               3 (!=)
 12 POP_JUMP_IF_FALSE       35

 15 LOAD_FAST                1 (b)
 18 LOAD_FAST                0 (a)
 21 LOAD_FAST                1 (b)
 24 BINARY_MODULO       
 25 ROT_TWO             
 26 STORE_FAST               0 (a)
 29 STORE_FAST               1 (b)
 32 JUMP_ABSOLUTE            3
 35 POP_BLOCK           

 36 LOAD_FAST                0 (a)
 39 RETURN_VALUE

Example: Abstraction layer

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def euclid(a, b):
    while b != 0:
        a, b = b, a % b
    return a

if __name__ == '__main__':
    print(euclid(21, 35))

Example: Software paradigm

import Data.List
 
seq1 n = snd . foldl sequ' (1, 0) . map (toEnum . fromIntegral) $ unfoldl divs n
    where
        unfoldl f x = case f x of
                Nothing     -> []
                Just (u, v) -> unfoldl f v ++ [u]
 
        divs 0 = Nothing
        divs k = Just (uncurry (flip (,)) (k `divMod` 2))
 
        sequ' (f, g) p
            | p         = (f*(f+2*g), f^2 + g^2)
            | otherwise = (f^2+g^2,   g*(2*f-g))

The common sense

Given is a particular documentation. Will programmer X understand the documentation?

• The abstraction level is given. We might need different tools to document C code or python code. The programmer has to be familiar with the abstraction layer anyhow.
  (WEB puts a preprocessor in front of Pascal. We don't need that for python! That's probably a difference.)
• The software paradigm is not given in a multi-paradigmatic environment. Paradigms affect terminology.
  • OOP is common sense, but also structured and functional programming are hip.
  • But programmers are not flexible.
  • A python file will primarily be read by people familiar with python and its popular paradigms
  • In a perfect environment everybody is highly motivated to understand all paradigms anyhow
• Readability is given by the language.
• Output style (ie. formatting) is not a big deal. Everybody has a general understanding of beauty.

The common sense

The real deal-breakers are

• Target audience
• Conventions & idioms
• Application domain
• Web & Order of consciousness

We are making 2 assumptions here:

• The reader is familiar with the programming language.
• We focus the programmers as our target audience
  End users require a different document anyhow and programmers are the major usecase.

What's wrong? What's right? Docu might help.

int errno = ERROR_NONE;
if (errno = delete_item(*prev))
{
    // deleted.
}

Documentation is only needed when program flow isn't obvious.

Guessing means software regression.

Established concepts in the software industry

• Programmer's terminology (iteration, modulo …)
• Changelogs
• Design Patterns
• Comments in the source code # // /* */ -- ;
• Docstrings

Yeah, but taking over an old project of an unavailable employee is probably the most common usecase for documentation?! Which of those concepts do help with the target audience, conventions & idioms and application domain?

Freedoms of a programmer

Structured programming (Pascal, C)

• File names
• Names for variables, types, constants, records
• Procedure and function names
• Comments

Freedoms of a programmer

Python

• File structure (Guido says: modules, packages)
• Filenames
• Identifier names (classes, functions, variables)
• Comments
• Docstrings

One command per line generates often one semantic per line.

Documentation support

No programming language has full support for literate programming natively ever since 1981; Haskell might have. For instance, no one provides tools to create compilable and readable text separately.

Probably, the WEB system implies to many things:

• At least one of the {weave, tangle} can be left out?
• Copy and Paste references? This is macro style and out of date!
• tangle makes source code ugly. This might be impossible in python

Haskell: Almost literally

Save as rule110.lhs (literate Haskell)

This Haskell file implements Rule 100.  Rule 110 is a cellular automaton rule
for a 1D cellular automaton.  

The state of the cellular automaton is a string that is infinite in both
directions and contains 0s and 1s.  Updating the state of the string consists
of replacing each value with a new value based on its context.  Patterns
"111", "100", and "000" result in the value "0" at the center, and all other
patterns result in the value "1" at their center.

Let's start with a replacement rule that isn't quite right but gets most
of the job done.

> repl110 :: String -> String
> repl110 s | (length s < 3) = s
> repl110 s | take 3 s `elem` ["111","100","000"] = '0' : repl110 (tail s)
> repl110 s = '1' : repl110 (tail s)

Clojure: Documentation strings

(defn func "A doc showcase." [name] (str "Hello World, " name "!"))
(doc func)

Docstrings like python. But besides :doc also eg. :test is available.

>>> def func(name):
...     """A doc showcase."""
...     return "Hello World, {}!".format(name)
... 
>>> func.__doc__
'A doc showcase.'
>>> func.__test__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute '__test__'
>>>

XML community

Yeah, there are numerous examples for possible XML formats out there. Inline SVG and MathML rocks and XSLT provide simple tools for the implementation.

But who wants to write XML?

Perl: POD

Documentation? Yes. Literate programming? No.

=pod

=head1 NAME

Date::Manip::Delta - Methods for working with deltas

=head1 SYNOPSIS

   use Date::Manip::Delta;
   $date = new Date::Manip::Delta;

=head1 DESCRIPTION

This module contains functions useful in parsing and manipulating
deltas.  As used in this module, a delta refers only to the amount of
time elapsed.  It includes no information about a starting or ending
time.

There are several concepts involved in understanding the properties
of a delta.

=over 4

=item B<fields>

A delta consists of 7 fields: years, months, weeks, days, hours,
minutes, and seconds, usually expressed as a colon-separated string.
For example:

   1:2:3:4:5:6:7

refers to an elapsed amount of time 1 year, 2 months, 3 weeks, 4 days,
5 hours, 6 minutes, and 7 seconds long.

=cut

print("Hello World")

CWEB

Switch between different modes

javadoc

/**
 * Returns an Image object that can then be painted on the screen. 
 * The url argument must specify an absolute {@link URL}. The name
 * argument is a specifier that is relative to the url argument. 
 * <p>
 * This method always returns immediately, whether or not the 
 * image exists. When this applet attempts to draw the image on
 * the screen, the data will be loaded. The graphics primitives 
 * that draw the image will incrementally paint on the screen. 
 *
 * @param  url  an absolute URL giving the base location of the image
 * @param  name the location of the image, relative to the url argument
 * @return      the image at the specified URL
 * @see         Image
 */
 public Image getImage(URL url, String name) {
        try {
            return getImage(new URL(url, name));
        } catch (MalformedURLException e) {
            return null;
        }
 }

—via oracle.com

Sphinx documentation generator

For specification! At least meant to be.

Summary

Problems include:

• Writing text is unpopular among software developers
• Documentation gets out of date with each major release
• Because it's not interpretable, it does not always describe the important parts.
• We need:
  • Developer's documentation
  • User manual
  • Software specification
  • Tests
  • Tutorials / Examples

Different opinions

I guess, Guido is in favor of Embed documentation in a markup-language inside source code as comments.

Knuth says Embed marked-up source code in documentation

Summary

Literate Programming is difficult. Jon Bentley's observation: "a small percentage of the world's population is good at programming, and a small percentage is good at writing; apparently [Knuth is] asking everybody to be in both subsets."

—S.Lott at StackOverflow

Message of the Day

You should be commenting why you are doing something, not what the code does. Documentation for programmers is the problem.

Documentation in monospace? Take it to the next level!

Think about doc tools! Pick one. Use it.