The Story of Mel, one of the appendixes of the Jargon File, tells the story of a programmer writing in machine code that comes up with very creative, unorthodox code (http://www.jargon.net/jargonfile/t/TheStoryofMel.html). In particular:

When the light went on it nearly blinded me.
He had located the data he was working on
near the top of memory —
the largest locations the instructions could address —
so, after the last datum was handled,
incrementing the instruction address
would make it overflow.
The carry would add one to the
operation code, changing it to the next one in the instruction set:
a jump instruction.
Sure enough, the next program instruction was
in address location zero,
and the program went happily on its way.

Of course, Mel’s code was pretty unreadable for other machine-code programmers (it is a true story) and the author finally gives up on changing it, but this has become one of the most famous examples of what a “true hacker” is. So: even in machine code there is room for creativity, and a need arises even there to try and write “readable” code – organized, easy to figure out and debug. (My own personal experience is that assembly needs to be very trimmed and organized to be even marginally workable, but I have only writen smallish exercises in college on it) You are right about letters, that was a sloppy example – I just needed to illustrate the difference between a rule (or formal requirement) and a convention (or practical principle).

I don’t think we disagree, actually. I like to draw a distinction between what code can be – as formal systems – and how code is written in practice. When you write a letter, there are, beyond the rigid rules of grammar, also conventions pertaining to letter-writing that decide that some things are expected (say, a salutation to the reader), some are unusual but ok, and some are surprising and possibly disruptive or unacceptable. That code is supposed to be readable is such a convention dictated by practice – not, as you have pointed out, a formal necessity. Of course, we are not bounded by such conventions, but I don’t think we can overlook them either – for one thing, they tell a lot about what code means, what motivates us to write code in the first place. Also, we can’t help facing them: if you choose to write unredeable code, this straying away from conventions will be in itself meaningful. If you follow the conventions, that too is meaningful.

Since the discussion has since taken into Turing Machines and Godel’s theorem, I would like to quote Kittler (in Hardware: das Unbekannte Wesen, if I’m not mistaken) in saying that only in Turing’s paper there has ever been such a thing as a Turing Machine, unbounded by restrictions of time, space and resources; or, as he puts it, it is just as possible for an actual computer to compute what tomorrow’s weather is going to be like as it is impossible for this prediction to be ready today. Any modern computer can emulate a Turing Machine – and can, in its turn, be emulated by a Turing Machine – but in this emulation there are, again, practical restrictions that should not go overlooked.

That said, I have found this discussion pretty stimulating, and as a sidenote, I point out that I don’t always find my own code very meaningful the morning after.

I’m glad you’ve taken up this topic, and am looking forward to the rest of your discussion.

I like your examples of real-life non-provable but true statements. In the case of a guilty person who is acquitted of a crime, the law itself pretty much assures that the question of guilt is non-provable, because you normally can’t be re-tried for the same crime if you have been acquitted.

When I said Godel had to be tricky to construct the non-provable true statement in the formal language, I was referring to the fairly complicated way he had to encode the statements in the formal system so as to be able to construct the self-referential statement “This proposition is not provable.”

You are quite right that the idea is not complicated — it’s similar to the Barber Paradox which asks “Who shaves the barber who shaves everyone who does not shave himself?” I ran into this in a book by Douglas Hofstadter (Metamagical Themas) as a child and annoyed everyone I knew with it. I certainly did not need advanced math to understand the paradox.

But the formal system used in the Godel Proof is meant to express very concrete statements about sets and numbers and it took a lot of cleverness by Godel to make it be self-referential, expressing propositions about propositions.

For instance, many a circumstantial conviction has been made in legal cases; the jury believed the prosecution although the case was not or even couldn’t be proved.

Faith and belief are usually matters of conviction in propositions that haven’t been or even can’t be proved.

The example that Godel came up with of a proposition that is true but not provable was this: ‘This proposition is not provable’. Which is pretty simple, really, in a sense. It can’t be false. Here’s why. If it’s false, then the proposition is provable. If it’s provable, then it’s true. Because only true propositions are provable (in a consistent system). But we assumed it was false; contradiction. So it can’t be false. If it’s a well-formed proposition, then it has to be either false or true. Since it ain’t false, it must be true. But if it’s true, it’s also unprovable.

In any case, we are profoundly familiar with the notion of propositions that are true but not provable. And so will any system be that can deal with our lives and thought. So future ontologies will certainly have a place for such propositions.

Turing machines are supposed to be (Church-Turing thesis, which everyone believes, proposed in various articles by Church, Kleene, Rosser, and Turing in 1935-7) able to solve any problem which _can_ be solved in a finite number of steps. This has practical implications for computer programming because all modern computers are thought to be equivalent to Turing machines (they can solve the same set of problems).

The very nuts and bolts subject of time/space complexity of algorithms where you use big-O notation, etc. to describe how long a problem takes to solve and how much computer memory is required came out of these pre-computer age questions of which problems are solvable. For example some problems (e.g the classic traveling salesman problem on n cities where the computer is asked to find the shortest path that goes through each city exactly once) are finite but virtually unsolvable in practice because the time taken to solve them goes up catastrophically as n increases.

Lots of practical problems (inventory management, planning power or transportation networks, circuit design, laying out shapes to be cut out of sheets of steel) turn out to be equivalent to this kind of thing, and the mathematics of computability tells us not to bother looking for a better algorithm. On the other hand, the classic results tell us it is very hard to find the _optimal_ solution to these problems, but in practice a “good-enough” solution which isn’t necessarily optimal can work very well, so there is still room for clever people to invent algorithms.

I really liked the article about Esoteric languages and look forward to the next installment. There are some real practical languages which are almost as small and incomprehensible, for example assembly languages for RISC (reduced instruction set) processors. Humans never have to write programs in such languages, but the designers of the first compilers had to be able to do so to some extent. Then there are large languages which seem almost like a parody of themselves, such as perl which can be very obfuscated.

Does anyone remember the obfuscated C contest? Apparently it still exists, I just found the site http://www.ioccc.org/main.html

i too have an interest in turing, turing completeness, and the theory of computation.

it’s ironic why turing created the turing machine. he invented it to prove a theorem in mathematics, mainly. he wanted to prove that there are some things that no machine will ever compute. so he needed to invent a machine that could compute anything any conceivable machine could compute. and then show that there are some things that machine can’t compute.

and he did. the modern computer was invented to show that there are some things it can’t compute.

in other words, he solved the entscheidungsproblem, a problem in mathematics that had been posed by david hilbert in 1928. it was considered one of the most important unsolved math problems. because the issue was basically whether math was over. could a machine be built that would prove all proveable theorems? so that mathematicians could build the machine and then close up shop and let the machine publish its results.