Learning To Drive a Stick Shift
Saturday, July 19, 2008   

I've always thought that programmers should know how to drive a stick shift

Pop the clutch, wrestle it into 2nd, finesse the gas, let the acceleration plaster you into those plush, faux-leather seats.

This is driving.

Now, I don't have anything against automatic transmissions. In fact, I love automatic transmissions:

  • C#
  • Java
  • VB.NET
  • PHP
  • Ruby
  • Python

But a programmer should know, or at least be familiar with, the low-level stuff.

  • C/C++
  • Assembler

That means: pointers. Threads. DLLs. Import tables. Memory allocation. Nuts and bolts. Stuff that's often abstracted away for us quite nicely by the .NET framework or the JVM. In a recent Stackoverflow.com podcast, Joel Spolsky makes the point:

But, but, but, but, but you see the thing about C is it's not COBOL, which is the language that the old programmers used, it's the language that is closer to the machine.  And so, things like stacks, they're still going on there, and, and malloc and memory allocation in C, yeah it's a lot of manual work that's done for you automatically in modern languages, but it's done for you automatically--it's still getting done.  And so studying C, for example, is like learning how to drive a stick shift car.  It's learning how your car works, and what the connections are, and how the gears work, and basically the main parts of the drive shaft, and parts of your car.  And you don't have to know about them to drive an automatic, it's true, but to really be a person who designs cars, or is an expert at cars, or who's an auto mechanic, or who's just kind of a wizard at cars, you have to know that anatomical stuff.  So you're not really--I don't think C is an obsolete language, because it's just an expression of assembler.  It's just an easier to use expression of assembler language.  And it reflects what actually goes on at the chip

Joe, you stole my metaphor. Or rather, my simile.

The typical attitude around the development water cooler is that "you only need C++ for performance, and performance is a non-issue in most applications." But this kind of either/or thinking misses the forest for the trees. As Visual C++ team member Stephan T. Lavavej put it in The Future of C++:

Aside from the elevator controllers and supercomputers, does performance still matter for ordinary desktops and servers?  Oh yes.  Processors have finally hit a brick wall, as our Herb Sutter explained in 2005 at http://gotw.ca/publications/concurrency-ddj.htm .  The hardware people, who do magical things with silicon, have encountered engineering limitations that have prevented consumer processors from steadily rising in frequency as they have since the beginning of time.  Although our processors aren't getting any slower, they're also not getting massively faster anymore (at least, barring some incredible breakthrough).  And anyways, there isn't plenty of room at the bottom anymore.  Our circuits are incredibly close to the atomic level, and atoms aren't getting any smaller.  The engineering limit to frequency has simply arrived before the physical limit to circuitry.  Caches will continue to get larger for the foreseeable future, which is nice, but having a cache that's twice as large isn't as nice as running everything at twice the frequency.

But let's ignore the mythology that performance isn't an issue. Let's just admit the following truths, and hold them to be self-evident:

  • That C/C++ is the most performant language in the world, with the exception of assembler itself.
  • That C++ brings this performance to bear using a high-level syntax which is remarkably similar to the syntax of Java and C#—because Java and C# borrowed heavily from C++.
  • That for some desktop applications, performance isn't an issue.

But these truths have little or nothing to do with why every programmer should know C and C++.

They have little or nothing to do with why I say: if you want to be a top-notch programmer, you can no more afford to ignore the C and C++ languages than a civil engineer can afford to ignore the difference between a plumb line and a snap line, a right angle and an oblique one.

By saying that, I'm not implying that I consider myself to be a top-notch programmer. And I'm not recommending that you or your organization start or continue developing in C++ as a way of conducting business. That's a considerably more complex question, and one nobody can really answer but you.

I'm not starting or contributing to a language war here, of any kind. Not even a minor skirmish.

No, I recommend learning C++ for the most selfless and utilitarian of reasons: learning C/C++ will make you a better programmer in your language of choice, whatever it may be.

Ruby. Java. C#. VB.NET. And yes, even Haskell. Whatever you program in: C++ will make you better at it.

Take a typical high-level programmer and immerse him for just a few months in a pot of boiling oil native C++ project:

  • Processes and threads
  • Stacks and heaps
  • Pointers and references
  • Memory addresses
  • Why hexadecimal is your friend
  • How strings are represented in memory

He'll emerge leaner, tougher, and with renewed confidence in the language of the day, whether it's C#, C++, or Ruby. He'll know, for example, exactly what it means to push a variable onto the stack. And he'll know why, when he tries to allocate and then start accessing an array with 2 million elements, the hard drive churns even though the machine has 4GB of memory.

And when the whole application crashes, and spits out some god-forsaken maintenance error code along with a 4-byte or 8-byte memory address: he'll have some clue what it means, or how to go about finding out what it means.

C++ informs and bolsters your software development skills across the board by acquainting you with the nuts and bolts that underlie every language in the world, almost without exception.

So the next time you have some spare time, pick up a good introductory or advanced C++ book, dust the cobwebs off of your C++ compiler, and point the nose of that lean, mean, muscle machine out your garage door, through the quaint byways of suburbia, and up into the mountains.

You'll thank yourself for it in the interview room.


Posted by James Devlin   33 comment(s)

SEARCH

COMMENTS

While I agree learning to drive stick is important, the case for performance is much better made by learning to go parallel like Erlang than by C/C++. Frequency isn't increasing, but number of cores are doubling now--do you really want to miss that with weaker forms of parallelism?

Eric B on 7/20/2008 1:33:20 AM (79 days ago)

C# can actually be more performant than C++, thanks to processor-specific compilation. A little known fact, but true.

Still I agree with the general thrust. All too often the so-called "lower-level" languages get the boot in favor of fancy buzzwords.

Anonymous on 7/20/2008 2:12:07 AM (79 days ago)

The following statement by Anonymous is FALSE:

"C# can actually be more performant than C++, thanks to processor-specific compilation. A little known fact, but true."

This is a Microsoft-sponsored delusion. Take a look at the benchmarks:

www.osnews.com/.../

Ed K. on 7/20/2008 3:37:35 AM (79 days ago)

Hm, I think studying at least the basics of operating systems is more important than working with C/C++ to gain a deeper knowledge of the "computer inside". But then C/C++ is probably the second most important thing to learn Smile

Florian Potschka on 7/20/2008 3:55:37 AM (79 days ago)

@Ed K: Not necessarily. There are benchmarks which point both ways. For example, check out:

Is Managed Code Slower Than Unmanaged Code?

I admit that MS wants C# to be perceived as fast, but that doesn't necessarily means it's not. ;)

Anonymous on 7/20/2008 5:34:39 AM (79 days ago)

Oy! You guys are starting a skirmish! heh. However, it is simply not possible for all the managed languages to be faster than native ones. Sure, you can have a smart compiler that takes advantage of each processor's strenghts and weaknesses, but that applies to every other language as well. Oh dear! I've contributed!

Ed M. on 7/20/2008 9:10:41 AM (79 days ago)

Thanks god I started my career with C++ Wink

David on 7/20/2008 12:07:38 PM (79 days ago)

I'm busy trying to learn C++ at the moment. I know PHP quite well but C++ is a whole load harder. I'm glad I'm learning it though Laughing

Poker League on 7/20/2008 12:37:42 PM (79 days ago)

C# *can* be more performance than C++. It is *not* a Microsoft delusion and is pretty simple to understand. C++ is compiled ahead of time into machine language. At that time a processor target is chosen. This can be P3, P4, PM, etc. along with a host of additional processor features that are available to aid an application with hardware acceleration for things like SIMD, DSP, and other multimedia instructions. Plain and simple, most C/C++ applications are compiled to target the lowest common denominator for the broadest supported PC base. This means the compiler will not take advantage of some newer processor features when optimizing the code. The .NET framework with its JIT compiler is able to tell the platform it is on as it compiles the code at runtime. This is not some fantasy, it's just a consequence of ahead of time compilation vs. just in time compilation. Java could do the same thing...

Anonymous on 7/20/2008 3:34:50 PM (79 days ago)

Question from someone who is currently learning C++ about the first comment:
How many people on the planet know Erlang? I've been told that Erlang was like a dead language but apparently you seem to think otherwise. Does anyone use it or is it supported by some group?
Thanks

Adam on 7/20/2008 4:29:22 PM (79 days ago)

check out steve yegge's video/blog post about dynamic language optimization: steve-yegge.blogspot.com/.../...s-strike-back.html

he mentions that Java is now often faster than C++, especially with multicore. overall, the notion is that just in time compiling can be far more intelligent than precompiling because you are at the horse's mouth

he also points out that attempts to similarly optimize C++ would likely have a lot of difficulty with pointers and memory handling stuff. the language has many unmanaged performance things built into it. so old C++ programs are likely screwed. and when writing a new C++ program that could be JIT'd, you'd best not use those performance things because the JIT compiler will probably be smarter than you about optimizing. but then in that case, you might as well use C#

@Adam: Erlang is sorta like lisp in that it's just been under the radar for who knows how long, but it's still used in serious applications

aoeu on 7/20/2008 5:56:43 PM (79 days ago)

I have no idea how many Erlang users there are. I wouldn't pick a language for performance based on it's current popularity. Instead, read the trends. If the core count keeps doubling say every 18 months and our current status quo is approaching 4 cores, in 4.5 years we're looking at 64 cores. Erlang is massively parallel with no extra effort on the developer's part, so if you compare a single threaded C program to the same in Erlang, you start seeing Erlang as something like 64 times faster (yes, probably somewhat less). That same program continues to run faster and faster exponentially every year, while the C program stays bottlenecked on the clock speed. Erlang is open source and maintained by Ericsson. There are other parallel paradigms (Haskell, Clojure) to look into too.

Eric B on 7/20/2008 5:56:54 PM (79 days ago)

Woops...should have been 6 years in my example.

Eric B on 7/20/2008 6:00:26 PM (79 days ago)

Thanks for the response Eric B. If I'm not mistaken Amdahl's Law is controlling on the matter of how much faster the parallel Erlang program would be in 64 cores than the serial C or C++ program.

I do think that there's probably going to be more adoption of more familiar parallel languages in the future, simply because they are easier to learn and incorporate. I've always thought of OpenMP, a language developed on C, as being the most likely language to be adopted though Cilk could probably catch on just as fast too.

I guess I know some people who work with Lisp but it's always struck me as the highly academic, artificial intelligence development language. I've always thought of Erlang as falling into that mold too.

Adam on 7/20/2008 7:00:19 PM (79 days ago)

I'm confused. Sometimes the article says "C", sometimes "C++", and sometimes "C/C++" -- even though these are very different languages. I would think combining "C#/Java" or "Ruby/Python" makes more sense than "C/C++". Or even "Java/C++", which you point out aren't *that* far apart.

I've got my copies of K&R and Stroustrup right here, and they seem quite different. Different language features, different design goals, different strategies for writing programs. One is about five times as big as the other.

Are you saying one is as good as the other? Suggesting we need to learn both, and simply slashing them to save space? Pulling in Spolsky's claim about C and squinting so it applies to C++ as well? I don't see any specific features (processes and threads, stacks and heaps, etc.) that are C++-specific.

Anonymous on 7/20/2008 9:31:38 PM (79 days ago)

>the case for performance is much better made by learning to go parallel like Erlang than by C/C++.

Everything you say about Erlang is true, but it seems likely that the way towards massive parallelism will be paved by reftrofitting existing popular languages to support "Erglang-style" concurrency, rather than a massive future exodus from C#/Java to Erlang.

What Is Erlang-Style Concurrency?

>I'm confused. Sometimes the article says "C", sometimes "C++", and sometimes "C/C++"

And to make it even more confusing, sometimes we say "C++/CLI" ;) I think usually people group these together because C is (almost) a perfect subset of C++. Almost any C program can be compiled as a C++ program. I can learn Java without learning a thing about C#, but I can't learn C++ without also learning C. But there are other ways to pair languages as you point out.

James Devlin on 7/21/2008 12:20:31 AM (79 days ago)

Erlang is the ONLY SMP language. All others must perish.

Anonymous on 7/21/2008 2:07:36 AM (78 days ago)

> That C/C++ is the most performant language in the world, with the exception of assembler itself.

Why do people always think that "low level" means "faster"? This might have been true 20 years ago, but those days are certainly gone.

Sure, you *can* write assembler code that's at least as fast as C++ code. But usually, you won't. Try it yourself if you like, beating the C++ optimizer with hand-written assembly code is really really hard, unless you're using stuff the C++ compiler doesn't know like SIMD instructions. Why do you think that's the case? Because C++ is on a higher level of abstraction than assembler, the compiler "knows" more about your code, so it can transform it to optimize it for current CPUs in ways a dumb assembler can't. But there are still lots of things it can't do, because C++ is still quite "low level": you still specify exact memory layouts in C++, there's no way to tell if a function has side-effects, is thread-safe, etc. Programming in a language that's on an even higher level than C++, the optimizer could transform your code a lot more, thus make it even faster than typical C++ code (in the same sense that typical C++ code is faster than typical assembler code.)

The main reason why C/C++ compilers yield the high performance they do is because compiler writers spent the last 30+ years perfecting their optimizers for C++. But as the .NET VM and the JVM get more and more mature, we can expect them to produce more efficient code, too.

(That said, I agree with your main point: In order to be a good programmer, you should have some real experience with C, C++ and assembler, for all the reasons you gave.)

Niki on 7/21/2008 9:25:09 AM (78 days ago)

In fact, language has nothing to do with if you are a good programmer or not. C++ is so 90-s - and I would say Java/C# are more efficient if you want have things done.

Now if you want to be something more than a lamer, or a coder - learn algorithms. It is so much more worthwhile to learn algorithms and techiques, than to learn C++ just for the sake of it.

P.S. Java deal with threads, sockets and a lot of low level stuff. It just abstract away the mostly useless stuff, which you will rarely need for any job. It sucks at GUIs, where .NET is better thou.

Indiana on 7/21/2008 1:58:19 PM (78 days ago)

>The main reason why C/C++ compilers yield the high performance they do is because compiler writers spent the last 30+ years perfecting their optimizers for C++. But as the .NET VM and the JVM get more and more mature, we can expect them to produce more efficient code, too.

Indeed, in some cases they (JIT compilers) already produce more efficient code. But a managed application's working set is larger and that imposes additional overhead which can't necessarily be optimized away...

Now if you want to be something more than a lamer, or a coder - learn algorithms. It is so much more worthwhile to learn algorithms and techiques, than to learn C++ just for the sake of it.

I agree. Algorithms are huge, but we still have to learn stuff like:

Foo* pFoo = &fooObject;

So we know what a particular implementation is doing when it takes the address of something and tucks it away, or what an iterator is under the hood.

James Devlin on 7/22/2008 5:08:17 AM (77 days ago)

For those interested, some more discussion of this this article over at Hacker News:

http://news.ycombinator.com/item?id=251584

Anonymous on 7/22/2008 5:31:19 AM (77 days ago)

pfft! here in the UK no one drives automatics, everyone learns using a manual transmission (unless you're special) doesn't mean we're all car mechanics though....

Anonymous on 7/22/2008 6:05:30 AM (77 days ago)

I started out using C++ and now write in mostly .NET languages (C# and VB.NET) I have to say that learning C++ first has made everything so much EASIER in the .NEt world

BC on 7/22/2008 9:08:52 AM (77 days ago)

I came across this article over at http://joel.reddit.com. This sort of thing has been stated before (as you prolly know) but I like your phrasing and agree, C++ is a seemingly retro but powerful way of bolstering your .NET skills. Anyway, cheers~

Anonymous on 7/23/2008 2:42:48 AM (76 days ago)

Yeah, learning C is useful is you want to write operating systems or compilers, probably the nearest equivalent of being an auto mechanic. Used to be necessary for race drivers as well, but not any more.

When you specify your program at such a low level, you don't leave the compiler much scope to optimise your high-level intentions. As you point out, the days of single-CPU systems seem numbered, and non-declarative languages just aren't going to have the scope to be optimised for massively multi-core processors. Languages like C will be out of the performance race, but still the necessary lingua franca for mechanics. Learn it so you can _talk_ to your mechanic and understand his diagnoses - and his bill!

Greg M on 7/23/2008 3:21:47 AM (76 days ago)

I have a question that relates back to the foldbot. In the code you hard coded the X,Y coords to push the Fold button. My question is how did you came up with that? I looked at Spy++ but I didn't see much there that I could use. Was it a program you used or trial and error? Any help would be apprecaited!

Adam on 7/23/2008 11:17:34 AM (76 days ago)

C/C++ are *hardly* the fastest non-machine languages. Languages like FORTRAN have been around for centuries (it seems) and for most cases, are much faster than C (simpler call semantics for one). Even super high-level languages like Haskell have been known to produce code which is significantly quicker than even C (due to aggressive optimizations allowed by a pure-functional language). Also, as has been pointed out several times in the comments already, JIT languages can be significantly faster than AOT.

I seem to have misplaced the link, but a well-known developer did a benchmark recently of seven or eight languages, comparing them on a real-world level. It was still a microbenchmark, but the use-case was drawn from his customer's needs, rather than some contrived ray-tracer. The results showed Java *way* out front - ahead of C++ by quite a margin. I checked the code too, it's not like the C++ was poorly written or anything like that. In fact, all of the language-specific ports were written in idiomatic, performant style. For example, the C++ version used pointer arithmetic and judiciously managed the heap.

I don't challenge that C would likely beat out Java in almost every test, it is a far less complicated language than C++, and certainly less difficult to optimize. However, saying that C/C++ are the *fastest* languages is ignoring the bulk of research turned out in the last fifty years.

Daniel Spiewak on 7/23/2008 1:48:01 PM (76 days ago)

To Ed. K.'s post on 7/20:
Those benchmarks are from 2004.

In general:
I think this article got a little sidetracked on it's points at times. The theme of the article seemed to be whether C/C++ made you a better managed programmer. The questions of performance, etc. is a design decision that's irrelevant to this question.

My 2 cents, after having just changed jobs from C# to C/C++ is that doing the C/C++ has made my coding in C# on the side MUCH better. C/C++ forces you to pay more attention to what you're doing.

ZagNut on 7/24/2008 11:27:45 AM (75 days ago)

Daniel -

I was going to make your point about Fortran, but ... much faster in most cases? Really? No. These days, in most cases, C will get you within about 90% of hand-tuned assembly. Fortran will get you within about 90%. C++ will get you within about 90%. It doesn't usually matter that much.

In particular, it doesn't matter that much between procedural languages, because (a) compilers are pretty good in most cases, so long as they're sufficiently mature and have had enough effort put into the quality of the code generation, and (b) they pretty much use the same internal representations no matter what language they're compiling. There are small differences, such as things that you can express in Fortran that you can't express in C, but those are pretty trivial for performance in most cases.

(Sure, there are pathological cases, but those are generally rare with anything that's had a decade to age and doesn't involve vectorization or similar weird processor instructions, and they tend to be rather dependent on platform.)

Brooks Moses on 7/24/2008 6:02:29 PM (75 days ago)

The analogy is flawed for most parts of the world as automatic is AFAIK only predominant in the US.
In Europe most cars are stick shifted, but I don't think that makes the drivers more "car aware" (side note).

Jonas on 7/27/2008 4:07:41 AM (72 days ago)

I agree that one should learn how to program in C, even though I disagree with the stick shift analogy (I'd rather liken it with an old car - one that still had serviceable parts inside Wink

It's a good idea to learn Erlang in order to play around with advanced concurrency modeling without sacrificing your sanity.

Ulf Wiger on 7/27/2008 4:57:15 PM (72 days ago)

"And he'll know why, when he tries to allocate and then start accessing an array with 2 million elements, the hard drive churns even though the machine has 4GB of memory."

Out of curiosity: why would the hard drive churn? 2 million elements, say doubles, would require 16 MB of memory. I would assume that there will be 16 MB free memory available on such a machine, so swapping shouldn't be necessary, I'd say? What did I miss there?

oliver on 8/1/2008 9:33:41 AM (67 days ago)

C & C++ are not different languages, C++ just added stuff to C. The first C++ compilers I worked with were just preprocessors for existing C compilers (Glockenspiel etc). In fact one of the advantages of learning C++ for me was that when I went back to C and Assembler (for embedded projects) I started using OOP techniques in these languages.

By the way, compiled C/C++ code is always faster than any interpreted language. We are sometimes fooled by the apparent speed of Java/C# applications because, of course, these applications spend a large proportion of their time executing code in their run-time libraries which has often been optimized extensively (and are often running native code, in the OS, or hardware assisted graphics cards etc). This is fine for most of the time but will fail you when your own application code has to do some serious heavy lifting.

JudgeD on 9/1/2008 11:03:01 AM (36 days ago)

Comment on this post:

Thanks for your interest in Coding the Wheel. All fields are optional.