My Favorite Programming Mistakes

Advertisement

Over my programming career, I have made a lot of mistakes in several different languages. In fact, if I write 10 or more lines of code and it works the first time, I’ll get a bit suspicious and test it more rigorously than usual. I would expect to find a syntax error or a bad array reference or a misspelled variable or something.

I like to classify these mistakes into three broad groups: cock-ups (or screw-ups in American English), errors and oversights. A cock-up is when you stare blankly at the screen and whisper “Oops”: things like deleting a database or website, or overwriting three-days worth of work, or accidentally emailing 20,000 people.

Mwnt beach
Coastline near Mwnt on the west coast of Wales. Read on to find out why this is halfway to being a very special place.

Errors cover everything, from simple syntax errors like forgetting a } to fatal errors and computational errors. When an error is so subtle and hard to find that it is almost beautiful, I would call it an oversight. This happens when a block of code is forced to handle a completely unforeseen and very unlikely set of circumstances. It makes you sit back and think “Wow”: like seeing a bright rainbow or shooting star, except a bit less romantic and not quite as impressive when described to one’s partner over a candlelit dinner.

This article discusses some of the spectacular and beautiful mistakes I have made, and the lessons learned from them. The last three are my favorites.

Leaving Debug Mode On

The first two mistakes in this article were full-fledged cock-ups.

When I first started freelancing, I wrote a set of PHP libraries for handling database queries, forms and page templating. I built a debugging mode into the libraries at a fairly deep level, which depended on a global variable called $DEBUG.

I also kept a local copy of every major website I worked on, for developing, debugging and testing. So, whenever a problem occurred, I could set $DEBUG=1; at the top of the page, and it would tell me various things, such as all the database statements it was running. I rarely used this debug method on live websites; it was for local usage only.

Except for one day when I was working late at night, debugging a minor problem on a popular e-commerce website. I put $DEBUG=1; at the top of several pages and was switching between them. It was all a tired midnight blur, but in the end I somehow added the debugging variable to the most important page on the website, the one after the user clicks “Pay now,” and I uploaded it to the live website.

The next morning, I went out early for the whole day. I got home at 9:00 pm to find 12 increasingly frustrated messages on my answering machine and a lot more emails. For about 20 hours, whenever a customer clicked pay, they saw something like this:

screenshot

What customers saw when they clicked “Pay.”

It took me about 10 seconds to fix, but a lot longer to apologize to my client for a day’s worth of lost orders.

Lessons Learned

I held an internal inquiry into this issue and established the following:

  1. Avoid working late at night;
  2. Make a full test order whenever I make a change to the order processing, however minor;
  3. Make sure debug statements never see the light of day on a live website;
  4. Provide some emergency contact details for me and/or a back-up programmer.

Thoughtful Debugging

For the third requirement, I implemented a couple of functions like this, to make sure that debugging messages are outputted only when I am looking at the website:

function CanDebug() {
 global $DEBUG;
 $allowed = array ('127.0.0.1', '81.1.1.1');
 if (in_array ($_SERVER['REMOTE_ADDR'], $allowed)) return $DEBUG;
 else return 0;
}
function Debug ($message) {
  if (!CanDebug()) return;
  echo '<div style="background:yellow; color:black; border: 1px solid black;';
  echo 'padding: 5px; margin: 5px; white-space: pre;">';
  if (is_string ($message)) echo $message;
  else var_dump ($message);
  echo '</div>';
}

Then, whenever I want to output something for debugging, I call the Debug function. This calls CanDebug to check the requesting IP address and the $DEBUG variable. The $allowed array contains my IP address for local testing (127.0.0.1) and my broadband IP address, which I can get from WhatIsMyIPAddress.com1.

Then I can output things like this:

$DEBUG = 1;
Debug ("The total is now $total"); //about a debugging message
Debug ($somevariable); //output a variable
Debug ("About to run: $query"); //before running any database query
mysql_query ($query);

And I can be confident that no one but me (or anyone sharing my IP address, such as my boss) will ever see any debugging messages. Assuming that the variables above were set, the above code would look like this:

screenshot

Outputting debugging statements.

For extra safety, I could have also put the error messages inside HTML comments, but then I would have had to sift through the HTML source to find the bit I was looking for.

I have another related useful bit of code that I can put at the top of a page or configuration file to ensure that all PHP notices, warnings and errors will be shown to me and only me. If the person is not me, then errors and warnings will be outputted to the error log but not shown on screen:

if (CanDebug()) {ini_set ('display_errors', 1); error_reporting (E_ALL);}
else {ini_set ('display_errors', 0); error_reporting (E_ALL & ~E_NOTICE);}

Debuggers

The method above is useful for quickly finding errors in very specific bits of code. There are also various debugging tools, such as FirePHP2 and Xdebug3, that can provide a huge amount of information about a PHP script. They can also run invisibly, outputting a list of every function call to a log file with no output to the user.

Xdebug can be used like this:

ini_set ('xdebug.collect_params', 1);
xdebug_start_trace ('/tmp/mytrace');
echo substr ("This will be traced", 0, 10);
xdebug_stop_trace();

This bit of code logs all function calls and arguments to the file /tmp/mytrace.xt, which will look like this:

screenshot

Contents of an Xdebug stack trace showing every function call.

Xdebug also displays much more information whenever there is a PHP notice, warning or error. However, it needs to be installed on the server, so it is probably not possible in most live hosting environments.

FirePHP, on the other hand, works as a PHP library that interacts with an add-on to Firebug, a plug-in for Firefox. You can output stack traces and debugging information directly from PHP to the Firebug console — again, invisible to the user.

For both of these methods, a function like CanDebug above is still useful for making sure that not everyone with Firebug can view the stack traces or generate big log files on the server.

Turning Debug Mode Off

Debugging emailing scripts is more involved. Definitively testing whether a script is sending an email properly is hard without actually sending the email. Which I once did by mistake.

A few years ago, I was asked to create a bulk emailing script to send daily emails to over 20,000 subscribed users. During development, I used something similar to the CanDebug function above, so that I could test the emailing script without actually sending an email. The function to send emails looked something like this:

function SendEmail ($to, $from, $subject, $message) {
  if (CanDebug() >= 10) Debug ("Would have emailed $to:n$message");
  else {
    if (CanDebug()) {$subject = "Test to $to: $subject"; $to = "test@test.com";}
    mail ($to, $subject, $message, "From: $from");
  }
}

If I set $DEBUG=1, it would send the emails (all 20,000 of them) to a test address that I could check. If I set $DEBUG=10, it would tell me that it was trying to send an email but not actually send anything.

Soon after launch, a problem arose with the script. I think it ran out of memory from doing some inefficient processing 20,000 times. At some point, I went into fix something, forgot to set my $DEBUG variable (or else my broadband IP address had inconveniently changed) and mistakenly emailed 20,000 people.

I apologized to the agency I was working for, but thankfully nothing much came of it. I guess that spam filters blocked many of the messages. Or perhaps the recipients were merely pleased that the email did not contain anything for them to do or read.

Lessons Learned

I was very glad that I just put “test” in the subject and message of the test email, and not some statement reflecting how frustrated I was getting at that particular bug. I learned a few lessons:

  1. Be extra careful when testing bulk emailing scripts — check that the debug mode is working.
  2. Send test emails to as few people as possible.
  3. Always send polite test messages, like “Please ignore, just testing.” Don’t say something like “My client is a ninny,” in case it gets sent to 20,000 unsuspecting investors.

PHP Blank Page

Now we’re in the realm of hard-to-spot errors, rather than cock-ups. If you’d like to see a hard-to-debug error in PHP, bury the following somewhere deep in your code:

function TestMe() {TestMe();}
TestMe();

Depending on the browser and the server’s Apache and PHP versions, you might get a blank page, a “This Web page is not available,” a fatal error due to running out of memory, or the option to “Save” or “Open” the page, like this:

screenshot

Infinite recursion, as dealt with by Firefox 3.6.

It basically causes infinite recursion, which can cause a Web server thread to run out of memory and/or crash. If it crashes, a small trace may or may not be left in the error log:

[Mon Jun 06 18:24:10 2011] [notice] child pid 7192
  exit signal Segmentation fault (11)

But this gives little indication of where or why the error occurred. And all of the quick debugging techniques of adding lines of output here or there may not help much, because as long as the offending code gets executed, the page will seem to fail in its entirety. This is mainly because PHP only periodically sends the HTML it generates to the browser. So, adding a lot of flush(); statements will at least show you what your script was doing immediately before the recursive error.

Of course, the code that leads to this error might be much more convoluted than the above. It could involve classes calling methods in other classes that refer back to the original classes. And it might only happen in certain hard-to-duplicate circumstances and only because you’ve changed something else somewhere else.

Lessons Learned

  1. Know the locations of error log files, in case something gets recorded there.
  2. This is where stack-tracing debuggers such as Xdebug can be really handy.
  3. Otherwise, set aside plenty of time to go through the code line by line, commenting out bits until it works.

Wrong Variable Type

This error often happens with databases. Given the following SQL statements…

CREATE TABLE products (
  id INT PRIMARY KEY AUTO_INCREMENT,
  name VARCHAR(60),
  category VARCHAR(10),
  price DECIMAL(6,2)
);
INSERT INTO products VALUES (1, 'Great Expectations', 'book', 12.99);
INSERT INTO products VALUES (2, 'Meagre Expectations', 'cd', 2.50);
INSERT INTO products VALUES (3, 'Flared corduroys', 'retro clothing', 25);

… can you guess what is returned when you run the following?

SELECT * FROM products WHERE category='retro clothing';

The answer is nothing, because the category column is only 10 characters long, and so the category of the last product is cut off at retro clot. Recently edited products or new menu items suddenly disappearing can create a lot of confusion. But fixing this is generally very easy:

ALTER TABLE products MODIFY category VARCHAR(30);
UPDATE products SET category='retro clothing' WHERE category='retro clot';

screenshot

The category has been cut off after 10 characters, as shown in phpMyAdmin.

I made a more serious error with the first major e-commerce website that I worked on. At the end of the ordering process, the website would ask the customer for their credit card details and then call a Java program, which would send a request to Barclays ePDQ system to take the payment. The amount was sent as the number of pence. Not being very familiar with Java, I based the code on an example I found, which represented the total as a short integer:

short total;

The Java program was called on the command line. If it returned nothing, then the transaction was considered successful, emails were sent, and the order was fulfilled. If there was an error in processing the card, the program returned something like “Card not authorized” or “Card failed fraud checks.”

Short integers can store a value between -32768 and +32767. This seemed plenty to me. But I neglected that this was in pence, not pounds, so the highest possible total was actually £327.67. And the really bad news was that if the amount was higher than that, then the Java program simply crashed and returned nothing, which looked exactly like a successful order and was processed as normal.

It took a few months and several large unpaid transactions before the error was spotted, either by the accounting department or a vigilant and honest customer. I believe they recovered all of the payments in the end.

Lessons Learned

  1. When assigning a type to a database column or variable, be generous and flexible, and try to plan ahead.
  2. Make sure that a program succeeding responds differently to a program crashing.

1p Errors

Among my favorite mistakes are those that cause a discrepancy of just 1 pence (or cent, öre or other denomination). I like them because they are usually very subtle and hard to trace and often boil down to a rounding error. I have to become a mathematical detective, a job that I would readily do if enough work was available.

For a website a few years ago, I needed to create a quick JavaScript function to output a monetary amount. I used this:

<script type="text/javascript">
function GetMoney (amount) {return Math.round (amount * 100) / 100;}
</script>

However, it was quickly discovered that amounts like 1.20 were displayed as 1.2, which looks unprofessional. So, I changed it to this:

<script type="text/javascript">
function GetMoney (amount) {
  var pounds = Math.floor (amount);
  var pence = Math.round (amount * 100) % 100;
  return pounds + '.' + (pence < 10 ? '0' : '') + pence;
}
</script>

The main difference is the extra 0 in the last line. But now that the pence is computed separately, the modulus % operator is needed to get the remainder when the amount is divided by 100. Try to spot the unlikely circumstances under which this code would cause an error.

It happened on a website that sold beads. I have since learned that beads can be sold in a huge range of amounts and configurations, including customized mixes containing fractional quantities. Once, a customer bought 1.01 of an item costing £4.95, and ended up paying just £4.00. This is because the amount was passed as 4.9995. The rounded pence was 100, and % 100 left 0 pence, and so the pounds were floored to 4.

screenshot

A subtle rounding error, where 101 beads sold at £4.95 per 100 were billed as £4 instead of £5

This is still just a rounding error, a superset of 1p errors. I made a quick change to fix it:

<script type="text/javascript">
function GetMoney (amount) {
  var pounds = Math.floor (amount);
  var pence = Math.floor (amount * 100) % 100;
  return pounds + '.' + (pence < 10 ? '0' : '') + pence;
}
</script>

This wasn’t a great fix, though, because it rounded £4.9995 down to £4.99, which put it out of sync with any corresponding server-side calculations. But even more dramatically, when someone ordered 0.7 of something costing £1.00, it ended up displaying 69p instead of 70p! This is because floating-point numbers like 0.7 are represented in binary as a number more like 0.6999999999999999 (as described in a recent Smashing Magazine article4), which would then be floored to 69 instead of rounded up to 70.

This is a true 1p error. To fix this, I added another rounding at the beginning:

<script type="text/javascript">
function GetMoney (amount) {
  var pence = Math.round (100 * amount);
  var pounds = Math.floor (pence / 100);
  pence %= 100;
  return pound + '.' + (pence < 10 ? '0' : '') + pence;
}
</script>

Now, I had four fairly complicated lines of code to do one very simple thing. Today, while writing this article, I discovered a built-in Javascript function to handle all of this for me:

<script type="text/javascript">
function GetMoney (amount) {return amount.toFixed (2);}
alert (GetMoney (4.9995) + ' ' + GetMoney (0.1 * 0.7));
</script>

Discounting With PayPal

PayPal is a 1p error waiting to happen. Many websites offer voucher codes that give a percentage off each order, computed at the end of the order. If you ordered two items costing 95p, the subtotal would be £1.90, and you would receive a 19p discount, for a total of £1.71.

However, PayPal does not support this type of discounting. If you want PayPal to display the items in your shopping basket, you have to pass each one separately with a price and quantity:

<input name="item_name_1" type="hidden" value="My Difficult Product" />
<input name="amount_1" type="hidden" value="0.99" />
<input name="quantity_1" type="hidden" value="1" />

Thus, you have to discount each item separately. 10% off of 95p leaves 85.5p. PayPal doesn’t accept fractional amounts, so you have to round up to 86p, for a grand total of £1.72 in PayPal, or round down to 85p, for a total of £1.70.

To solve this, I had to also make the website discount each item individually. Instead of just doing 10% × £1.90, it accumulates the discount item by item, using a whole amount of pence each time. Assuming $items is a PHP array of order item objects:

$discount = 0; $discountpercent = 10;
foreach ($items as $item) {
 $mydiscount = floor ($item->price * $discountpercent) / 100;
 $item->priceforpaypal = $item->price - $mydiscount;
 $discount += $mydiscount * $item->quantity;
}

Lessons Learned

  1. Don’t reinvent the wheel, even very small wheels that look easy from the outside.
  2. If you get a 1p discrepancy, check where and how numbers are rounded.
  3. Avoid representing prices using floats when possible. Instead, store the pence or cents as integers; and in databases, use a fixed-point type like DECIMAL.

Daylights Savings

I would not call the last two mistakes in this list “errors.” They require a very specific set of fairly rare circumstances, so they are more “oversights” on the programmer’s part. Oversights are like the acts of terrorism that are excluded by home insurance policies. They go beyond what a programmer could reasonably be expected to think of in advance.

Can you guess what is wrong with the following seemingly innocuous line of code, which selects orders that were completed more than one week ago?

mysql_query ("SELECT * FROM orders WHERE completeddate < '" .
  date ('Y-m-d H:i:s', (time() - 7 * 86400 + 600)) . "'")

I used a similar line in a system for a weekly repeating order. It looked up orders that were completed last week, duplicated them, and processed them for the current week. 86,400 is the number of seconds in a day, so time() - 7 * 86400 was exactly one week ago, and +600 gives it a leeway of 10 minutes.

This was a low-budget method of implementing repeating orders. Given more time, I would have created a separate table and/or shopping basket to differentiate between repeating and non-repeating items. As it happened, this code worked well for several months and then mysteriously failed in late March.

It took ages to recover from the oversight and to process those orders manually. And even longer to find the reason, especially because I had to fool the whole website into thinking that it was a different date.

I’ve pretty much given the trick away in the title of the section: I forgot to account for daylight savings, when one week is less than 7*86400 seconds.

Compare the following three ways of getting the date exactly one week ago. The last is the most elegant. I only recently discovered it:

$time = strtotime ('28 March 2011 00:01');
echo date ('Y-m-d H:i:s', ($time - 7 * 86400)) . '<br/>';
echo date ('Y-m-d H:i:s', mktime (date ('H', $time), date ('i', $time), 0,
  date ('n', $time), date ('j', $time) - 7, date ('Y', $time)));
echo date ('Y-m-d H:i:s', (strtotime ('-1 week', $time))) . '<br/>';

Lessons Learned

Drawing general lessons from a mistake like this is difficult, but there is a specific lesson here:

  1. On websites that repeat things, remember to consider time zones and daylight savings.
  2. Consider storing all times and dates in UTC5 (Coordinated Universal Time).
  3. Don’t reinvent the time wheel either: strtotime is a powerful function.

The next time I do a website for repeating orders, I won’t make that mistake.

Spam Error

My favorite mistake of all time is an even subtler oversight. Can you spot what is unusual about these made-up email addresses:

  • beckyrsmythe@somewhere.com
  • glynnfrenk@someplace.co.uk

A few years ago, spammers began targeting contact forms on websites, injecting headers6 and forcing the forms to send millions of messages to harvested addresses and later just to the form’s usual recipient.

This necessitated anti-spam filtering directly on the Web page that processed the form. When I was first asked to do this, I combined a few anti-spam scripts7 that I found on the Internet. Spammers now often put blocks of random letters in their messages to try to fool spam filters. So, one anti-spam technique is to check for these random letters by looking for certain consonants in a row.

I read somewhere that words with more than six consonants in a row are extremely rare in Latin-alphabet languages. The most consonants in a row in English is six: in “latchstring.” Other languages like Polish have many more diphthongs than English (dz, sz, cz), so I used seven to be on the safe side. The PHP code uses a regular expression and looks something like this:

foreach ($_POST as $key=>$val) {
        if (preg_match ('/[bcdfghjklmnpqrstvwxyz]{7,}/i', $val))
                die ("<h1>Spam Detected</h1><p>Too many consonants in $val</p>");
}

I had to revisit the script when it blocked someone with an email address like the ones above:

A customer whose email address had seven or more consonants in a row would have received this upon submitting a form.

Based on a small sample of 10,000, I found that approximately 0.2% of all email addresses would be filtered as spam, according to the rule above. One valid email address had nine consonants in a row. Increasing the number of allowed consonants from seven to ten decreases the script’s usefulness significantly, so instead I considered the letter “y” a vowel.

This worked well, until a customer from Cwmtwrch near Swansea attempted to place an order. According to my sample, only 1 in 5000 customers have a name, email or address like this. Small but important, especially if you are one of them. So, I allowed “w” as a vowel, too. You can check for this in your own customer database with a MySQL query like the following:

SELECT CONCAT_WS(' ',firstname,lastname,email,city,address1,address2) AS thefields
FROM visitors HAVING LENGTH(thefields)>20 AND thefields RLIKE '[bcdfghjklmnpqrstvwxz]{7,}'

Lessons Learned

I learned that my anti-spam script was blocking potential customers only once my client forwarded me their complaints. When I received the first one (an email address containing a couple of “y”s for vowels), I was amazed. It seemed so unlikely. A couple of weeks later, when shoppers in a small Welsh village were still mysteriously unable to place an order, I almost didn’t believe it. It seems that if a piece of code has a hole, someone somewhere will fall into it. So, I’ve learned to do the following:

  1. Take all error reports and complaints seriously. They may uncover something amazing like this.
  2. Jot down the really unlikely mistakes. You will impress other programmers… or me, at least

More specifically, logging everything that is processed by a spam filter is useful, because you can then try to spot any false positives or false negatives and use them to improve the filter.

Conclusion

Programming mistakes come in many shapes and sizes. This article has ranged from the very obvious cock-ups to the extremely subtle oversights. And it looks like they all support Murphy’s Law8: if something can go wrong, it will.

However, for every mistake found, reported and fixed, probably a few more aren’t. Either they aren’t found (because they are so incredibly subtle that the set of circumstances that would cause them has never happened) or they aren’t reported (because most users don’t bother reporting errors — which is why any error reports that do come in should be taken seriously) or they aren’t fixed (because doing so would be too time-consuming or expensive).

Mistakes are also more likely to be found on popular websites, mainly because so many more people are putting those websites to work, but partly because fixing one mistake could cause another somewhere else.

The best lessons, therefore, are to plan ahead and to debug thoughtfully.

(kw)

↑ Back to topShare on Twitter

Paul Tero is an experienced PHP programmer and server administrator. He developed the Stockashop ecommerce system in 2005 for Sensable Media. He now works part-time maintaining and developing Stockashop, and the rest of the time freelancing from a corner of his living room, and sleeping, eating, having fun, etc. He has also written numerous other open sourcish scripts and programs.

  1. 1

    Justin ( @yoostin )

    July 7, 2011 4:35 am

    Yes, but what I really want to know is – Was it upper or lower Cwmtwrch? ;)

    Nice read Paul, thanks. Mwnt is a pretty cool place too and isn’t a typo, I’ve been there.

    0
  2. 2

    Very interesting, indeed. Thanks for writing this article :) !

    0
  3. 3

    What a great read! I appreciate the opportunity to learn from your mistakes in such a fun way. It’s amazing how simple errors like these pop up.

    0
  4. 4

    I use more than oops! lol when i made a mistake, but is part of our daily work. great article!

    0
  5. 5

    Thanks for the read. The rounding thing has caught me out a few times in the past when working with financial systems.

    0
    • 6

      You shouldn’t be using floats at all in financial application. You should use fixed point calculations (base everything on cents (or tenths of cents); so $1.29 == 129 and let your _presentation layer_ format it). The only “middle ground” is if your platform supports a datatype like “currency” of “money” which usually is able to work with up to 4 decimals without rounding errors.

      0
      • 7
        • 8

          That’s why COBOL still works best for finance – 38 decimal digit arithmetic. No rounding errors!

          0
          • 9

            38 decimals you say? Then yes, you can still have rounding errors. Granted, they’re not as serious as using standard 32 bit floating points numbers, but you will still get rounding errors, even if they’re oh-so-tiny.

            0
      • 10

        I’ve had similar issues with Paypal, where the client was uploading prices inclusive of VAT. Customers from outside the EU don’t pay VAT, so their prices were reduced to the ex-VAT price. For some numbers, e.g., 39.99, the ex-VAT price has to be accurate to three d.p. in order that when VAT is added the inc-VAT price becomes 39.99.

        0
  6. 11

    I bet I am not alone in this and missed it in the article:

    If a Boolean (If then) test requires more than one check then I almost always get the logic wrong.

    For example: If (this and that) or (those) then… might actually be If (this OR that) AND those then…

    It takes several tests to get it just right, and then of course the user will try the one you didn’t test for right out of the gate.

    0
    • 12

      The user will ALWAYS find the one that is wrong, and typically they end up finding it first as well… haha oh well.

      0
    • 13

      Always group boolean operations to make your intentions clear and to force (e.g. ensure) your intended “precedence”:

      if ((foo==bar) && ((a==b) && ((c!=d) || (e==f)) {

      }

      I can’t format it correctly in this post but in a decent editor you would spread this over several lines and indent the operations.

      0
  7. 14

    With regard to your daylight saving issue and how long it took to “trick” the website, I’d like to share something with you. I too have worked on large projects that do many things depending on the date. Right at the outset, I created a function called Now() which returned a pre-defined INI code if present, or DateTime.Now if it was not set. In normal operations, everything works as expected. Every part of the program used this Now() function rather than DateTime.Now.

    In cases when I had to debug things to do with dates, I could set that ini code to whatever I wanted on my development machine, and hey presto, the entire application operated as if it was that date. Saved me lots of time.

    0
  8. 15

    Russell England

    July 7, 2011 7:31 am

    Funny article… think I have used stronger words than “oops”… ;)

    I’ve made many faux pauxs. In my first year working for Birmingham City Council Housing Department – which was the largest housing department in Europe at the time and had the largest IT department, all on an open plan level. They were still using clock cards when I joined, so I wrote a little program in Clipper for myself to keep track of hours, holidays etc. Being easy to copy, it soon spread around the development team, IT, housing services and most of the neighbourhood offices. In the early stages, just for a giggle, I had put in a flashing Christmas tree (really high tech stuff!! with asterixes as baubles changing colours, whoop whoop) as well a couple of Christmas tunes.

    It was all set to go off on the Monday of the Christmas week…

    Which I had completely forgotten about…

    I was late for work that day…

    When everyone logged into the program to update their clockcards, not only was there a flashing tree but all you could hear was the “Oh Christmas Tree” tune – turned out the Christmas Tree wasn’t very PC… But much worse was the rendition of “Oh Christmas Tree” also known as “The Red Flag”, the old Labour Party “Citizen Smith” theme ringing around the whole housing department and neighbourhood offices.

    Oh how we laughed… I had a tongue in cheek telling off from the senior management with a much more serious how fast can you fix it and reinstall on a few hundred computers… Turned out I was quite quick that day.

    On a more nerdy note, after 15 years of development typing in the word “test” almost everyday, my fingers went slightly left one day and spelt out a word rhyming with swat but without the s. I broke into a fit of hysterical giggles as did my fellow developers. I got a blank look from my wife… You had to be there really… ;)

    0
    • 16

      Clipper!

      I wrote my very first commercial code in Clipper back in 1997. Man, I’m getting old.

      0
  9. 17

    The “Variable Type” problem is just MySQL being terrible. A good DB will throw an error if you try to insert 20 characters into a CHAR(10) field, so you can catch the problem earlier. Also: you’d probably avoid the DST problem if you used the DB’s built-in timestamp arithmetic.

    0
  10. 19

    This article readily sums up why I enjoy programming so much.. It’s all just so limitlessly fascinating. Reminds me of a the time I built an app which was built to ingest a feed which I didn’t have any control over so I built an admin page where I could select from a list of the columns I had in my table for each of the column headers in the feed.. It was a bunch of data.. took a long time to map each piece of data to the table column where it needed to be and then when I was just about done something happened with the local IT which temporarily removed POST data..

    0
  11. 20

    good read. refreshing to hear about the challenges of other programmers and know that everyone makes mistakes.

    0
  12. 21

    Mine: Phone Number field setup as int(10). Guess what happened to area codes > 429? Guess how many area codes other than 416 I tested?

    0
    • 22

      Never, EVER, use an integer to store phonenumbers. Yes, phone-numbers only contain digits but NO, they are NOT numbers!

      My phone number is: 0031 123 456 789. Store it in an integer and then output that number. Spaces might get lost or only the first group of digits might get stored depending on your platform and the way you stuffed the user input in the integer. But even if you manage to get all digits in the integer, you will have lost the first 2 zeroes. Whichs makes a pretty big (huge) difference when dialling that number.

      Also, many people prefer to enter phonenumbers like: +31 (123) 456 – 789. Now store that in an integer. Even if you strip out the brackets, dash and replace a + with 00 then putting the result in an integer will again lose the leading zeroes. It also gets problematic displaying the same number again to the user in a readable way.

      If you are dealing with systems that can only handle digits then store the number 2 times; once as input by the user (so as string, keep all brackets/dashes/plusses etc.) and to be used by the system you stri the number from extra stuff by a regex ([^0-9]) after replacing a + with two zeroes and store that result in a _string_ also. Now you would have:
      field_userinput: +31 (123) 456 – 789
      field_stripped: 0031123456789
      When using the number for dialing or whatever, use field_stripped; when displaying (presentation) the data to the user, use field_userinput.

      0
  13. 24

    Awesome, saving this for future reference.

    0
  14. 25

    You are good writing articles, not code.

    And you have a small error in the code block; after ‘So, I allowed “w” as a vowel, too.’ (the “w” is still in the regular expression).

    I recommend anyone reading this article to take it as for educational purpose only and not to apply these snippets in production environments.

    0
    • 26

      I have to agree; I apologize to the author but a lot of this code and the examples given are pretty “rookie” mistakes and “newbie code”. That _doesn’t_ mean I never made these mistakes or wrote ‘ugly’ code; we _all_ have. But a lot of the code displayed is just plain butt-ugly and a lot of the examples given would’ve been prevented if not coding “cowboy style” (see http://en.wikipedia.org/wiki/Cowboy_coding). Not to mention some “solutions” given here which aren’t _solutions_ but butt-ugly workarounds for simple problems solved many times by many people.

      0
      • 27

        It’s a light and fun article about mistakes – some made years ago when I was a newbie. And usually working by myself, as it was for small clients, but I would not consider myself a cowboy. It was meant to highlight how interesting mistakes can be, not as a guide to programming.

        0
    • 28

      Reading articles like this IS only for educational purposes. They’re not meant to be tutorials but rather a way to share some info about some things that hung up this particular developer at 1 time or another in his career. I see lots of comments like this and RobIII’s across the net and it’s sad.

      You’ve missed the point of the article. Even more so, you sound like the kind of guys I won’t hire. They come into the office claiming to be the the “1″ dev on earth that writes bug free code….all the time. Not having everything spelled out for you before a project starts is just the way of the industry and not cowboy coding.

      If you can’t work until you have every function spelled out for you on paper for every use case, you won’t be very helpful to an organization.

      I thought your article was a fun read Paul.

      0
  15. 29

    Regarding the 1P JavaScript charging the customer the wrong amount… you shouldn’t be doing cost calculations on the client-side! That should only be for display purposes.

    0
    • 30

      Author mentions this himself:
      “which put it out of sync with any corresponding server-side calculations”

      Client-side calculations _aren’t_ bad, as long as they’re backed by a server-side calculation.

      0
      • 31

        Except it doesn’t… the reddit post that links to this page has a comments section where someone pointed a link where you can simply bypass it

        0
        • 32

          Why don’t you post the link? I find it hard to believe you can bypass a server-side calculation. Did you read my comment thoroughly?

          0
  16. 33

    10 lines is not a lot.

    0
  17. 34

    Christopher Smoak

    July 7, 2011 12:03 pm

    I really enjoyed this type of article. I would like to see more like it. It makes you think about doing things before committing them.

    0
  18. 35

    Mistakes?
    That’s child play.
    There’s nothing like dangling pointer crashing production code with violation exception error. Fun for the whole family.

    0
  19. 36

    “Leaving Debug Mode On” part should never be a problem.

    don’t use echo to display debugging messages, that’s all.
    example :
    function decho($msg)
    {
    if ($_SERVER['SERVER_NAME'] == ‘localhost’ || $_SERVER['REMOTE_ADDR']==’INSERT_YOUR STATIC_IP_HERE’)
    {
    echo(” . $msg . PHP_EOL . ”);
    }
    }

    You might want to improve this by using print_r if the parameter type is an object/array, or add a switch to log the messages to a file, etc. But the important thing is : it will never display the message to a user, even if the debug mode is left on.

    0
    • 37

      ok, the PRE tags in the echo have been removed and it should be written without ( ) and commas instead of dots :
      echo ‘pretag’, $msg, PHP_EOL, ‘end_pretag’;
      you get the idea…

      0
      • 38

        You should also escape $msg and I don’t see the need for PHP_EOL:
        echo sprintf(‘%s’, htmlentities($msg));

        0
        • 39

          Damnit… stupid ‘html filter’….

          echo sprintf(‘<pre>%s</pre>’, htmlentities($msg));

          0
        • 40

          you certainly don’t want htmlentity your message or use sprintf !
          for example, if you want to check what you’re inserting into a DB table, you won’t be able to differentiate an é from an &eacute… in other words : to debug properly.
          If you don’t alter the string, you can check the source code of the debug message and see what’s really going on.

          0
          • 41

            Why force the need for a “view source” when you can easily output the correct data in the page itself?

            0
          • 42

            oh yeah, i forgot your pre tag.
            anyway, don’t alter the debug message, don’t use htmlentities or your debug output can’t be 100% reliable.

            0
  20. 43

    I suggest that with regards the email mistake, the biggest mistake was trying to do any sort of filtering on the email at all. You say you did it for spam reasons, yet I think it is a very poor spam check. Not only for the reasons you mention, but also for the fact that there are various valid email addresses which might be used for whatever purpose.

    I suggest that you look up an article called “Stopping spambots with hashes and honeypots” by Ned Batchelder. It provides two key ways to stop spambots from all web forms, without less inconvenience for users than blocking legitimate email addresses.

    (On the subject, for what purpose is the email address field on this form used for? I can not think of a legitimate reason to require the email field to be filled in. Especially as it is too easy to fill in obviously invalid data such as none@example.org.)

    0
    • 44

      Thanks for the article reference – the honeypot idea is a good one I hadn’t come across. Detecting random characters was a suggestion I read elsewhere, and can produce false positives, but it made for an interesting insight (for me at least) into the nature of errors and data. Many forms – particularly for registration and ecommerce – require valid email addresses.

      0
  21. 45

    Using PHP’s date() (or a related) function to create a date string for a MySQL query can lead to problems of its own. I once had to work with a web server a

    0
    • 46

      Damned; some typo’s to be fixed…

      Just use parameterized queries and if that isn’t available/supported by your platform then use ISO 8601 notation (http://en.wikipedia.org/wiki/ISO_8601):

      $query = “insert into `mytable` (`foo`, `somedate`) values (‘bar’, $date(‘Y-m-d H:i:s’)”;

      All databases (SQL Server, MySQL, Postgres, etc. etc.) support that notation and this ensures every system (database, webserver, php/asp/whatever, etc..) is on the same page regarding d/m/y, m/d/y and other i18n/l10n problems (http://en.wikipedia.org/wiki/Internationalization_and_localization).

      0
      • 47

        RAAAAAAAH!!!
        Disappearing quotes for some reason….

        Last try… The query above should read:

        $query = “insert into `mytable` (`foo`, `somedate`) values (‘bar’, ‘$date(‘Y-m-d H:i:s’)’”;

        0
  22. 48

    Using PHP’s date() (or a related) function to create a date string for a MySQL query can lead to problems of its own. I once had to work with a web server and a database server that didn’t have their clocks running in sync — by a long shot! (The reason for that was that they were virtual machines on a VMware server: http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf for those interested.)

    Guess what happened when I inserted NOW() in SQL and used the result from date() to check what was inserted before/after a certain point in time?

    0
  23. 49

    You forgot your mistake number zero: Pushing code directly onto production. Code and configuration go to testing first, then to staging, then to production. All of which are isolated from each other!

    0
  24. 50

    Nico Schneider

    July 7, 2011 6:28 pm

    Nice read, taught me a lot! Thanks, and take care!

    0
  25. 51

    Great write up and I have certainly come across one or two of these. This was a very interesting article and well written. Kept my attention and helped me learn a few things :)

    0
  26. 52

    I love this article! I relate a lot to it. In fact I remember couple years ago buying a sand bag to kick and punch when these beautiful errors ruined my day, because sometimes it may take a whole day and a whole code redo just to realize that it was just right the way it was and it was just missing the “.” as in $(.sortable) or the extra ” = ” as in if($value == 0 ), hahahaha yeah, exactly, wahh!!!! kick!!! punch!!!! steam off, debug done!

    0
  27. 53

    Excellent post, i will be keeping for my future development, really thanks full to writeup ur programming experience. :)

    0
  28. 54

    You make some more programming mistakes:

    - NEVER use mytable.* (wildcard selection)
    - NEVER use global variables!

    0
  29. 55

    My favorite programming mistake was actually not purely a programming mistake. I made some hours of programming modifications but instead of uploading modified file I replaced the modified file with old one.

    So, programming of several hours got disappeared.

    0
  30. 56

    The consonant error was pretty inexcusable. It’s even more shocking that, when confronted with evidence that your “fix” is stupid, you simply loosened the restriction rather than realising it is completely senseless.

    0
    • 57

      Thanks – but this is an article about mistakes. And about how some user out there will always manage to slip through the net.

      0
      • 58

        BlueCollarCritic

        July 25, 2011 7:55 am

        Don’t you just love it when people are more than happy to beat down in a useless and unproductive manner someone who has openly admitted (and detailed ) their mistakes? Constructive criticism is good even if it’s not welcomed but being plain old A-hole is not only unnecessary and rude its unproductive. I can’t believe someone voted up his reply (I voted down to negate it).

        You gave him a very professional reply, something I doubt most would.

        Speaking of constructive, on the item about database data types and being generous with which ones you use keep in mind that you can also over a lot something to and depending on the platform (SQL Server, Oracle, MySQL, ect) you could have just as big a problem or worse by over doing it. While disk space is cheap these days compared to the not too distant past, when it comes to data types in SQL (for tables) you want to shoot for a balanced approach; not to little but not too much either. For the most part its better to side on too much then too little but too much can also cause problems to such as with indexes.

        Kudos for putting your errors on display, most don’t have the guts to do that even though they have plenty to criticize others mistakes.

        0
  31. 59

    I would have to agree with the comment that you are a better writer than coder.

    Calling daylight savings time a “very specific set of fairly rare circumstances” that goes beyond what a programmer “reasonably should be expected to know in advance”……

    Well, changes in DST are 100% predictable and happens every year.

    The rounding errors were quite interesting to read about though.

    0
  32. 62

    My email mistake I send 40k 3 times. First wasn’t a mistake the next 2 where. I used a email client that didn’t show progress and waited and waited and no emails.
    Then did it again and again. Everyone received 3 emails three days later. Total of 120,000 + OOOPS

    Learned that lesson quickly.

    0
  33. 63

    Wow you’re such a knob for leaving the $DEBUG=1 statement. Well, no wonder, you program in PHP… If you were in my team, I would fire you on the spot.

    0
    • 64

      That’s very harsh. This is after all, an article about mistakes. If the article was called “My Favourite Programming Things I Did Absolutely Correctly”, then you might have a point.

      0
      • 65

        I’m very impressed by the way you’ve handled negative comments here. That by itself speaks volumes about your experience, professionalism and maturity. Thank you!

        0
        • 66

          Thank you Shane – I appreciate that a lot.

          0
          • 67

            I have to join Shane Turner – some of these people should try and remember their first mistakes, instead of bashing your examples, when the true point of the post was obviously a more general and valuable one.

            0
  34. 68

    Gordon McLachlan

    July 8, 2011 2:28 pm

    My favourite programming mistake? Gzipping the root of our live production server…

    Suffice to say I f**king shat myself.

    0
  35. 69

    For the PHP debugging thing : just use an environment variable set by Apache in the configuration file (SetEnv). The major frameworks like Zend actually do that.

    0
    • 70

      Exactly!

      I can’t believe the trash he has coded above as a ‘solution’. Shouldn’t we be using environment variables by now?

      0
    • 71

      That’s the solution I use, too. Makes life easier just to check for the presence of the environment variable and if it doesn’t exist, assume it’s production.

      In all fairness, I learned it the hard way, after letting some debugging stuff slip through. At the time, I just turned on the debugging stuff as I needed it. After a couple times of forgetting to remove it, I just have it turned on permanently for development.

      0
  36. 72

    Big like! interesting :)

    0
  37. 73

    Oh happy days. I remember when I was a junior hardware designer back in the early 1980′s our senior programmer was writing a format program for 8″ floppies (- yes I am ~that~ old) in assembler for the 8086
    Of course programs don’t work first time do they? Well actually, yes very occasionally and this was one of those times. He hit run, the head snapped down, chug, chug, chug, chug……whirrr, clunk. He’d formatted the floppy on which his source code resided. A pin dropping would have deafened us all at that point. After what seemed like an eternity of no-ops on our part, he sighed and said with a very resigned tone, “Ah well, at least I have the listing”

    Thanks for the excusion down memory lane.

    0
    • 74

      That reminds me of the time when I was writing one of the very first floppy disk interfaces for the Sinclair QL !!! (lovely 68000 machine code). After some time of working on the code, I started saving all my source code on floppies – until the day that is when I introduced a bug that meant it would not read floppies any more!

      Doh…

      ok guys, you have to remember this was a long time before hard disks and the internet and git. I had to resort to rolling back to old code on Microdrive. Hands up all those that can remember what they were…

      0
  38. 75

    Hi everybody !

    I’ve done a little modification to the debug function… Just see :

    function Debug () {
    if (!CanDebug()) return;
    echo ”;

    foreach (func_get_args() as $message ) :
    if (is_string ($message)) echo utf8_decode($message);
    else var_dump ($message);
    endforeach;

    echo ”;
    }

    It will allowed to write something like that :

    $my_array = array(1, 2, 3, 4, 5);
    debug ( ” My favorite Array is : “, $my_array);

    What do you think about this tip ?

    Allan, your frenchie guy =)

    0
  39. 76

    Thank you for the comments – many of which are appreciative. To the others, I would like to say that this is supposed to be a fun article about mistakes. It is full of things that went wrong, so please read and consider it in that light. For example, client-side item pricing is a bad idea, but I just used it as an example of a certain type of mistake and the lesson learned – not as an example of how to add up prices. In cases like those, there should also be a server-side check.

    0
  40. 77

    nice article thanks for sharing.

    i particularly liked your debug suggestions. i’ve been doing something pretty similar for years, but never thought to color-code the debug output, and checking that the IP is your own before displaying the output is genius!

    well done.

    0
  41. 78

    Mark Urquhart-Webb

    July 12, 2011 1:31 pm

    Embarrassing code error?

    While moonlighting (cash under the table) at my company’s client site I added a quick temp file cleanup script to the crontab. Somehow an aberrant space popped in and when I tested it out, did an ‘rm -rf / tmp’

    … praise the lord for backups. Not proud but it made me stronger.

    0
  42. 79

    In my rookie days I was always fond of the infinite loop, “While x < 15" and then never incrementing x. I'd run my code and wonder why it was so slow.

    0
  43. 80

    Ch. Kaleem Ahmad

    July 13, 2011 5:48 am

    These are pretty small errors and we usually repeat them and do not care for minor mistakes but it’s quite a simple but detailed and focused article and I like it.

    0
  44. 81

    Nice Article Paul.

    You have been very humble to highlight the mistakes, but the most important part is you have learnt from those mistakes and moved on.

    Good Job.
    Prasanna

    0
  45. 82

    Good reading, certainly brought to mind a few of mine .. but a minor issue: Perhaps YOUR DBMS didn’t choke on inserting a character string too large for the column, but Oracle, in general, would have failed the insert.

    0
  46. 83

    debug messages can be extremly useful by development time, but in production it can cause a serious problem. In one of my first projects I’ve used lots of debug logs to make sure everything goes right. However I did not pay attention on the performance effect. The log messages were neatly created eventhough they were not written as log level was set to warning.
    After cleaning up logs the customer still complained on performance. The next found was usage of “sleep”. In the early versions decided not to bother on synchronisation issues so put a “TODO: clean it later” comment and a sleep instead of proper code. Of course at last under time pressure the “TODO” was forgotten in code…
    Another one: happened to have a bigger project in old FORTRAN code, among others there was an easy task to achive with copy-paste and a little change, however it was not obvious how to test it. It went to production without test. The carefully coded and tested parts all run Ok for quite a while, the easy part – the clean up that was called later failed – a goto statement were copied without changing the label and caused an endless loop. Facit: goto is devil, copy-paste is devil, but checking-in untested code is the biggest sinn.

    0
  47. 84

    Nice write up! :-)

    Hopefully I’ll avoid these mistakes and make a favorite list of my own! ;-)

    0
  48. 85

    Nice and well layed out article. I am not much into web programmming but still liked it. Thanks for awakening.

    0
  49. 86

    Thanks for being so honest about your mistakes. They show us how when we try sometimes to cure the 99% problem (eg, the spam/email filter), it really results in a total breakdown.

    Just yesterday I was commenting my code when I realised that I had used the word ‘probably’.

    I decided that such words indicate maybe I needed to do a bit more validation on that part of the program!

    My favourite coding mistake is still the semicolon after the for loop..

    for( i=0; i<100;++i);{
    helloWorld(i);
    }

    Getting a job as a coding tutor revealed a lot about simple mistakes, too!

    Joe

    0
  50. 87

    Reading this stresses me out.

    0
  51. 88

    My favourite.. a backup copy that looks so like my working copy…
    Those hours wasted debugging on the wrong copy and testing another one!..

    0
  52. 89

    I don’t have a fixed IP address so I had to come up with another solution for enabling/disabling debugging. I currently have several methods of enabling debug:

    - SERVER_ADDR is my development machine (obviously)
    - HTTP_USER_AGENT contains a special 20 random character string that noone else will have
    - A cookie that I can set through a special password protected script on the server

    0
  53. 90

    We were recently bit on the backside by an INT oversight: Our database INT can store 1.5 billion numbers, but our programming platform INT acts like a database SMALLINT – with 32,768 numbers. It works fine to use INTs for storing table IDs on both when the database is young, but watch out when the first table reaches 32,769 rows! You will be retrieving on table ID -32767. It won’t necessarily bomb – the query just won’t find a matching row!

    What that looks like to the programmer is “That app has been deployed for a year with no problems – if it’s failing now it has to be a problem. The program has been working fine, and we haven’t touched it!”

    We decided that for developing purposes, we will seed our transaction tables to begin at 33,000 to avoid such oversights in the future.

    0
  54. 91

    * insert it has to be a problem with the network, or a master table or user error… This comment box deleted everything I placed inside angle brackets!

    0
  55. 92

    Excellent article, well-written, candid, and personal, with decent graphics and disarming honesty. It was refreshing, and a delight to read. I especially appreciated the clarity of your writing and presentation, and the Lessons Learned for each level.

    Some of us noticed that not once did you seek corrections to your code, or validation as a programmer supreme; many did not.
    Pity.

    Thanks Paul – I really enjoyed the time I spent reading this.

    gh

    0
  56. 93

    I find it utterly shocking that this garbage was posted on smashingmagazine! I don’t know what is more pathetic, your mistakes or your solutions NOW in 2011 given what you currently know (your solution to the first mistake STILL involves using a global!?).

    Seriously, you need to learn some more before you start writing ‘expert’ articles because you are most definitely not an expert.

    0
    • 94

      Damien, are you trolling just for fun or what’s your deal? Read the title of the article. This is not a “Use this code and you’ll have no problems ever” tutorial. It is a relaxed article about lessons learned throughout his programming career and some basic logic solutions to overcome the problem at the time.

      If you’re an expert programmer, why are you even reading this? Many people reading this are probably self-taught independent developers. They don’t have a Masters in Computer Science or have experience working on large development teams where there are tools, money, and job roles in place to ensure everything goes smoothly.

      I highly doubt you’ve never made a stupid mistake before, but if not then I apologize. Enjoy your perfect life and your high horse that gets you to work every day.

      0
  57. 96

    Hey thats why us programmers make a lot of money, its too easy to make mistakes and after a while we become good at debugging and we get paid for it :)

    0
  58. 97

    About 1p problem, clients should use 5$ instead of 4.5$, it will be easier for everyone: consumers, coders, clients.

    0
  59. 98

    Very interesting. Thanks for posting, Paul!

    0
  60. 99

    It is common for web designers/javascript kiddies to not know how to use floating point.
    It’s not a mistake, but a requirement for your kind.

    0
  61. 100

    I was thinking about the Spam Error information above, and why would you just not set up a captcha form for the 0.02% of people that have messed up email addresses… seems like an elegant solution to the problem… well until spammers break the captcha form, but still! better being proactive than reactive.

    0
  62. 101

    Stupidest coding mistake?

    Using == instead of $= for comparing strings.
    I was programming for a game, making a script that prevented people from connecting more than one of themself to the game, and I was comparing names, and then I uploaded it to the server, and activated it. Unbenounced to me, I had forgotten that comparing two strings in this particular language with == always returned true. So, someone connected.
    Then it kicked everyone but that person out of the game. >.>
    ONE charachter completely screwed it up. Stupidest mistake I ever made in programming.

    0
  63. 102

    «Other languages like Polish have many more diphthongs than English (dz, sz, cz)»

    s/diphthongs/digraphs/

    0
  64. 103

    Cristi Constantin

    October 19, 2012 5:38 am

    Hi, in case you have an error that you are trying to find in your code, here is an article I wrote as a general guideline for finding it:
    http://efficient-work.blogspot.ro/2012/08/find-error.html
    Feedback is welcomed!

    0
  65. 104

    As someone making their first inroads into learning PHP, thanks for this article and the honest – and essential – approach to your mistakes, i.e. understanding them as opportunities for learning and growth.

    Without sounding presumptuous, I can’t seem to find much mention of the possibility of forgetting to include Leap Days within DOB selections for online forms. I know TWO people born on Leap Days, and as much as they’re rare creatures, it seems terribly unfair that in a majority of web forms they have to erroneously input their DOB or just walk away (sadly, shoulders hunched, etc).

    I’m going to assume that there’s some standardised response to this now, but if not, spare a thought for all those Leap Babies – their lives are bad enough losing every 3 out of 4 birthdays!

    0
  66. 105

    “Gzipping the root of our live production server”

    LOL that one made me laugh :P

    0

↑ Back to top