The Lust for Results

Or more specifically, commentary on why bugs cannot always be fixed right away when they are first reported. But first we again have need of the obligatory...

DISCLAIMER: I am not employed by FarSight, not representing FarSight, and nothing in this post should be construed as the Official Opinion[SUP]TM[/SUP] of FarSight Studios or its development staff.

All software bugs have six traits that together help determine when they get fixed:

• Severity - How much impact does this bug have on the software? Is it a minor aesthetic issue? A UI mistake but one with a workaround? Does it affect the player's score or does it make it impossible to continue? Does the software itself crash or freeze?
• Frequency - Does the bug occur all the time? Once every few times? Once every few hundred times?
• Reproducibility - Can the bug be demonstrated at will by following a few well-defined steps? Or does it happen at random? Or somewhere in between? Is the bug more likely to occur when certain conditions are met?
• Distribution - Does this bug affect all or nearly all users, or just a tiny fraction? Note this is not the same as frequency, a particular bug may affect everyone but only occasionally, or it make affect a small number of users but make their lives hell.
• Entanglement - Where in the code is this bug occurring? Is it in some isolated section where making the changes to fix it will affect little else? Or is it in code that is heavily accessed by many different functions, such that making changes could have unexpected consequences for the rest of the software?
• Difficulty - Having found the bug, can the fix be made quickly to a few lines of code, or will the fix require overhauling major sections of the software?

(Note to developers: Yes, I'm aware this arrangement does not correspond to any of the standard methods. But I'm aiming for clarity to a non-coding audience, not for technical precision.)

Let's take two examples from TPA:

At one point (and this still happens on the 360, since it hasn't had an update since this bug was fixed) slow-moving shots to the ramp in TOTAN would result in the ball oscillating back and forth several times before falling through the hole in the ramp and feeding to the right inlane. Let's evaluate this bug according to the factors above:

• Severity - Usually not a huge deal, but if something with a short timer is running (Fireball or Lightning Lamp) is running, the user will get screwed out of the opportunity for good scoring. The worst result possible is that all 4 balls get stuck up there during Genie Battle, which results in the player losing the wizard mode.
• Frequency - This doesn't occur constantly, but it happens pretty often, certainly more than once every few games.
• Reproducibility - It's not 100% reproducible, but eventually making enough slow shots to the ramp will cause the issue.
• Distribution - Pretty much anyone who's played TOTAN has seen this one.
• Entanglement - There is a special-case code section that handles the hole in the ramp for TOTAN. Being special-case, this code is not used by other parts of the software.
• Difficulty - In this case, the fix was able to be made quickly...in 5 minutes, actually.

Now let's look at something more interesting - the flipper quantum tunneling phenomenon:

• Severity - Not game-breaking, but close to it - each time the bug occurs, the player loses the ball.
• Frequency - Some players report this happening all the time, others rarely see it.
• Reproducibility - Not reproducible at will.
• Distribution - Almost everyone's probably seen it at least once, but again, some players only have problems rarely and others seem cursed with Swiss-cheese flippers.
• Entanglement - The flippers are fired hundreds of times per game. Therefore it seems reasonable to assume that the relevant code is heavily accessed.
• Difficulty - I'm not sure, since I don't have TPA's source code. Since the issue was acknowledged several months before a fix was announced, we can assume the fix was not easy.

This is all very nice, but what does it mean for TPA users? Let's say that a new table is released - Attack from Neptune or something - and you're playing along in multiball and all of a sudden a ball gets stuck in a kickout and the attendant won't help. Cursing and swearing, you start a new game of AFN and reach multiball again, but the bug does not happen. Three more games and no bug, but during the fourth game the ball gets stuck again. You report the bug, but you wonder how long it will take to get fixed. Let's make an assessment:

• Severity - It quite clearly prevents the current game from continuing normally.
• Frequency - You've seen it twice now in 6 games, so not 100%, but not at all uncommon.
• Reproducibility - Hmm. You're not sure. So you play a few more games of AFN, and try to pay attention to what's going on immediately before the bug occurs...and also what occurs immediately before situations where the bug does not occur (this is important!). Let's say you still can't reproduce the issue at will, but you notice that the problem seems to happen a lot more often when the ball goes into a kickout while the Jackpot animation is playing, and doesn't seem to be happening during other times.
• Distribution - You browse the relevant sections of these forums and see that some users are reporting similar issues, but others haven't had any problems and are wondering what the fuss is about.
• Entanglement - Not sure. It could be a problem with the kickout which is used by the table in various situations, or it could be a problem with the Jackpot sequence, which is used only in this particular multiball.
• Difficulty - We have no idea, not having source code.

We see that we have a relatively severe, relatively common bug - both factors that tend to lead to a high priority for a fix. We are not able to reproduce the bug at will, but we have a general idea of what may be causing it, so that's helpful. But we also see that not everyone seems to be having the problem, it's possible that the bug is in heavily-used code, and we don't really have a good handle on how difficult the fix will be. So we conclude that FarSight will likely start looking at this bug right away, but depending on the cause it might not be a quick fix, so we should not go ballistic if the bug is still present in the next update.

Again, this is not the exact procedure FarSight uses. But hopefully this will give players an appreciation for what goes on behind the scenes when bugs are reported, as well as give them some way of estimating when a fix might be available.