“If we truly want to analyze the impact of extended overtime in any scientific and objective way, we should start by recognizing that any individual game project must be considered meaningless by itself – it is a single data point, or anecdotal evidence.”
“We can learn absolutely nothing from whether a single successful or unsuccessful game involved crunch or not, because we cannot know how the project might have turned out if the opposite path had been chosen – that is, if a project that crunched had not done so, or if a project that did not employ crunch had decided to use it.”
“To truly measure the effect of crunch, we would need to look at a large sample, ideally involving hundreds of game projects.”
“According to this line of thinking, ordinary development with ordinary schedules cannot produce extraordinary results. We believe an accurate characterization of this viewpoint from the gamesindustry.biz article quoted above would be: “Extraordinary results require extraordinary effort, and extraordinary effort demands long hours.””
“This position (we’ll call it the “extraordinary effort argument”) leads directly to two falsifiable hypotheses:”
“1. If the “extraordinary effort argument” is correct, there should be a positive correlation between crunch and game outcomes, and higher levels of crunch should show a measurable improvement in the outcomes of game projects.”
“2. If the “extraordinary effort argument” is correct, there should be relatively few, if any, highly successful projects without crunch.”
“So if our combined score has a negative correlation with ALL our crunch questions except the one about crunch being purely voluntary (which itself does not imply any particular level of crunch), that means that we’ve disproven the first part of the “extraordinary effort argument” – the correlation is clearly negative, not positive.”
“in each case, there were more successful games made without crunch than with crunch.”
“We can see clearly that a higher percentage of no-crunch projects succeed than fail (17% vs 10%) and a much larger percentage of high-crunch projects fail rather than succeeding (32% vs 13%).”
“Additionally, a higher percentage of the successful projects are no-crunch than high-crunch (17% vs 13%), while a higher percentage of the unsuccessful projects are high-crunch vs no-crunch (32% vs 10%).”
“where the “crunch salvage hypothesis” tells us to expect correlations that are strong, positive, and statistically significant, we see correlations that are weak, negative, and statistically insignificant.”
“more crunch did not, to any extent that we can detect, help the projects in our study achieve better outcomes than they otherwise would have experienced … and in many ways appears to have actually made them worse.”
“We are left to conclude that crunch does not in any way improve game project outcomes and cannot help a troubled game project work its way out of trouble.”
“Does a lack of mandatory crunch then eliminate the negative effects of the quantity of crunch? In other words, do higher levels of voluntary crunch then turn crunch from a net negative into a net positive?”
“In short, no.”
“Our analysis shows that although crunch seems to be significantly less harmful when it’s voluntary, low levels of crunch in each case above (voluntary, mandatory, and mixed) are consistently associated with better outcomes than high levels of crunch.”
“If crunch doesn’t correlate with better outcomes, what does it correlate with?”
““There was a lot of turnover on this project.””
““Team members would often work for weeks at a time without receiving feedback from project leads or managers.””
““The team’s leads and managers did not have a respectful relationship with the team’s developers.””
“This seems to indicate that crunch does not, in fact, derive from any sort of fundamental drive for excellence, which would have resulted in higher correlations with completely different input factors on our survey. Rather, it appears to stem from inadequate planning, disorganization, high turnover, and a basic lack of respect for developers.”
“There is an extensive body of validated management research available showing that extended overtime harms health, productivity, relationships, morale, employee engagement, decision-making ability, and even increases the risk of alcohol abuse.”
“An enormous amount of validated management research demonstrates that net employee productivity turns negative after just a few weeks of overtime. Total productivity actually declines by 16-20% as we increase our work days from 8 hours to 9 hours.”
“Even just a few weeks of working 50 hours per week reduces cumulative output below what it would have been working only 40 hours per week – those 10 extra hours of work actually have a negative impact on productivity”
“All of that while also increasing employee stress, straining relationships, and increasing product defect rates.”
“our industry tends to value industry experience highly while undervaluing fundamental management skills.”
“crunch doesn’t lead to extraordinary results. In fact, on the whole, crunch makes games LESS successful wherever it is used, and when projects try to dig themselves out of a hole by crunching, it only digs the hole deeper.”
““effort” – as defined by working extra hours in an attempt to accomplish more – is actually counterproductive.”
“what actually generates “extraordinary results” – the factors that actually make great games great – have nothing to do with mere “effort” and everything to do with focus, team cohesion, a compelling direction, psychological safety, risk management, and a large number of other cultural factors that enhance team effectiveness.”