CRITICAL STAGE ANALYSIS

1. CRITICAL STAGE ANALYSIS Integrating CSA into the game development process

2. What is Critical Stage Analysis? • Simple mid-project feedback • Designed to get feedback from all team members • Requires team to take responsibility for cited problems ASAP • Intended to improve game quality and reduce crunch time by fixing problems early

3. The Origin of the Process • Wolfgang Hamann of Radical Entertainment • Article published in the July 17, 2003 issue of Game Developers’ Magazine (available at gamasutra.com) • Tutorial session at 2003 GDC (slides are archived at the GDC website)

4. Wolfgang’s Theory We plan for our game production process to be very linear, but it never really is. We’re unprepared for the loops we’re thrown for, and we have no tools to help us see what’s coming in advance. CSA adds a feedback loop that allows us to see our true production path more clearly and be better prepared.

5. Who am I to talk to you about Critical Stage Analysis? • I’m just a producer using the process • Amaze Entertainment; small Austin studio • Handheld projects on Nintendo DS • Licensed properties, 3rd party developer • Small teams, about 15 people or less • About a year of development time (or less!)

6. CSA – A couple of caveats • Hamann thinks postmortems should be scrapped; I disagree. • The literature presents this as a very simple process; and while the doing a survey is simple, executing solutions for problems identified mid-project is not.

7. Surveying the team • The big three questions, with answers ranked in order of importance: • Name five things that went well during the stage • Name five things that went wrong during the stage • Name five things that could be improved for the future. • The rating process is considered critical.

8. Analyze data and deal with problems • Data should be gathered and distributed to all team leads. • Team leads should meet to analyze the data, determine solutions, assign ownership. • Data and solutions should be presented to the entire team for discussion. • Everything should happen within one week of the milestone or stage that has just past.

9. The Ideal Timeline • Survey made available no more than three days after a milestone. • Team members will spend about 10-15 minutes answering survey. • Information should be compiled and distributed to leads no more than two days after it is gathered, and should be discussed immediately after receipt. • Discuss the results in the next team meeting, within the week.

10. Tips and tricks • Give team members multiple ways to fill out surveys. • In team meetings, bring up remaining issues from the last CSA first. • Lead with the positives. • Always deal openly and honestly about all issues – don’t hide things from the team. • Unsolvable problems should be broached. • Unedited CSAs should remain accessible. • Follow up is essential.

11. Measures of success • Fewer issues are raised repeatedly • Fewer things ranked with high importance over the course of the project

12. What I wanted out of CSA • Feedback before the game was over • Feedback from the quietest team members • Feedback on both big and little issues • A new communication tool • Morale measurement • A chance for team members to bestow accolades upon one another.

13. Things I changed I added a couple more questions, essentially to help get a measure of morale: • How satisified were you with the look and feel of this milestone, and why? • How satisfied were you with the deliverable we sent overall, and why?

14. Things I changed • Everybody was required to fill out the survey in the same format. • Surveymonkey.com • Zoomerang.com • I didn’t follow the suggested timeline.

15. My CSA Timeline • Monday after milestone – send out CSA survey to team. Deadline to fill out – noon on Thursday. • Wednesday afternoon – send out reminder to do survey. • Thursday afternoon – producer compiles data and sends to all leads and directors. • Friday morning – meeting with producer, leads and directors to determine what kind of solutions can be offered for all problems identified. • Friday afternoon – producer creates a nifty CSA presentation • The following week’s team meeting – team participates in a CSA presentation

16. Things I’ve learned • The CSA process has been a great thing – most of the time. • People did, indeed, laud their coworkers. • We did, indeed, identify problems we were able to solve earlier in the process. • I started getting a whole new level of feedback from team members about the project – both macro and micro issues. • Some project teams have become more emotionally invested in the process than others. • It’s very hard to keep things anonymous.

17. Things I’ve learned • The producer has a huge influence on gets discussed in the CSA meeting. • It’s difficult to tell what the rankings mean. • The two questions about what went wrong and what could be improved basically always ended up being 10 things that went wrong. • People who don’t communicate well in meetings don’t always communicate well on the surveys either. • It’s hard to give people unlimited ways to participate. • It takes a lot of time.

18. Things I’ve learned • People may wait until a CSA survey to present a problem, and that’s a problem. • CSA results really give you a leg up when you’re getting ready for a postmortem! • Opposing viewpoints come up all the time; thus, you can’t make everybody happy. • Some team members may be triggered in unexpected ways by the process.

19. But the most important thing is … Doing a bunch of surveys doesn’t do a thing unless you as a team commit to fixing the issues that are raised by them. • Team members need to feel that team leads and directors have got their back, and are willing to make changes to the production schedule, game design and perhaps even the staffing and budget plan if an important problem sited needs to be solved.

20. Future Plans • Start an online repository of CSAs to be used company-wide. • Ask even fewer questions. • Ask contract vendors to participate. • Get the client involved.

21. That’s it! Questions? Comments? Do you have a CSA-type process you’ve used? How does it work for you?