Experimental Velocity in Game Development

What is Experimental Velocity?

Experimental velocity in game development refers to the rate at which a development team conducts experiments to gain insights about their game or business model. This metric is essential in innovation and iterative development, where rapid experimentation can lead to faster learning and better decision-making.

Definition and Purpose

Experimental velocity measures the number of generative or evaluative methods executed over a specific time to generate knowledge. In game development, this could involve A/B testing various game mechanics, user interface designs, or monetization strategies to understand what resonates best with the target audience.

Importance

Measuring experimental velocity helps gauge how actively a team is learning and improving their product. A high experimental velocity indicates frequent testing of hypotheses and gathering data, leading to more informed decisions and potentially more successful games.

Challenges

Despite its benefits, experimental velocity can be problematic. Teams might inflate their numbers by conducting numerous low-impact experiments, focusing on quantity over quality. This approach can overshadow actual insights gained.

Velocity as a KPI in Game Development

Velocity, in the context of software and game development, typically refers to the rate at which a team completes user stories or delivers features. It is commonly measured in story points or the number of features completed per iteration or sprint.

Definition and Measurement

Velocity is calculated by summing the story points or features completed in each iteration. This metric helps teams estimate how much work they can complete in future sprints and plan their projects accordingly.

Criticisms

Using velocity as a Key Performance Indicator (KPI) has several drawbacks:

• Inconsistency: Different teams may have different definitions of a “point,” making it difficult to compare velocities across teams.
• Non-actionable: Velocity does not provide actionable insights into improving processes or product quality.
• Local Optimization: Focusing on the velocity of individual teams can lead to local optimization, which may not benefit the overall organization.

Alternative Metrics

Given the limitations of velocity as a KPI, other metrics such as cycle time, defect density, and throughput can provide more comprehensive insights into a team’s performance. These metrics can help identify bottlenecks and areas for improvement more effectively than velocity alone.

While experimental velocity and development velocity are useful concepts in game development, they have significant limitations as KPIs. Experimental velocity measures the rate of knowledge generation but can be easily gamed and may not reflect the actual value of the experiments conducted. Development velocity, while useful for planning, suffers from inconsistency and a lack of actionable insights. Therefore, it is essential to use these metrics in conjunction with other performance indicators to get a holistic view of a team’s productivity and the quality of the game being developed.