The Uncertainty of Design

 

uncertainty of design title

I was facing a frightening dilemma in my first years as a designer. I was supposed to make decisions. Decisions that would create great value. That would create interesting gameplay and perfect retention KPI’s.

I had the feeling that I could never know whether what I was designing is “right” or “wrong”. I felt lost and that every decision I made as a designer was merely a coin-flip, a bet on whether that design decision went into the intended direction or not. How could I even justify getting a salary for being the “coin flipper”? I finally came to peace with this uncertainty of design and I try make its implications work for me, not against me.

Let’s go

To tackle the problem of “design uncertainty” I would like to introduce the concept of “unself-conscious design” and “self-conscious design”. I will briefly explain those two approaches and how they apply to games. Then I’ll assess how accurate a design (decision) can actually be by comparing game design to social sciences. At the end, I’ll try to share some learnings on how to actually deal with design uncertainty.

Un/self-conscious Design

The Austrian architect and a promoter of “Design Science“, Christopher Alexander, describes two approaches to designing: “selfconscious design” and “unselfconscious design”.

Katherine Neil wrote a fantastic article on that matter in the game design context. A brief summary: Unselfconscious designs advance a product by modifying it (replacing, adding, removing elements), and then assessing the outcome.  Is it better, keep the change? Is it worse? Roll back the change. This means gradual improvement to a product, step by step.

On the contrary, there is “self-conscious design”, where the “design is freed from its reliance on making”, because it is based on methods and knowledge of the substance. E.g. “a composer’s ability to write pages of orchestral music at a desk, without needing to literally hear the music they are writing.”  Katherine Neil wrote a fantastic article on that matter in the game design context. She describes in the article that working purely in unselfconscious design (or should I say “as an unselfconcious designer”?) would make it impossible to make intentional changes. Predicting the outcome is only possible by looking at the results of previous modifications of this product. A good example for “unselfconscious design” is A/B testing which is heavily used in Mobile games. A change is made, and a group of players is exposed to that change. That demographics’ behavior is then compared to that of a control group without the change. After a time-span the numbers say whether a change had the desired effect or not. This creates a learning about how the system behaves when changing certain condition. But the experience remains a black box.

This is why it is dangerous to copy systems/mechanics from other games. As they can and probably ​will​ work differently. Learnings from unselfconscious design methods such as A/B testing, rarely translate to other games. How a system works, depends on the context it’s never standing by itself, it is connected to other parts of the game, other systems, and content which is geared towards a certain experience. Shoehorning a system or mechanic into that ecosystem will not create the same experience as in its origin.

As soon as we start asking ourselves why a certain change leads to a particular outcome, we step into the realm of “selfconscious design”. The result would be that we can abstract the reasons for an emerging behavior of our design. By knowing why something works, we have the power to apply generic rules to intentionally instigate a certain user-behavior or emotion through our design.  ​Now we can take a system and try to recreate ​that experience within the context of our game. This makes Design more formal and discussing it objectively. Those principles, when passed on from generation to generation, will evolve, leading to a sustainable advancement of the ​whole ​ field.   

Between the knowledge gained through observation, and formalized design rules lies the  “design-instinct”, which is developed by a designer undergoing many iterations of modifications and observation and then transformed them into maybe generalizable models. The problem is that designers might draw different conclusions from their various projects. This knowledge is shared by collaboration and discussion. There is no guarantee that the “correct” models are passed on. The evolution is rather slow.  I have observed many people (including myself), making the mistake of generalizing too rash. A good reminder to bring up the Dunning-Krueger effect; which describes our overconfidence due to lack of knowledge!

To counter this, we should try to transform the learnings from observations to generalizable models. This was already postulated by Dan Cook in one of my all time favorite articles on that topic, where he compares game design to alchemy.. A process in which by combining arbitrary elements, hypotheses were formed about the inner workings by making observations (Similar to the beforehand mentioned unselfconcious design). This describes how games are built: there is an idea, we build it, then we see if it works. Based on the outcome we repeat or iterate.

Since we are human beings with limited resources, this process is very inefficient as only a limited amount of insights are possible.

Especially now as designing games has already come a long way, it is harder to gain profound insights with that method. He, therefore, concludes, that the field should mature from alchemy to chemistry. Where the deep understanding of the underlying parts, assumptions can be made about the behavior of those parts (selfconscious design).

But Chemistry has an advantage: it deals with a consistent substance. A hydrogen molecule is always exactly the same, the same experiment can be repeated as every scientist in the world has access to the same hydrogen molecule. With games we are dealing with one of the ​least ​ consistent and ​least ​ understood substances in the known universe: The human brain, psychology. “Design” is facing the same challenges like social sciences. Both fields are based on the behavior of independent actors (people) under varying conditions. Unfortunately, people are more complicated than non-meant-physical-objects and behave differently and “irrationally” under varying conditions (it is probably only called irrational because we do not fully understand it yet).  Studies of social sciences are often inconclusive or only apply to specific situations. Does that uncertainty sound familiar?

Economists, for example, are formulating models that try to explain and predict economic behaviors. The more enclosed the subject is, the more accurate the model can be. When going to a higher level problem, models have to be combined, and their uncertainty multiplies, as multiple assumptions and experiments are and interacting with each other. That’s why we have lengthy discussions whether minimum wage has a positive impact on the economy or not. What happens long term? What happens short term? What side effects will there be? All of these things are very hard to predict. Because in reality, a colossal number of variables go into a complex system that, over time, and under the influence of outside factors behaves in a chaotic way.  I would say we are dealing with a similar situation when creating games. There is an endless amount of permutations of the experiences that the game is creating in each individual player’s head.

Where the science becomes the craft is when it is decided what model to apply to what situation. And what models to combine depending on the situation. In science, models are not perfect, but they don’t need to be. In science, we always must assume that our understanding is imperfect. But it is ​good enough ​ to get us to the moon, or to predict the existence of the Higgs boson.   

In the same manner, the design of a game is not a coin flip, by improving our models we improve the accuracy of our predictions when it comes to the players’ behavior or emotions.

The Current State

Katherine Neil writes: “No language of game design, nor anything that could be described as ‘formal, abstract design tools’, has been adopted into mainstream game design practice”.  From my experience, this is quite accurate. Every Designer that I’ve met so far, has some sort of technique, or method, or ways of abstracting and describing things. But each one has their own. It always takes some initial effort to find a common ground when diving deep into a game design discussion. Explorations into the formality are being done by designers like Jason VandenBerghe who is iterating on a psychology based model that tries to explain and predict why people play and what keeps them going: ​Engines of Play​. Also “Project Horseshoe“, a “think tank”, explores the opportunities of selfconscious designs.

 

Those are more or less still based on individuals or single groups, creating brief islands of knowledge and insight. In other fields, such as psychology or political, schools form, exploring a specific angle on a topic by which they hope to fully grasp their field. I’ve not seen that happen in game design yet.  

Designing for the Uncertainty

What became clear to me is that my initial problem of not knowing whether I was “wrong” or “right” was flawed, to begin with. There is no right answer, there is no 100% confidence if I just invest enough time. This can be tough. Many people, including myself, want to mitigate uncertainty and risks. I want to explore all options before making a decision. Reaching a high confidence seems like a desirable goal, but if taken too far it becomes a problem. Some decisions just need to be made, and the confidence level in those decisions is limited.  Time spent on investigating has a diminishing return. At the end of investigating there will be no “perfect design”, therefore it is important to know when to stop. Also, the further you wander into the design space in relation to what you already know, the more uncertain decision become. On the other side of the scale: The more accurate our models the higher our confidence can be, yet, validation still needs to come from the test as any model that we can have is far too inaccurate to predict the players’ behavior.

Never trust a designer that is overconfident in being right. Especially if that designer can not explain what the applied model or assumption is.

Like a scientist, designers must operate under the assumption that they might be wrong. This is not just a rational precept, it is also vital to keep an open discussion. Chances are high you have had a discussion with someone who had a fundamental belief in his or her idea, without the ability to rationalize it. Don’t be that person! And also keep in mind, the more certain you are about something, the more silly you will look when you’re wrong :)

Which leads to another important topic. In a team, in production, the designer can also not be the person that is never confident in what they’re doing. So how can you deal with the dilemma of knowing that the direction everyone is walking towards might be a mistake?

  1. Every effort of building something should be seen as an experiment, stay open and observant, test early.
  2. Probably not all of your assumptions are wrong. As mentioned, experience, instinct, formal models increase the confidence. A design is a series of choices, and chances are that only some of them were “wrong”.
  3. Communication. As a good designer, you should be able to rationalize your choices or let you guide choices of the team with a more rational approach. Every single design decision has a pro & con. It’s a jungle. But at each intersection you stand, there should be a reason and understanding of how to make that choice. What builds trust with teams or stakeholders is not magically always being right enough times (because you won’t!). What builds trust is communicating the rationale of the choices to team members or stakeholders. This allows them to question the choices that were made. Only 2 things can happen: A) They have a concern that you have already considered. Then you can explain what assumption led to the decision. B) They bring something up that you have not considered. Cool! This might lead to a better choice because you could just update the rationale and maybe avoid a stupid edge case or flaw in the design.

 

Conclusion

Designing games is tough. If you feel frustrated with not being sure what you’re doing is promising, keep in mind that you can never be sure. Try to see it like a scientist, like an experiment. Rationalize your hypothesis, then you can communicate it to others. This will earn their trust and leaves enough wiggle room for changes and discussions. More experience as a designer will lead to more accurate predictions, it will not make you more right. Try to have as rational design discussion as possible, the argument should matter, not the person who is making it. As designers we are not entitled to are a more valuable opinion. The only thing that differentiates the designer from any other person on the team is that they get the time to apply their problem-solving skills and abstract thinking to the problems of the game and hopefully end up with a good repertoire of arguments and rationales that can stand on their own feet.

 

Less is more: 4 reasons to keep your rewards scarce

less is more

Rewards are at the heart of games. They come in many flavors.  

When balancing economic rewards such as Currencies, XP, Items and other quantifiable things, we often face a challenge like: “Where do we give the player what?” And “How Much of what?”

This article discusses how giving fewer rewards is more desirable for making a game’s economy tighter and the game more enjoyable for players.

Argument 1: The Presenters Paradox

A study shows that adding less desirable information to more desirable information actually diminishes the total value of all the information that are presented.

The same is true for items with a quantifiable monetary value (see study 1 in the paper). The problem is that the evaluator of a “bundle” (multiple items) subconsciously averages the value of the all the items.  Mixing something very valuable with something that is less valuable will reduce the value of the higher valued item. Hence, when both are combined, the sum is lower. The perceived value is lower than the sum of its parts. If your bundle aims at conversion, this might not be your biggest concern. But if the “bundle” is a set of rewards that players receive, then you probably create a lower motivation to pursue that reward by adding more items to the rewards.  Less, in this case, is more.

From the study

“They could either bundle an iPod Touch MP3 player with 8MB of memory with a cover or the same iPod Touch MP3 player with 8MB of memory with a cover and one free music download. […] They were willing to pay more for the smaller package that contained only the iPod (Mean: $242.19) than for the larger and economically more valuable package that contained the same iPod plus a free music download (Mean: $176.71)“

presenters paradox


 

So what does it mean?

As the balancer, we have 2 Axises to balance our rewards

  • Type (What do we give?)
  • Amount (How much do we give?)

It sometimes hard to decide how to tweak the rewards along these vectors. The presenters’ paradox can be used to inform that decision. As the designer/balancer, you should be cautious about mixing different TYPES of items. Because you risk watering down the individual reward. You would give out less bang for a bigger buck!

  • When your economy has multiple resources and systems: Giving out high valuable items together with lower value items, makes it less special!
  • This is something you should strongly consider when you are in charge of designing bundles/prices for items in shops of free to play games! The perceived value of the bundle will be lower than the combined value of the individual items.

The presenters’ paradox also works for penalties (check “Study 3: Littering penalties” in the study). Hence, if your game has a losing condition where players lose something, then just losing 1 Item/resource can be more powerful than losing multiple things.

Argument 2: Exclusive Rewards Guide Player Behavior

Rewards can be a tool to reinforcing or incentivizing a certain behavior. By receiving a reward, players know they have done something “right”. That they have reached a desirable game state. Players, of course, come with a certain intrinsic motivation and don’t play the game for rewards. Yet, players expect to get something back from the game when they have achieved something or made a choice. Explicit rewards can also motivate the player to engage in an activity that they otherwise not have e.g. Achievements. 


 

What does that mean???

  • Consider what reward types in your game should be exclusively tied to one activity
  • Ask yourself what rewards should the player receive for which activity and why
  • Exclusive rewards motivate players to dig into a certain system or practice a certain skill or to play the game in a special way
  • An exclusive reward tied to a performance goal can create a high sense of achievement
    • If possible, give players the opportunity to revisit their exclusive rewards. Players can relive the memories of their journey towards that reward (items, cutscenes, achievements)

reward

Argument 3: Choices

So we know that giving out multiple things for one activity can have a diminishing effect. Also, we know that we can incentivize player behavior with certain rewards. This means we introduce a choice. That’s cool. But as the designers, we must carefully adjust our reward-sources to meet the sweet spot between “exclusivity” and “uniformly distributed” rewards. Uniformly distributed means that you multiple reward-types from every system in the game. This means that players actions or choices come at a lower opportunity cost since they get more of the different types of rewards from any action. On the other hand, this reduces the sense of choice.

balance_design_choice

Again I would like to emphasize that there are tons of other factors that make players enjoy a game. Rewards are just one (important) element. Players will have preferences for mechanics/systems based on many factors (which we only have partial control over). What we do have control over, is the rewards balancing. So let’s continue with that.

The point is: Know your players, decide what kind of decisions you want them to make, and then adjust the reward balancing to that. None of the approaches are perse bad. But we should make it a choice in the design rather than just letting it happen. These small decisions heavily influence how player experience the game and informs their decisions. As designers, we have to use every opportunity to shape that experience.


 

What does that mean???

  • Opportunity costs and resource scarcity drive players decisions, so by reduction of rewards/opportunities, you end up with a higher player agency. Less is more.
  • Carefully adjust the rewards of your game through the lens of players choices
    • What systems are part of the core loop, which rewards belong to these systems and which ones not?
    • Analyze sources sinks, and try to assign the sources only to those systems where you need them.
  • By deciding what players receive for what activity, you also inform their mental model of how the game works

 

Argument 4: It is easier to understand

Argument 2+3 lead to players having an easier time of building a mental model of the game’s economy or system. When it is clear what choice yields what reward, the players can grasp more easily the systems intentions.

  • Reduce noise to help players understand what really matters and what they should do

 

Real Life example

With a small team, I am working on a turn based squad tactic game. We can place loot on the map but also enemies can drop loot of various kinds: currency, med packs, and other consumable items.

Instead of adding loot crates that drop a bit of everything, we decided to have distinct loot-crates for each of the item types. Players can then get an idea what the loot crate might contain and incorporate that into their strategy. Whereas if all the loot was the same, it doesn’t really matter where players go with their squad (Argument#3). We also wanted players to feel rewarded when they get the loot. Since it is actually quite rare to get it, we also decided to put just one item into the crates for the sake of Argument#1.

 

A resumé of 20 side projects

Many professional game developers maintain personal projects as a creative outlet. The day to day work can be very restrictive and we operate under tough requirements. Working on the same game for a long time means that the mind is mostly busy the same set the problems within a certain design space.

For me, it is crucial to get my mind of the everyday problems and give me some fresh problems from time to time. This is a personal resume of almost 2 years of working on side projects.

The Goal

When I started a project, the goal was usually pretty simple: “Create something fun”. To see if I have reached my goal, I off course need to create a prototype and then validate the games concept in a playtest.

If a project does not reach the stage of a player holding it in his or her hands, then I consider that project as failed. If there is no player, then there is no game. Projects who reach a stage where I can test them I consider as a success. No matter whether players proved or disproved the concept. These are the ones I could learn the most from as a designer.

20 Projects

In the last 2 years, I have started 20 projects Of which I released one. 3 Other prototypes turned out to be not as much as fun as I thought to be while 16 others I would consider as a failure. I only count projects that I attempted to actually build, so all of the “failed” projects ended during the development phase, for the following reasons.

Reason #1: Design

These projects ended because something went wrong mostly on the design side. I would divide this into two further groups.

The Premature Idea

The reason for these games to get cancelled is that I simply didn’t think them true. The typical story is that I have a “great idea” which sounds super fun in my head but once I start building it, I realize that there are some major holes in the design. Luckily these projects end quite fast and not so much time is wasted.

The Spiral

These are the most dangerous projects. That were stuck in a place where I believed in them and tried really hard to make the design work. One cause for these projects were too high ambitions. In some cases I wanted to create a game that would be valid in the market. This opposed more challenges on the design the project became everything else than fast. In other cases, the designed turned out to be harder to prototype and I went back and forth between paper – and digital prototypes. These projects are dangerous because of a strong Sunk Cost bias combined with the feeling of an imminent breakthrough. Needless to say that the time-cost here is huge. And the more time I invested, the harder it was to end it.

Reason #2: Development

These projects turned out to be too complicated to build in order to prove the initial design. I would argue that the problems of these projects are also rooted in the design phase, though since I could have foreseen many of the issue and complications ahead.

My Learnings

In the creative industry, it is widely accepted that “failing” is part of the process. Motivational quotes that encourage failing are all over the place. But the time spent while failing is only worth it if we take a step back and reflect upon the shambles that lay before us. After all, we want to improve (mostly by not repeating mistakes).

Personally, my biggest challenge is that I stress out too fast. Which is not really helping since these were side project which ought to  provide me with some relief by letting me exercise my design-brain.  My personal lesson from this time of trying really hard is that the journey is its own reward.  While I was going for a solid end result, I forgot to enjoy the moments that lead there. I forgot that the quality of a designer is determined by his or her creative-process and not necessarily the outcome.  The outcome of a project (if it is released), varies on many factors. The only thing we can do is to make sure our decision now is the best that we can do.

PS:  In my next article, I will lay out some tips which I hope will help people who are on the same path as I am.