Designing Psychological Stickiness: Variable Rewards

This is Part 1 of a series I’m calling Designing Psychological Stickiness.

In each edition, I’m breaking down key psychological mechanisms companies harness to make their products super sticky multi-billion dollar generating machines.

Today, I’m diving into a big one. The psychological principles of Flow and Operant Conditioning.

Let’s dive in.

Liberty Bell

Sometime during the 1890s, Austrian-born inventor Charles Fey was busy in his San Francisco workshop building a prototype for a new style of gambling slot machine — the Liberty Bell.

It was the culmination of years worth of experimentation and relative success within the then-budding coin-operated gambling device industry.

Here’s Charles Fey and his Liberty Bell. 👇

Today, the Liberty Bell looks like an antique. It would comfortably blend into the background of a 19th century Wild West saloon, surrounded by trigger-happy outlaws, high-stakes poker games, and half-drunk bottles of bourbon

But, even for a citizen of the 21st century, the look and feel of the machine is still highly recognisable as a slot game. Many people would know how to operate it today, unlike most other mechanised devices from the same period.

And, it’s deceptively complex. Not so much in how it works mechanically, but how it engages the human brain.

Charles Fey had unknowingly pieced together a machine that artificially tapped into powerful and otherwise dormant psychological hooks that drove compulsive repetition and rhythmic excitement.

The Liberty Bell stimulated satisfying and compulsive patterns within the mind that have been useful for the survival of the human race since prehistoric times. But, on a turbo-charged mode. It isolated and magnified it to a degree otherwise not replicable in the natural world.

It was a potent moment in history that helped gear society on a steep adoption curve of automated obsession and addiction — where our leisure time, attention, and money became increasingly absorbed by devices and machines with immediate input-output (I/O) feedback.

The Test

After completing the Liberty Bell prototype device, Charles Fey eagerly placed his ‘19th century MVP’ in a local saloon and stood back to observe the results.

The game was an instant hit that proved irresistibly popular with players; to the degree that it likely performed far better than his wildest expectations.

Punters were captivated by the possibility of instantly doubling, tripling, or even 10X-ing the return on a nickel through the machine’s unique feature combination of:

• Spinning three-reels (randomly consistent, visually pleasing motion)

• Simple rules (minimal learning curve)

• Card symbols (familiarity)

• Stagger stop (tension-inducing gameplay)

• Automatic payout (instant gratification)

Interestingly, the Liberty Bell was by no means the first gambling slot machine. Several of its core features (like automatic payout) were borrowed from preceding devices.

Many other slot machine variations were fashioned and distributed all across the U.S. in saloons and other ‘establishments of vice’ around the turn of the 20th century. Most modern consumers probably wouldn’t recognise or immediately know how to play them.

Here are examples of other gambling slot machines from the era, some of which Charles Fey was involved with. 👇

The Liberty Bell’s success was so overwhelming compared to these competing devices that its basic formula “rapidly became the backbone of the entire gambling-machine industry”, as characterised by Charles Fey’s grandson.

It’s true. Most other slot machine formats became obsolete or were pushed to the fringe. By the 20th century, the term ‘slot machine’ became synonymous with the Liberty Bell three-reel formula and its descendants.

Competitors exhaustively copied and improved upon this basic formula for over 100 years, eventually creating a U.S. regulated market worth around $56bn annually today, $100bn+ globally including black markets, and much more when you take into consideration the full picture — unregulated gaming and software.

Modern smash hit games share significant game DNA with the ‘bell slot’ design— as it became known.

Prime examples are Candy Crush ($4bn+ in lifetime revenue) and Coin Master ($2bn+ in lifetime revenue).

The similarity with Coin Master is immediately apparent. Here’s a screenshot, with the classic three-reel layout.👇

The link with Candy Crush is not so obvious, at first.

It looks completely different, from a surface-level glance. 👇

But, when you start to peel back the layers, a slot game comes into focus.

This is what a 100+ year descendent of the Liberty Bell looks like.

How? Each vertical row is a reel strip (9 in the example above). Each piece of candy, a symbol on that strip. Each game, a ‘randomly generated’ spin result.

The premise of the ‘match 3’ skill-mechanic, where players match three or more symbols together via performing a limited amount of input actions, has been around in ‘bell slot’ machines for a long time. Often, in the form of a ‘nudge’ or ‘hold’ action.

Even the theme of the game — colourful candy and fruit-flavoured deliciousness — was popularised in slot machines as far back as at least the 1930s.

The hugely popular Operator Bell slot machine used symbols of lemons, cherries, raspberries, and other tasty fruits to entice players with visually appealing and disarmingly familiar stimuli. 👇

During prohibition — when gambling was widely outlawed across the US —payouts were often in the form of candy (mainly chewing gum), the flavour of which correlated to the combination of winning symbols present on the machine.

Dissecting Success

This raises a question. Why did the ‘bell slot’ format prove so successful?

It’s tempting to explain away the success of slot machines as a gambling issue — gambling is addictive.

There is truth in that. But, it’s an oversimplification.

Gambling as a standalone construct is not that sticky. There is theoretically an infinite amount of ways to gamble. But, only a handful of game genres populate casino floors.

It’s the delicate weaving of gambling into a randomised game or experience with which humans are psychologically engaged that creates stickiness.

Slot manufacturers have perfected this with scientific precision because it’s such a lucrative business model.

Gambling — as in the staking and winning of money — isn’t even a prerequisite to driving stickiness with this framework.

The stake and prize can be anything of perceived value. And, eventually, the significance of this as a mechanism to foster long session times starts to erode.

Like Candy Crush and Coin Master, there’s a whole sub-genre of slots games that players can play to play but cannot withdraw any real money from.

This genre is called “social casino”. Players exchange real ‘fiat’ currency for virtual currency and are unable to exchange it back.

It’s not a small amount of change that is pummelled into social casino games; it’s billions of dollars annually.

In any of these scenarios, with heavy usage the appeal of prizes starts to dissipate and users transcend into a state of sedation-like absorption, continuing to use the device or software as a form of relaxing, untaxing, and low-effort escapism.

As Natasha Dow Schüll wrote in her seminal book, Addiction By Design:

This is the equivalent of passively flicking through social media feeds for hours per day, the mind obviated from self-reflective thoughts and immediate problems.

The foundation for this primarily lies in two powerful psychological concepts—Flow and Operant Conditioning. Both of which I’ll go into, and neither of which are unique to the gambling industry.

These underlying psychological mechanisms are also utilised by other game genres and the world’s most successful consumer apps.

You can trace the common underlying psychological mechanisms of Flow and Operant Conditioning in automated addiction from the Liberty Bell in the 19th century all the way to platforms like Facebook, Instagram, Twitter, YouTube, Tinder, and TikTok in the 21st.

Let’s compare how, by using the Facebook newsfeed as an example.

Like pulling the lever on a one-armed bandit slot machine, each time you ‘swipe up’ on a social media newsfeed you are placing a ‘bet’ with your time and personal data that the result (content that scrolls into view, just like a slot reel) will excite you.

1. The action is low energy and fast

2. The motion is rhythmically satisfying and repetitive

3. The result is random and instant

Sometimes the result is a loss (boring content… like an advert), sometimes it’s a small prize (OK content… like a recipe idea), and sometimes it’s a big win (thrilling content… like a friend’s engagement announcement).

Here’s an easier way to visualise it. 👇

Next, I’ll break down the significance of this.

Operant Conditioning

So, what’s going on here?

First, there’s the Law of Effect. This was established by psychologist Edward Thorndike at the turn of the 20th century.

The Law of Effect says behaviour followed by pleasant consequences is likely to be repeated. Behaviour followed by unpleasant consequences is less likely to be repeated.

This concept led to the positive and negative reinforcement ideas that have been floating around in popular society for some time — mainly within discussions around parenting, rehabilitation, and animal care.

Later, psychologist B.F. Skinner built upon this framework with an area of study he coined ‘Operant Conditioning’.

Operant Conditioning dove into the nuances of the Law of Effect with the concept of schedules of reinforcement (rewards).

In another words, Skinner investigated to what degree a behaviour would be repeated by delivering results (rewards, punishments, and neutral responses) under differing programmed conditions.

The device he designed to test this is known as a Skinner box. 👇

As per SimplyPsychology:

The breakthrough in this research is a concept known as ‘schedules of reinforcement’.

That is: delivering rewards under different types of schedules.

What does that mean?

Consider a rat in the Skinner box. If it presses the lever and a food pellet is delivered the first time, how many more times would it press the lever if a pellet was delivered after fixed versus random time intervals?

How many more times would it press the lever if a pellet was delivered after a fixed versus random amount of presses?

And, how frequently would it press the lever under each of these conditions?

These were all questions Skinner discovered the answer to, by creating specific reward schedules and testing them.

• Fixed ratio. Fixed amount of presses between each reward.

• Variable ratio. Random amount of presses between each reward.

• Fixed interval. Fixed amount of time between each reward.

• Variable interval. Random amount of time between each reward.

Skinner discovered that variable-ratio reinforcement (delivering rewards after a random number of presses) produced the fastest and largest overall response rate (pressing the lever) and the slowest rate of extinction (the behaviour will be repeated for the longest time without reward). 👇

This behaviour is common across the animal kingdom, including humans.

In 1953 Skinner used slot machines to exemplify this.

As Natasha Dow Schüll wrote:

It’s specifically the variable-reward ratio schedule that helps make slot games and modern-day apps so sticky.

Reward schedules are meticulously designed to tap into this inherent behaviour. Slots mathematicians configure exactly how frequently various degrees of win (or no win) occur at scale.

By rigorously testing and tweaking the game’s reward schedule, they can encourage players to continue playing even if they receive more losing spins than winning spins — which is key to profitability.

The spin results are programmed pseudo-randomly from a mathematical script and delivered to the player theatrically through the game device’s interface.

When tuned to certain win frequencies it creates a harmonious balance of tension and release, keeping the player stimulated and engaged.

This maps out to the prior comparison with the Facebook newsfeed.

If a slot machine delivers too many losses, session duration decreases. If Facebook delivers too many ads, session duration decreases.

If a slot delivers too many small wins and not enough high-value wins, session duration decreases. If Facebook delivers mainly OK content and not enough exciting content, session duration decreases.

Delivering ‘too much’ of a reward can also be detrimental to session times. If a user receives a high-value prize (like a mini jack pocket on a slot game) it increases the chance the session will end.

It’s mostly an exercise in finding the ‘Goldilocks zone’. Keeping users ‘on the edge’, eager to discover the next randomly generated result.

Facebook (and many other consumer apps) are constantly refining their reward schedule to find the optimum balance of app-visit frequency and session times.

This is partly why Facebook had a hard time shifting its newsfeed away from algorithms that surface inflammatory and clickbaity content.

Flow

You may have noticed that while the variable-reward ratio had the joint-largest amount of lever presses, it did not have the longest overall time of engagement.

In fact, it was 5x less than interval-based reward schedules. This means intense short periods of high productivity (lever pressing, swiping, etc) but not necessarily long, extended session times.

This is where a second psychological concept comes into play — flow.

You’ve very likely heard of this before. When people refer to being ‘in the zone’—such as during intense writing, coding or painting sessions—this is what psychologists refer to as a ‘flow state’— as coined by Mihály Csíkszentmihályi.

If you’ve ever tried to talk to someone with eyes glued to their phone and were met with a silent or muffled response, they were likely in the zone. That means completely focussed on the activity and disengaged from the surrounding environment.

In Csikszentmihalyi’s words:

Flow is usually referenced in a more positive life-advancing context. For example, performing an activity that helps you finish a project for a client. It’s usually more personally constructive than spinning slots reels or flicking through social media feeds.

So, it’s not *exactly* the right term to use here, but it’s the same thing but with a less constructive — even destructive — undertone. Sometimes people use the term ‘hyperfocus’ instead, which has overlapping attributes to flow and is used in a negative life-destructing context.

I’m using ‘flow’ and ‘the zone’ as a reference point since they follow the convention Natasha Dow Schüll used in her book.

People can enter and maintain a state of flow for long durations. In the computer age — with solitary and uninterrupted session times — this can be sustained for many hours, sometimes days. That applies to work or entertainment.

If you’ve read Thinking, Fast and Slow by Daniel Kahneman, you’ll be familiar with his thesis of two modes of thought: ‘System 1’ and ‘System 2’.

System 1: Your brain running on autopilot, performing an activity it’s used to. Like answering the question what is 10x10? Or, riding a bike.

System 2: Your brain working harder, performing an activity it is not used to. Like answering the question what is 18x59? Or, learning to fly a helicopter.

It’s the former, System 1 mode, that eases the mind into a state of flow with slot games, social media apps, and the like. Easy, untaxing, engagement.

The brain is operating on autopilot—eliciting a sense of numbness and pacification.

At a core level, this is achieved by:

• Simple I/O mechanics (swipe up, new content scrolls into view).

• Input Repeatability. (the same action, over and over).

• Output Consistency. (the same response, over and over)

• Fast Cadence. (quick I/O rhythm—no time for distractions).

Powerful Combo

When System 1 flow is paired with a variable-reward ratio delivery schedule it produces a steady, trance-like state that distracts the user from internal and external issues such as anxiety, depression, and boredom.

It’s the combination of these two psychological concepts that are so powerful in driving deep, repeat engagement.

Variable-reward ratio delivers the immediate appeal followed by a knife-edge balance of anticipation and excitement. Flow delivers comfortable long-term repetitiveness and the ability to ‘zone out’.

In her book, Natasha Dow Schüll reports that heavy players become so absorbed in the rhythm of play that the boundaries between game and mind break down and fuse together, psychologically leaving the body behind to fend for itself.

Players develop a rhythm of play in harmony with the audio and visual patterns of the device. To synchronise mind and body with game. To incept the flow of the zone. To continue.

Players will forgo sleep, bathroom breaks, and other bodily needs to continue playing, tapping, and swiping.

If that sounds odd, it’s kind of like Jake Sully in the movie Avatar. Whilst his mind is fully absorbed as a Na'vi character, his human body is just kind of neglected and left behind. At one point, choking for oxygen.

That’s a wrap — here’s Part 2 of Designing Psychological Stickiness.

Martin 👋

Did someone forward you this email? Happens allll the time.

Subscribe to receive more content like this, weekly(ish). 👇