How Roblox thumbnail A/B testing works

Roblox first shows each active thumbnail to a random group of players. Once it has data, it gives more impressions to the winner for each group of players, while the others keep a little traffic so the test goes on. Different players can see different winners: that's the "personalization".

The test only runs with 2 or more active thumbnails. With one, every player sees that one.

Not clicks. Roblox ranks thumbnails by qualified play-through rate: qualified plays divided by Home page impressions, where a qualified play means the player stayed and played for a significant time. A flashy thumbnail that brings players who leave at once loses.

In Roblox's tests, experiences using it saw an average +8.5% in qualified play-through rate, and some saw +50%.

• Open your experience on the Creator Hub, then Thumbnails, then the Home Page tab.
• Choose Edit active thumbnails.
• Upload new images or tick Active next to existing ones, 2 to 5 in all (1920×1080, under 3 MB).
• Save to start. The Thumbnail Performance table shows how each one does.

• Test real differences. A new character, a new scene or a new hook teaches you something. The same image with a new color rarely does.
• Keep several active. Roblox advises against picking one winner early: the others keep learning, and players differ.
• Test with updates. Add a few new thumbnails with each major update, then leave the test alone until the next one.
• Keep your winner in. When a new test includes the current winner, it keeps most of the traffic while the new ones are measured (Roblox DevForum announcement).
• Follow the rules. Each image is moderated; see Roblox thumbnail size and rules.

Bloxshot makes Roblox thumbnails (1920×1080) and icons (512×512) from your game link, a full set at once so you can test them. It opens soon: see how it works or get one email at launch.

More guides:

• Roblox thumbnail size and rules
• Roblox game icon size and tips