Experiment context
Hypothesis. Pinned USP headlines were expected to improve CTR by drawing attention and improve CR by pre-qualifying users before the click. The economic target was lower CPA despite possible CPC growth.
How the test ran. Bidding and targeting were held constant to isolate the creative effect; weaker directions were used first to limit risk.
Traffic split. 50/50.
Visual summary — winning combinations
The strongest pinned-headline variants improved conversion efficiency in five game and service combinations.
Performance results
| Label | Impr. | Interactions | Interaction rate | Avg. CPC | Cost | Conversions | CPA | Conv. rate |
|---|---|---|---|---|---|---|---|---|
| New Ad Format | 6,771 | 987 | 14.58% | 2.02$ | 1,990.74$ | 180.56 | 11.03$ | 18.29% |
| Old Ad Format | 7,073 | 1,099 | 15.54% | 1.99$ | 2,191.51$ | 159.45 | 13.74$ | 14.51% |
| Change, % | 95.73% | 89.81% | 93.81% | 101.15% | 90.84% | 113.24% | 80.22% | 126.09% |
Result. Successful: CR increased 26% and CPA decreased 20% for service3; roll out the pinned format.
| Label | Impr. | Interactions | Interaction rate | Avg. CPC | Cost | Conversions | CPA | Conv. rate |
|---|---|---|---|---|---|---|---|---|
| New Ad Format | 9,897 | 618 | 6.24% | 2.00$ | 1,233.81$ | 43.15 | 28.59$ | 6.98% |
| Old Ad Format | 19,511 | 1,249 | 6.40% | 1.86$ | 2,319.55$ | 54.41 | 42.63$ | 4.36% |
| Change, % | 50.73% | 49.48% | 97.54% | 107.50% | 53.19% | 79.31% | 67.07% | 160.28% |
Result. Successful: CR increased 60% and CPA decreased 33% for service4 despite lower traffic; roll out the pinned format.
| Label | Impr. | Interactions | Interaction rate | Avg. CPC | Cost | Conversions | CPA | Conv. rate |
|---|---|---|---|---|---|---|---|---|
| New Ad Format | 7,542 | 539 | 7.15% | 2.16$ | 1,162.45$ | 21.91 | 53.06$ | 4.06% |
| Old Ad Format | 9,294 | 633 | 6.81% | 2.12$ | 1,343.24$ | 18.22 | 73.72$ | 2.88% |
| Change, % | 81.15% | 85.15% | 104.93% | 101.63% | 86.54% | 120.25% | 71.97% | 141.22% |
Result. Successful for service4: CR increased 60% and CPA decreased 33%; roll out selectively.
| Label | Impr. | Interactions | Interaction rate | Avg. CPC | Cost | Conversions | CPA | Conv. rate |
|---|---|---|---|---|---|---|---|---|
| New Ad Format | 15,496 | 1,926 | 12.43% | 2.12$ | 4,085.69$ | 109.19 | 37.42$ | 5.67% |
| Old Ad Format | 6,521 | 714 | 10.95% | 2.20$ | 1,567.96$ | 32.12 | 48.82$ | 4.50% |
| Change, % | 237.63% | 269.75% | 113.51% | 96.60% | 260.57% | 339.94% | 76.65% | 126.02% |
Result. Successful for service5: conversions increased 240% and CPA decreased 44%; roll out.
| Label | Impr. | Interactions | Interaction rate | Avg. CPC | Cost | Conversions | CPA | Conv. rate |
|---|---|---|---|---|---|---|---|---|
| New Ad Format | 1,549 | 208 | 13.43% | 1.64$ | 341.23$ | 14.89 | 22.92$ | 7.16% |
| Old Ad Format | 2,539 | 372 | 14.65% | 1.55$ | 575.11$ | 44.61 | 12.89$ | 11.99% |
| Change, % | 61.01% | 55.91% | 91.65% | 106.11% | 59.33% | 33.38% | 177.76% | 59.70% |
Result. Unsuccessful for service5: conversions were 2.2× lower and CPA was 2.7× higher; retain the control.
| Label | Impr. | Interactions | Interaction rate | Avg. CPC | Cost | Conversions | CPA | Conv. rate |
|---|---|---|---|---|---|---|---|---|
| New Ad Format | 525 | 52 | 9.90% | 1.44$ | 74.93$ | 0 | N/A | 0.00% |
| Old Ad Format | 737 | 79 | 10.72% | 1.40$ | 110.28$ | 6.50 | 16.97$ | 8.23% |
| Change, % | 71.23% | 65.82% | 92.40% | 103.22% | 67.95% | 0.00% | N/A | 0.00% |
Result. Successful for service4: CR increased 28% and CPA decreased 29%; roll out only for this direction.
Conclusions
Overall verdict: mostly successful. Pinned headlines improved conversion efficiency for game-2 service3/service4, game-5 service4, game-6 service5 and game-7 service4. Roll out the pinned format for these combinations. Retain the control format for game-7 service5, where conversions decreased and CPA increased.