AI Loop

Conversion, one test every week.

One agent that runs conversion as a weekly loop. Every Monday it reads the funnel, sees what competitors changed, finds where yours leaks and picks one thing to test. Then it writes the variant, builds it and sets the test up to be trusted. Nothing changes until you've said yes. Two weeks later it reads the result.

Step 01 · The funnel

We read where
the money leaked

Every Monday the agent pulls last week's funnel straight from the sources. Not one conversion rate, but every step: visits, who reached a room and a price, who started to book and who paid — split by phone and desktop, and by where the visitor came from.

  • Visits, booking page, checkout, paid — vs last week, last year
  • The same four numbers on mobile and on desktop, kept apart
  • Per source: Google, the OTAs' billboard effect, email, paid
  • Load time on a phone, measured the way Google measures it
  • Bookings and revenue — the two numbers everything is judged by

The site

Visits24,600
On a phone62 %
Load time · mobile3.4 s
Reading the weekDone

Booking page

Saw a room and a price8,900
Left without dates4,120
Started to book2,140
Reading the weekDone

Checkout

Started2,140
Completed415
Gave up81 %
Reading the weekDone

Bookings

Bookings415
Revenue · SEK1,482,000
Visit → booking1.7 %
Reading the weekDone
Step 02 · Competitors' flows

We see what
the others changed

Then the agent walks through the competitors' booking flows, the way a guest would — the room list, the price, the steps to pay. It keeps last week's copy of every page, so what changed shows up on its own. The OTAs are on the list too, because that's where the guest compares.

  • Price per night or whole stay, fees up front or at the end
  • How many steps to pay, and whether an account is required
  • By the button: free cancellation, best price, pay at the hotel
  • "2 rooms left", "booked 14 times" — and whether it's true
  • What changed since last Monday, page by page

Booking flows · since last Monday

Competitor A total price now on the room list
Competitor B unchanged · 4 steps, account required
Competitor C "only 2 left" on every room
Booking.com free cancellation moved above the price
Expedia unchanged · our rate SEK 40 lower there
Reading 1 of 5 Three flows changed
Step 03 · The leak

We find the step
where people give up

A funnel doesn't leak evenly. Most of the money goes at one step, on one device. So the agent lays the four steps side by side and finds the biggest drop — then splits it by phone and desktop, and checks whether it's the page, the form or the load time that's doing it.

  • Drop-off per step, in people and in kronor
  • The same step on mobile and desktop — rarely the same leak
  • The form field people stop at, and how long they sat on it
  • Load time at the step: every second past three costs bookings
  • One leak named, with the number it costs per week

Where the week leaked

Visits24,600
Saw a room and a price8,900
–64 % · never picked dates
Started to book2,140
–76 % · saw the price, didn't start
Completed415
–81 % · gave up in checkoutthe leak
Mobile · gave up86 %at the step where fees appear
Desktop · gave up71 %same step, half as bad
1,725 started and left≈ SEK 540,000 a week
Step 04 · Signal watch

We catch what's
moving the numbers

A funnel moves for reasons that have nothing to do with the page. School holidays, a rate change, a new phone model, a competitor's offer. The agent keeps watch on all of it, so a drop isn't blamed on a button when it was the season — and so a real chance isn't missed.

  • Season: holidays, events and the weeks when families search
  • Our own price changes, and what they did to the booking page
  • Device shift: a new phone, a new browser, a new traffic mix
  • A competitor's new offer, or a new badge on the OTAs
  • What broke: a form error, a slow script, a failed payment

Signal feed

Watching continuously
Step 05 · The decision

Everything weighed
into one test

This is where it all meets. The funnel, the competitors, the leak and the signals are laid side by side, and the agent picks one thing to test this week. One, because two changes at once can't be told apart afterwards. It also sets the smallest effect worth finding — if the test can't see that, it's not worth running.

  • One test, at the step that leaks the most money
  • The hypothesis: what we change, what it will do, and for whom
  • The smallest effect worth finding — here 20 % more bookings
  • Visitors and weeks needed — whole weeks, never fewer than two
  • The guardrail: what must not get worse while we win

This week's test

The funnel
Competitors
The leak
Signals
Decisions
  • Test: the whole-stay price on the room list, taxes included
  • Where: the fee step at checkout — 81 % leak, 86 % on a phone
  • Smallest effect worth finding: +20 % booking page → booking
  • 14 days, 8,100 visitors per variant, 95 % confidence
Step 06 · The hypothesis

We write down
what we expect

Before anything is built, the test is written as a sentence you can be wrong about. Because we saw this, we believe that change will do this, for these visitors, and we'll know by this number. A test without one of those five parts is a guess with a dashboard.

  • Because: the observation, with the number from step 03
  • We believe: the one change, described so it can't be misread
  • Will cause: the outcome, and the smallest lift worth finding
  • For: which visitors — here everyone, mobile read separately
  • We'll know when: the metric, the confidence, and on which day

Test card · wk 38

Because
86 % on mobile leave at the step where fees appear
We believe
Showing the total price on the room list
Will cause
+20 % bookings from the booking page
For
All visitors · mobile read on its own
We'll know when
Day 14 · 95 % confidence · Sunday 4 October
Step 07 · The variant

One thing changes.
The rest holds still

The variant is the control with exactly one thing different. Everything else the week turned up is sorted into two other piles: plain fixes that don't need a test — a broken date picker, a field nobody uses — and good ideas that wait their turn. A test only answers one question at a time.

  • Change: the price line and the words around it, nothing else
  • Fix without testing: what is simply broken or pointless
  • Queue: the next tests, in the order the leaks are worth
  • Variant tested on the slowest phone we see, not the designer's
  • Both versions load equally fast — or speed becomes the test

What changes this week

Under test one change
Total price on the room list Taxes and fees included "Nothing added at checkout"
Fixed no test needed
Native date picker on mobile Phone field removed Newsletter box unticked
Queued next weeks
Guest checkout Free cancellation under the button Exit popup
One change measured. The rest is fixed, or waits.
Step 08 · AI agents

The right agents
are put on the job

Now the right agents are brought in. One knows pages and forms and why people stop, one knows how to run a test that can be trusted, one knows popups and when they help rather than annoy, one knows the flow after the click, one knows why people decide, and one makes sure every step is measured.

  • One reads the page as a visitor does: promise, price, button
  • One sets the sample, weeks and split — and never stops early
  • One knows popups and forms: fewer fields, the right moment
  • One knows what follows the booking: confirmation, first email
  • One checks the tracking, so the answer isn't a tracking error

Agents on the job

cro
StartingWorking
ab-testing
StartingWorking
popups
StartingWorking
signup
StartingWorking
onboarding
StartingWorking
marketing-psychology
StartingWorking
analytics
StartingWorking
Step 09 · The copy

Then the variant
gets written

This is where the change becomes words. The price line and the sentence under it, the button, the trust line next to it, the popup for the visitor about to leave, and the form with the fields that are left. Short, concrete, and nothing the hotel can't stand behind.

  • The price as the guest pays it: the whole stay, all included
  • A button that says what happens next, not "Submit"
  • Trust in a fact: cancel free until when, best price vs whom
  • Urgency only when it's true — "2 rooms left" means two rooms
  • Every line is tagged with its test, so it can be judged later

Writing the variant

Price line · the room list
Price Button Trust Headline Popup Form
Step 10 · The test setup

Enough people,
enough days

A test that stops the day it looks good is how most teams fool themselves. So the sample size and the end date are set before it starts, from the baseline and the smallest effect worth finding. Whole weeks, at least two, because Tuesday's visitors don't book like Saturday's.

  • Baseline 4.7 % → find +20 %: about 8,100 visitors per variant
  • 8,900 booking-page visitors a week → 14 days, ending Sunday
  • 50/50, and a visitor keeps the same version if they come back
  • Metric: booking page → booking. Guardrail: revenue per booking
  • No peeking, no stopping early — the end date is the end date

Test · total-price-first

page /rooms · mobile + desktop split 50 / 50 · sticky per visitor metric booking page → booking baseline 4.7 % mde +20 % relative → 5.6 % sample 8,100 per variant runs 14 days · Mon 21 Sep → Sun 4 Oct guardrail revenue per booking · not down // no peeking before day 14
Configured · 95 % confidence · 80 % power
Step 11 · Approval

Here everything stops
and waits for you

The week is done. The variant, the fixes and the queue sit ready exactly as they'll look on the site — nine drafts, each marked test, fix or queued. Nothing changes on the site until someone says yes.

Showing 1–3 of 9 drafts
H Test · wk 38
/rooms · the price line · draft
HHotellethotellet.se › rooms › double room
Control · AFrom SEK 1,495 per night · taxes and fees added at checkout
Variant · BSEK 3,590 for 2 nights · taxes included · nothing added at checkout
50/50 · 14 daysGoal: +20 %
H Hubso · email
Monday 08:00 · to the GM · draft
This week's test: the total price, first
One test · one hypothesis · your yes
Last week 2,140 people started to book and 1,725 gave up — 86 % of them on a phone, at the step where the fees show up. We believe showing the whole price on the room list lifts bookings from the booking page by 20 %. 14 days, 8,100 visitors per version, answer on Sunday 4 October. Nothing goes live until you say yes.
Approve the test
H Queued · wk 40
exit popup · mobile back-swipe · draft
× Before you go

Keep these dates for 24 hours?

Fri 9 – Sun 11 Oct · Double room · SEK 3,590 total. We'll hold the price, no card needed.

Hold my dates No thanks
Shows once · only after dates pickedCloses in one tap
H Queued · wk 39
checkout · step 2 · draft
Now · 14 fieldsCreate an account to continue · name, email, password, phone, address, company…
Proposed · 6 fieldsContinue as guest · name, email, card. Address and phone asked at check-in
14 → 6 fieldsBaymard: 18 % quit over forced accounts
H Test · wk 38
/spa-weekend · price framing · draft
HHotellethotellet.se › spa-weekend
Control · ASpa weekend from SEK 1,795 per person per night
Variant · BSpa weekend for two · SEK 7,180 · two nights, dinner, spa and breakfast — all in
Same change, second pageRead together
H Queued · wk 39
/rooms · trust line · draft
Now · in the footerFree cancellation · Best price guarantee · Secure payment
Proposed · under the buttonFree cancellation until 18:00 the day before · Same price as Booking.com, breakfast included here
Moved 1,400 px upFacts, not badges
H Fix · no test
mobile only · 62 % of visits · draft
NowDate picker loads a 380 kB calendar script · 3.4 s on a phone
ProposedThe phone's own date picker · the Book button stays at the bottom while you scroll
3.4 s → 1.9 sOutside the test
H Queued · wk 41
homepage · headline · draft
HHotellethotellet.se
NowWelcome to Hotellet
ProposedSpa hotel by the sea. Book direct — same price as the OTAs, breakfast and late checkout on us.
Says what, where, why directTest after the price
H Queued · wk 42
/rooms · the button · draft
NowBook now
ProposedSee rooms and total price
Says what happens nextSecond button in wk 43
Step 12 · The test runs

Two weeks
without touching it

After your yes, the test goes live on Monday morning and the fixes go out beside it, outside the test. Then the hardest part: nothing is decided until day 14. The agent checks every day that both versions work, that the tracking fires, and that the guardrail holds — and reports without calling a winner.

  • Monday 09:00: 50/50 live, both checked on six real phones
  • Day one: tracking checked, every event fires on both versions
  • The fixes ship the same week, clearly marked as not the test
  • A daily line to the team: visitors per variant and guardrail
  • Stopped early only if the guardrail breaks — not for good news

21 September – 4 October

Mon 09:00 Test live · 50/50live
Mon 09:40 Both versions checked on 6 phonesdone
Tue Tracking verified · 8 events, both sidesdone
Thu Mobile fixes shipped · outside the testdone
Sun 27 Week one · 4,300 per variant · no peekingweek 1
Wed 30 Guardrail check · revenue per booking holdsongoing
Sun 4 Oct Day 14 · the readqueued
Mon 5 Oct Winner to 100 % · next test startsqueued
Test in progress
Step 13 · The read

We read the test
the way it was set up

On day 14 the answer is read exactly as it was written on day one: the primary metric, at 95 %, on the full sample. Then mobile on its own, because that's where the leak was. A winner goes to everyone. A loser is a fact worth as much — most tests lose, and the ones that win rarely win by much.

  • Control vs variant on the one chosen metric, nothing else
  • Mobile and desktop read separately — a win can't hide a loss
  • The guardrail: revenue per booking, cancellations, tickets
  • Lift in bookings and kronor per week, at this season's traffic
  • Winner to 100 % on Monday — loser rolled back and written down

Day 14 of 14 · booking page → booking

0.0 %
Control · A 8,140 · 383 booked
0.0 %
Variant · B 8,090 · 445 booked
Confidence0 % · needs 95
Lift+17 % · 62 more bookings
Mobile on its own+24 % · 96 %
Guardrail · revenue per bookingSEK 3,571 → 3,604
Reading the sampleSignificant · variant to 100 % on Monday
Step 14 · And then it starts again

Next Monday
it starts again

The week after, step 1 reads a funnel that has one leak fewer, and the next test in the queue moves up. Every result — win, loss or nothing — goes into a library with the hypothesis, the number and the reason. After a year the site has been tested fifty times, and it's built on the hotel's own guests, not someone else's best practice.

  • Every test judged against the hypothesis it was written with
  • Winners stay, re-checked next season — October isn't July
  • Losers logged with the reason, so nothing is tested twice
  • The queue re-ordered every Monday by what the leaks cost now
  • A year on: fifty tests, less leakage, proof of what guests do
Week 38 And again 14 steps · every week