Case: AT&T Optimization Testing

Case: AT&T Optimization Testing

Testing Case: At&T(Which Test Won)

 Primary Analytics tools: HP Autonomy, HP Optimost

 Test Scenario: Optimization Testing: Overlay V/S Inline Form

 Known Objective: Increase Leads



At&T conducted  an A/B Test on visitors to the AT&T Enterprise Products and Service Page.

Test Results:

  • Inline Form, increased leads by 81.9% at a 99% confidence level.
  • Over 400 completed forms per month.
  • Preferred Means of Contact:
    • Email: 67%
    • Phone: 30%
    • Chat/IM: 2.4%

**Summarized from Which Test Won

At&T1                          at&t

Possible Reasons for Success:

This test seems to be more than just an A/B test, since it is testing various elements on the page. WhichTestWon results are unclear as they are attributing the results only to “Form Layout”. Following can be the possible reasons which collectively contributed to the success:

  • Visibility of CTA: It is easier to find in the Inline Form
  • Verbage in CTA:  “ Contact Me” v/s “Have Us Call You”.  There is a dispute in the Overlay form, as the link on the page says,” have us call you” while the form gices the visitors the option to choose contact method (phone or email”). There may be visitors who drop out of the site because of the CTA.
  • Reduction of compulsory fields: Inline Form seems less imposing because of the absence of “ *”
  • Reduction in form fields: “ Select a Service” is absent in the Inline form. This form field may be driving down the traffic if the visitor is not able to find a relevant option.

Possible Next Tests:

Now that AT&T is using Inline forms, what could they possibly be testing next? We need to design a multivariate test that allows us to explore alternative hypothesis about what might have driven the traffic. This will provide a more granular approach in personalizing customer experience on the website.


Multivariate Testing Chart

At&T can further explore and focus on incorporating behavioral targeting, so as to increase the conversions based on segment analysis.

Possible Segments:

  • Purpose: Business or Personal
  • Region
  • Gender
  • Number of Visits before a Visitor actually converted                 


Posted in CRM

Leave a Reply