TSI: An Ad Text Strength Indicator Using Text-to-CTR and Semantic-Ad-Similarity

Coming up with effective ad text is a time consuming process, andparticularly challenging for small businesses with limited advertis-ing experience. When an inexperienced advertiser onboards with apoorly written ad text, the ad platform has the opportunity to detectlow performing ad text, and provide improvement suggestions. Torealize this opportunity, we propose an ad text strength indicator(TSI) which: (i) predicts the click-through-rate (CTR) for an inputad text, (ii) fetches similar existing ads to create a neighborhoodaround the input ad, (iii) and compares the predicted CTRs in theneighborhood to declare whether the input ad is strong or weak.In addition, as suggestions for ad text improvement, TSI showsanonymized versions of superior ads (higher predicted CTR) inthe neighborhood. For (i), we propose a BERT based text-to-CTRmodel trained on impressions and clicks associated with an ad text.For (ii), we propose a sentence-BERT based semantic-ad-similaritymodel trained using weak labels from ad campaign setup data. Of-fline experiments demonstrate that our BERT based text-to-CTRmodel achieves a significant lift in CTR prediction AUC for coldstart (new) advertisers compared to bag-of-words based baselines.In addition, our semantic-textual-similarity model for similar adsretrieval achieves a precision@1 of0.93(for retrieving ads fromthe same product category); this is significantly higher comparedto unsupervised TF-IDF, word2vec, and sentence-BERT baselines.Finally, we share promising online results from advertisers in theYahoo (Verizon Media) ad platform where a variant of TSI wasimplemented with sub-second end-to-end latency