Clustering App Ad Hooks in Go: TF-IDF, Cosine Similarity and a Hand-Labelled Sample, and the Walls You'll Hit (2026)
When I look at a competitor's ads, the first line matters most. The hook. "Stop living paycheck to paycheck." "No gym? No problem."
After copying fifty of them into a note, you see a few angles coming back, but it's hard to say which ones and how often. So I wanted a script that groups hooks by the words they share.
This post builds that script in Go, using only the standard library. It tokenizes a CSV of hooks, builds TF-IDF vectors, compares them with cosine similarity, and merges them with average-linkage agglomerative clustering. For each cluster it prints the top terms, a representative hook and the members. Each hook has a hand label (the app category), so the script can check how often a cluster mixes categories.
I ran everything below with Go 1.27.3 on Linux on 16 November 2026. The output excerpts are copied from those runs.
The data is made up, on purpose
The hooks.csv in this post is a made-up sample dataset . I wrote all 50 lines myself, in the style of habit, fitness and budgeting app ads. They are not real ads, and numbers like "1 million people" are not claims about any real app.
I added a few Spanish and German lines and some emoji, because real hook lists look like that. For real data, open the public ad libraries you can access, read the ads, and paste the first lines into the same CSV by hand. Respect each library's terms of service and don't scrape them. Fifty hooks from your own niche is a good start.
The format is three columns: id,label,hook
5,habit,Stop scrolling. Start a streak 🔥18,fitness,Drop 10 pounds before summer with 15-minute workouts35,budget,Stop living paycheck to paycheck
The label column is only for checking. The clustering never sees it.
Setup
mkdir adhooks && cd adhooks
go mod init adhooks
# save main.go and hooks.csv here, then:
go run .
The code
The whole program is about 200 lines. Save it as main.go :
// adhooks: cluster ad hooks (first lines of ad copy) with TF-IDF and cosine
// similarity, using only the Go standard library.
package main
import (
"encoding/csv"
"flag"
"fmt"
"math"
"os"
"sort"
"strings"
"unicode"
)
type Hook struct {
ID , Label , Text string
}
var stop = map [ string ] bool {
}
func init ( ) {
for _ , w := range strings . Fields ( `a an the and or but to of in on for with at by from is are was be it its this that these those you your i my me we our do does did don't doesn't isn't won't not no can will just here's it's what how why who when where than then so if as up out more most all one` ) {
stop [ w ] = true
}
}
// tokenize lowercases, splits on anything that isn't a letter, digit or
// apostrophe, drops stopwords and 1-letter tokens, and strips a plural "s".
func tokenize ( s string ) [ ] string {
f := strings . FieldsFunc ( strings . ToLower ( s ) , func ( r rune ) bool {
return ! unicode . IsLetter ( r ) && ! unicode . IsDigit ( r ) && r != '\'' && r != '’'
} )
var out [ ] string
for _ , w := range f {
w = strings . Trim ( strings . ReplaceAll ( w , "’" , "'" ) , "'" )
if len ( [ ] rune ( w ) ) < 2 || stop [ w ] {
continue
}
if len ( w ) > 3 && strings . HasSuffix ( w , "s" ) && ! strings . HasSuffix ( w , "ss" ) {
w = strings . TrimSuffix ( w , "s" )
}
out = append ( out , w )
}
return out
}
type Vec map [ string ] float64
// tfidf builds one L2-normalised vector per document.
// idf = ln((1+N)/(1+df)) + 1, the "smooth" variant, so no term gets weight 0.
func tfidf ( docs [ ] [ ] string ) ( [ ] Vec , map [ string ] float64 ) {
df := map [ string ] int {
}
for _ , d := range docs {
seen := map [ string ] bool {
}
for _ , t := range d {
if ! seen [ t ] {
df [ t ] , seen [ t ] = df [ t ] + 1 , true
}
}
}
n := float64 ( len ( docs ) )
idf := map [ string ] float64 {
}
for t , c := range df {
idf [ t ] = math . Log ( ( 1 + n ) / ( 1 + float64 ( c ) ) ) + 1
}
vecs := make ( [ ] Vec , len ( docs ) )
for i , d := range docs {
v := Vec {
}
for _ , t := range d {
v [ t ] += idf [ t ]
}
norm := math . Sqrt ( cosine ( v , v ) )
for t := range v {
v [ t ] /= norm
}
vecs [ i ] = v
}
return vecs , idf
}
// cosine of two unit vectors is just their dot product.
func cosine ( a , b Vec ) float64 {
var s float64
for t , x := range a {
s += x * b [ t ]
}
return s
}
// cluster runs average-linkage agglomerative clustering: keep merging the two
// clusters with the highest mean pairwise similarity until it drops below
// minSim, or until only k clusters are left (if k > 0).
func cluster ( sim [ ] [ ] float64 , minSim float64 , k int ) [ ] [ ] int {
cs := make ( [ ] [ ] int , len ( sim ) )
for i := range sim {
cs [ i ] = [ ] int {
i
}
}
avg := func ( a , b [ ] int ) float64 {
var s float64
for _ , i := range a {
for _ , j := range b {
s += sim [ i ] [ j ]
}
}
return s / float64 ( len ( a ) * len ( b ) )
}
for len ( cs ) > 1 && ( k <= 0 || len ( cs ) > k ) {
bi , bj , best := - 1 , - 1 , - 1.0
for i := range cs {
for j := i + 1 ; j < len ( cs ) ; j ++ {
if s := avg ( cs [ i ] , cs [ j ] ) ; s > best {
bi , bj , best = i , j , s
}
}
}
if k <= 0 && best < minSim {
break
}
cs [ bi ] = append ( cs [ bi ] , cs [ bj ] . . . )
cs = append ( cs [ : bj ] , cs [ bj + 1 : ] . . . )
}
sort . SliceStable ( cs , func ( i , j int ) bool {
return len ( cs [ i ] ) > len ( cs [ j ] )
} )
return cs
}
func main ( ) {
path := flag . String ( "f" , "hooks.csv" , "CSV with id,label,hook" )
minSim := flag . Float64 ( "sim" , 0.10 , "stop merging below this average similarity" )
k := flag . Int ( "k" , 0 , "stop at k clusters instead (0 = use -sim)" )
flag . Parse ( )
fh , err := os . Open ( * path )
rows , err2 := csv . NewReader ( fh ) . ReadAll ( )
if err != nil || err2 != nil || len ( rows ) < 2 {
fmt . Fprintln ( os . Stderr , "cannot read" , * path , err , err2 )
os . Exit ( 1 )
}
var hooks [ ] Hook
var docs [ ] [ ] string
for _ , r := range rows [ 1 : ] {
hooks = append ( hooks , Hook {
r [ 0 ] , r [ 1 ] , r [ 2 ]
} )
docs = append ( docs , tokenize ( r [ 2 ] ) )
}
vecs , idf := tfidf ( docs )
sim := make ( [ ] [ ] float64 , len ( vecs ) )
for i := range vecs {
sim [ i ] = make ( [ ] float64 , len ( vecs ) )
for j := range vecs {
sim [ i ] [ j ] = cosine ( vecs [ i ] , vecs [ j ] )
}
}
cs := cluster ( sim , * minSim , * k )
fmt . Printf ( "%d hooks, %d terms, %d clusters\n" , len ( hooks ) , len ( idf ) , len ( cs ) )
agree , singles := 0 , 0
for n , c := range cs {
if len ( c ) == 1 {
singles ++
continue
}
weight , labels := map [ string ] float64 {
} , map [ string ] int {
}
rep , repScore := c [ 0 ] , - 1.0
for _ , i := range c {
labels [ hooks [ i ] . Label ] ++
for t , x := range vecs [ i ] {
weight [ t ] += x
}
var s float64 // medoid: highest mean similarity to the others
for _ , j := range c {
if j != i {
s += sim [ i ] [ j ]
}
}
if s > repScore {
rep , repScore = i , s
}
}
terms := make ( [ ] string , 0 , len ( weight ) )
for t := range weight {
terms = append ( terms , t )
}
sort . Strings ( terms ) // alphabetical first, so ties stay stable between runs
sort . SliceStable ( terms , func ( a , b int ) bool {
return weight [ terms [ a ] ] > weight [ terms [ b ] ]
} )
top , most := "" , 0
for l , cnt := range labels {
if cnt > most || ( cnt == most && l < top ) {
top , most = l , cnt
}
}
agree += most
fmt . Printf ( "\n#%dsize=%dlabels=%v\n" , n + 1 , len ( c ) , labels )
fmt . Printf ( "top terms: %s\n" , strings . Join ( terms [ : min ( 4 , len ( terms ) ) ] , ", " ) )
fmt . Printf ( "representative: %q\n" , hooks [ rep ] . Text )
for _ , i := range c {
if i != rep {
fmt . Printf ( "- %s\n" , hooks [ i ] . Text )
}
}
}
fmt . Printf ( "\nsingletons: %d\n" , singles )
fmt . Printf ( "purity of clustered hooks: %d/%d\n" , agree , len ( hooks ) - singles )
}
A few choices worth explaining:
Tokenizing. Split on anything that isn't a letter, digit or apostrophe, so emoji and punctuation disappear. Accented letters survive, and curly apostrophes become straight ones. The only stemming is cutting a final "s".
TF-IDF. Common words get a lower weight, using the smooth formula ln((1+N)/(1+df)) + 1 . Each vector is normalised to length 1, so cosine similarity is a dot product.
Clustering. Every hook starts alone. The loop merges the two clusters with the highest average similarity until it drops below -sim (default 0.10) or -k clusters are left. With 50 hooks the O(n³) loop takes milliseconds.
Representative hook. The medoid: the member most similar to the others. It's always a real hook, which reads better than centroid weights.
main_test.go has three small tests (tokenizing, cosine, the two stop rules). go vet and go test pass.
What it found
go run . on the sample prints:
50 hooks, 192 terms, 23 clusters
#1size=6labels=map[budget:1 fitness:1 habit:4]
top terms: day, free, try, changed
representative: "Tiny habits. Big results. Try it free for 7 days"
I tracked one tiny habit a day for a year, here's what changedYour morning routine is broken. Fix it in 5 minutes a dayJournaling 2 minutes a day changed my morningsTry a free 7-day workout plan built for beginnersSave money on autopilot. Try it free for 7 days
#2size=4labels=map[fitness:4]
top terms: workout, minute, 15, 10
representative: "Drop 10 pounds before summer with 15-minute workouts"
No gym? No problem. Home workouts in 15 minutesBurn belly fat with a 10-minute daily workoutThe workout plan that adapts to your body
#3size=3labels=map[habit:3]
top terms: habit, build, 21, willpower
representative: "Build a habit in 21 days without willpower"
Don't break the chain: build daily habits that stickThe habit tracker that actually makes you keep your habits
#4size=3labels=map[fitness:2 habit:1]
top terms: start, stop, calorie, counting
representative: "Stop counting calories. Start eating smarter"
Stop scrolling. Start a streak 🔥Calorie counting made simple: snap a photo of your meal
#5size=3labels=map[budget:1 habit:2]
top terms: people, join, million, habit
representative: "Join 1 million people building better habits"
Why do most people quit new habits? (It's not motivation)Join 1 million people saving more every month
#6size=3labels=map[budget:1 fitness:1 habit:1]
top terms: finally, pov, checked, full
representative: "POV: you finally checked your subscriptions 😬"
POV: you finally kept a streak for 30 days 🔥🔥POV: you finally did a full push-up 💪
#7size=3labels=map[budget:2 fitness:1]
top terms: without, spend, feeling, less
representative: "Spend less without feeling broke"
Lose weight without giving up the food you loveTrack every dollar you spend without spreadsheets
#8size=3labels=map[budget:3]
top terms: budget, rule, 30, 20th
representative: "I saved $5000 in 6 months with one simple budget rule"
The 50/30/20 budget rule explained in 30 secondsBroke by the 20th? Here's a budget that works
#9size=2labels=map[fitness:1 habit:1]
top terms: ohne, aufbauen, gewohnheiten, willenskraft
representative: "Gewohnheiten aufbauen ohne Willenskraft"
Abnehmen ohne Fitnessstudio: 15 Minuten am Tag ... singletons: 8 purity of clustered hooks: 33/42
Cluster #2 is the "short home workout" angle, #3 is "build a habit without willpower", and #8 is the budget rule. Each has one label.
The mixed clusters are more interesting. #6 is three "POV: you finally..." hooks, one per category. #5 joins the two "Join 1 million people" lines, and #1 collects "Try it free for 7 days" hooks from all three apps. As category checks they are mistakes. For ad research they show a format used across niches.
33 of 42 clustered hooks share their cluster's main label, and 8 hooks stayed alone. I treat that as a number to watch, not a goal.
The walls you'll hit
- Hooks are too short for TF-IDF. The average hook in my sample has 5.7 tokens after stopword removal. 142 of the 192 distinct terms appear in only one hook, and 1,075 of the 1,225 hook pairs have a similarity of exactly 0. So the clustering decides on one or two shared words. One "without" put "Lose weight without giving up the food you love" and "Spend less without feeling broke" together (#7). A longer stopword list helps a little. Adding each ad's second line helps more.
- Synonyms don't match. "Lose weight", "Drop 10 pounds" and "Shed those extra kilos" mean the same thing to a person. To TF-IDF, "Lose weight without giving up the food you love" and "Drop 10 pounds before summer with 15-minute workouts" have a similarity of 0.000. "I lost 20 pounds" doesn't match "lose" either, because my stemmer only removes an "s". Embeddings fix this, but then you need a model. A small synonym map you maintain yourself ( pounds, kilos, weight → weight ) is a decent middle ground for one niche.
- Tokenizing decides more than you'd think. Emoji are dropped, so "🔥" never links the two streak hooks. Fine for topics, bad if you study emoji use. "50/30/20" becomes three number tokens. Copied ad text often has curly apostrophes, and without the normalising line "don’t" would slip past the stopword list. "Habits" becomes "habit", but "spending" and "spend" stay apart. Print the tokens for ten hooks before you trust any cluster.
- Choosing the threshold or k is the real work. Same data, different -sim values:
-sim clusters singletons purity 0.05 16 4 31/46 0.10 23 8 33/42 0.15 28 12 30/38 0.20 34 22 21/28 0.30 41 34 10/16
Lower thresholds cluster more hooks but mix more categories. Higher ones leave most hooks alone. -k 3 was worse: one cluster held 46 of the 50 hooks, with 15 or 16 of each label. Once it's forced to keep merging, average linkage on sparse vectors chains everything together. I pick a threshold, read the clusters and adjust by hand. Your data will need a different number.
- Languages cluster by language, not by topic. The stopword list is English only. Cluster #9 joined a German habit hook and a German fitness hook because both contain "ohne" (without). Cluster #12 joined a Spanish fitness hook and a Spanish budget hook through "sin" and "de". "Crea hábitos que duran" never matched an English habit hook. Split hooks by language first and give eac
Comments
No comments yet. Start the discussion.