Methodology

1. Sources we sample

We aim for at least 40 distinct mentions per review. Sources fall into four buckets:

• Hacker News — story comments and Ask HN threads via the Algolia search API. High signal, dispersed crowd.
• Independent blogs — Medium, Substack, personal sites and niche-specific publications (dev blogs, design journals, language-learning logs). We prioritise authors with no affiliate relationship to the course.
• Community forums — course-specific forums and Discord communities, Stack Overflow / Stack Exchange tags, GitHub Issues for the course's open repos when they exist.
• Open course-platform reviews — Coursera and similar, when the platform makes them publicly readable.

What we don't sample: Reddit and YouTube block automated reading, so we honestly say so on each review instead of pretending.

2. Anti-manipulation filters

Before AI counts a mention, it has to pass three quality gates:

• Not promotional — affiliate-laden posts that read like marketing copy are discarded.
• Not aggregator-spam — sites that re-publish the same paragraphs across many courses don't count as independent voices.
• Not coordinated — astroturfing patterns (many identical-tone reviews in a narrow window) are weighted down or excluded.

3. Every quote is real

For each quote shown on a review page:

• The source URL opens and actually contains that text.
• The author handle matches what's visible on the platform.
• The date matches the timestamp at the source.
• No quote is fabricated. If AI cannot find a verifiable quote, the review simply has fewer quotes — never invented ones.

4. Sentiment & recency weighting

Each mention is classified as positive, neutral or negative based on the opinion about this specific course, not the poster's general mood.

Recent opinions matter more — an opinion from the current year is weighted roughly 2× one from five years ago. Courses age out of relevance; their content and instructors change.

5. The formula

Two-step computation that penalises small samples and weights by the positivity ratio:

Step 1 — Bayesian average

bayesian = (C × m + sum_of_scores) / (C + n)

With C = 20 (prior weight — like 20 imaginary reviews at the global mean) and m = 3.5 (neutral prior). A course with 3 perfect reviews scores lower than a course with 200 mostly-good reviews. Small sample, low confidence, lower score.

Step 2 — Positivity multiplier

positivity = positive / (positive + negative)
multiplier = 0.85 + 0.30 × positivity 

A course where 90% of opinions are positive scores higher than one where 50% are — even if the raw averages match. Negative outliers matter.

Final score

final = clamp(bayesian × multiplier, 0, 5)

6. Per-criterion scores

The overall number is an average of five sub-scores — each computed independently from mentions about that criterion:

• Content quality — depth, accuracy, currency
• Instructor — clarity, engagement, teaching ability
• Value for money — what you get vs what you pay
• Support — forums, mentors, response time
• Real-world applicability — does it transfer to actual work

Some criteria may vary by niche — design courses care about portfolio outcomes more than theoretical depth, language courses care about retention more than instructor charisma.

7. What the human editor does

The editor-in-chief reviews every article before publication, but with a strict scope:

• ✓ Spot-checks 2–3 quotes by opening the source URL
• ✓ Flags obvious factual errors (wrong course name, wrong price, etc.)
• ✗ Does not change the score — ever. That's the whole point of letting AI compute it.

8. Limitations we admit

• Only public opinions are visible — the silent majority isn't represented.
• English-language sources only. Russian, Spanish, Chinese course communities are not sampled.
• Reddit and YouTube aren't reachable for automated reading — we disclose this on every affected review.
• We don't sample Trustpilot or G2 — they require paid scraping APIs.

9. The affiliate firewall

Some links on the site are affiliate links — we may earn a commission when you buy through them. Two firewalls exist between affiliate revenue and the score:

• AI doesn't know which courses pay commission when it computes the score.
• The editor can't override the score, regardless of commercial pressure.

Proof: the highest-commission affiliate in our current catalog is also the lowest-scoring course we review. We didn't dial the number to keep them happy.