We built a skincare rating.
Then we killed it. Twice.

How the two big scores are built

Neither app hides its method, and both are worth reading in the original. The mechanics below are theirs, described from their own documentation.

Four ways an ingredient score goes wrong

1. 01

   One ingredient decides everything

   Yuka sets a product’s band from its single worst ingredient. One red ingredient forces the score under 25 out of 100 whatever else is in the bottle. It guarantees a wide spread, which is exactly what our own flat rubric lacked, and it means a formula is judged by its least flattering line.

   The rule we took from itWeight by position. Never take the maximum.

2. 02

   Hazard is not risk

   A substance can be hazardous in a beaker and irrelevant in a bottle at the amount actually used. Cosmetic chemists make this point about both apps: the scores flag presence, not exposure. Salicylic acid, titanium dioxide and phenoxyethanol all score badly on apps that ignore how much is there.

   The rule we took from itAn ingredient list gives order, never percentage. Say so.

3. 03

   Following the law costs you points

   European rules require an essential oil to declare its allergens separately by name. Apps that count each declared name as another additive therefore penalise the brands that complied and reward the ones that did not. It is the cleanest example of a metric measuring paperwork rather than product.

   The rule we took from itA declared substance must never cost more than an undeclared one.

4. 04

   Not knowing is treated as bad news

   EWG applies a penalty for missing data, so a newer, well-studied-in-a-different-way molecule scores worse than an old one simply for having a thinner file. The rating ends up measuring how long something has been around.

   The rule we took from itUnknown is not bad. Exclude it from the score and publish the coverage instead.

Test one

We scored safety, and everything came out the same

The obvious rating is a safety score. Take irritation, take pore-clogging, take how well studied the actives are, average them, print stars. We built it and ran it across everything we publish. Fifteen of nineteen products came out at four stars or more, inside a band of about a third of a star.

The reason turned out to be more interesting than the failure. Our own ingredient library says low pore-clogging risk for 89%of the 211 ingredients in it, and low irritation risk for70%. You cannot separate products on a measure where almost everything scores the same.

So we tested the one list that is a matter of law rather than opinion: the twenty-six fragrance allergens European regulations require to be named on a label. Across 107products from the brands we cover, 92 had nothing to declare at all. Standard deviation across the whole set: 0.36, against the0.5 we had written down in advance as the pass mark.

Standard deviation by category. Every one of them flat.

We could have fixed the spread in an afternoon by covering the perfumed drugstore products where these numbers vary enormously. That would have meant choosing what to write about in order to make our own rating look clever. The rating was the thing we were prepared to lose.

Test two

We scored value, and it passed everything except the point

The second idea was better. Forget whether a product is good, ask how much formula you get for the money. Price is the one thing our shelf is not pre-selected for: the brands we cover run from about five euros to about thirty-five, and what goes in the bottle at those prices genuinely differs.

It passed on the numbers, comfortably, on both sets we ran. Our own products: standard deviation 1.19. A wider set of 92 labels from the same brands: 1.91. Against a pass mark of 0.5, with both ends of the scale occupied and every category clearing the bar on its own.

Then we read the list.

The cause was in our own method. To compare a cleanser with a cleanser rather than with a serum, the rubric needs a benchmark per category, and we had picked those numbers ourselves. Ours made it very easy for a cleanser to reach full marks and very hard for a serum to.11 of 18 products sat on the ceiling. The score was not reporting a relationship between formula and price. It was reporting a constant we chose.

The wider set failed the same condition for a second reason:14 of its 92 labels scored zero, every one of them because we could not decode a single active ingredient on them. That is a fact about our library, not about the products.

The fix would have taken ten minutes. Adjust the constants until the retinol serum scores well, then publish. Not doing that is the entire point of writing the pass mark down first.

The number nobody publishes about themselves

Every score above presents itself as equally confident whether it read the whole label or half of it. Here is ours. Across 106 labels from the brands we cover, our library currently decodes a median of 43% of the ingredients on a page, and only 7 of them pass the eighty-percent bar we would want before rating anything.

We would rather print that than imply we read everything. It is also the reason the value test could not be rescued: a distribution partly produced by what we cannot read is not a measurement of what we can.