in Data Science
CatBoost leaf indexes: turn a tree ensemble into a sparse matrix a linear model can use, and why to fit it on rows the trees never saw
Aunt Rivka does not remember people by their names. She remembers them by where they sat. The cousin who always argued about the brisket was table four, left side. The one who owed her money was table nine, near the door. Ask about anyone and she gives you an address, and from the address she knows the rest.
in Data Science
CatBoost object importance: find the training rows that hurt your model, and drop them
Everyone knows which features matter. You plot the importances, you argue about the top three, you move on.
How CatBoost works: ordered target statistics, ordered boosting, and why a row should never see its own label
Uncle Shimon does the books for half the family, and for every wedding he predicts who will RSVP yes. His method is simple. He looks at each family’s track record. The Goldbergs said yes 80% of the time, so a Goldberg gets an 80% chance.
in Big Data
Data contracts in the age of agents that write back
My nephew Baruch spent a summer “helping” in my mother’s kitchen. He meant well. He also moved the salt to where the sugar used to be, because in his opinion that made more sense, and he never told anyone. Three weeks later my aunt made a brisket that could strip paint, and the investigation took longer than the brisket did.
Transformer Encoders: Or, How Multi-Head Attention Listens to Everything Simultaneously Without Missing the Point
[This is Part 3. Part 1 covers autoencoders. Part 2 covers self-attention.]