418 stories
·
5 followers

This Is Probably A Good Time To Say That I Don’t Believe Robots Will Eat All The Jobs

1 Comment and 2 Shares

As consumers, we virtually never resist technology change that provides us with better products and services even when it costs jobs. Nor should we.

#

Read the whole story
sillygwailo
1 day ago
reply
OK, but what about politics?
Toronto, ON
Share this story
Delete

The Imperfect Pursuit of a Perfect Baseball Forecast

1 Comment

As you navigate through the sports media this week, chances are you’ll see at least one set of predictions for the upcoming Major League Baseball season. This preseason forecasting game was being played as far back as 1929, and the prediction options fans have at their disposal today are more plentiful — and sophisticated — than ever before.

But they can never be perfect. There’s a statistical limit to how accurate any projection about a team can be in the long run. Years ago, sabermetrician Tom Tango researched the amount of talent and luck that go into team winning percentages and found that chance explains one-third of the difference between two teams’ records. That makes it hard to predict how many times a team will win over a season. The smallest possible root-mean-square error(a mathematical way of testing a prediction’s accuracy) for any projection system over an extended period of time is 6.4 wins. In a single season, forecasters can — and do — beat an RMSE of 6.4. But whenever that happens, it’s due to luck. The amount of random variance that goes into team records makes the 6.4 barrier literally impossible to beat over a large number of seasons. 54 1 Over time, no forecaster’s system can ever do better.

But baseball fans still clamor for the projections, and so the cottage industry of preseason forecasts lives on. Modern projections come from three different directions: analytics, expertise and the wisdom of the crowd. All three are more or less equally accurate despite their vastly divergent inner workings (and even motivations), andnone gets anywhere close to that 6.4-win mark over multiple seasons. Not that it’s stopped folks from trying.

The sabermetricians who attempt todivine a team’s success in the coming season are relying on algorithms to do most of the heavy lifting. What makes up those algorithms varies. The general idea of a computer projection system such as Baseball Prospectus’PECOTA 55 2 is to take a player’s past performance, 56 3 regress it towards the mean to account for the fact that statistics are an imperfect measurement of talent, and adjust it for aging effects. A simple systemthat performs these three basic tasks will be practically as accurate as ones with far greater complexity. The competition comes in the margins: PECOTA, for instance, adds the extra wrinkle of basing its aging curves for any given player on the career paths of comparable historical players.

However elaborate the projection model, it should spit out a set of forecasted statistics for each individual player, which can then be fed into team-depth-chart projections to generate win predictions for every team. In PECOTA’s case those predictions have come within an RMSE of 8.9 wins, 2.5 wins away from perfection. 57 4

Those who trust algorithmic projections say they do so because of the projections’ empiricism: While plenty of simplifications and assumptions are being made throughout the process, those hypotheses are at least applied consistently across every team. Of course, computer projections are far from infallible. Predicting injuries, for example, is a crucial — yet cruelly unscientific — aspect of team forecasting. To a certain extent, durability can be predicted from a player’s prior history, but the degree to which we can reliably forecast injuries is still quite modest. A more qualitative assessment of a player’s current talent can take into account injuries, mechanical changes, managerial whims and other more nuanced factors that are too fine-grained for pure data-processing methods to capture.

These are the types of appraisals potentially more suited to a process like the one employed by Sports Illustrated’s team of preseason forecasters. Every year, SI, the most widely read sports magazine in the United States, produces a baseball preview that reaches more than 3 million readers.

Its baseball editor, Stephen Cannella, said there’s nothing algorithmic about the magazine’s prediction process. “It’s more a combination of statistics, scouting reports and ‘boots on the ground’ reporting,” he said. Essentially, Cannella’s system taps into the wisdom of a very baseball-savvy crowd. He starts with a basic straw poll of his writers and asks them to rank the relative strength of every major league club. Later, Cannella and his team convene to debate the results and come to a consensus about each team’s projection.

Cannella said he does use statistical forecasts like PECOTA as a sanity check once his team finishes constructing its rankings. “Projections are most worthwhile as a comparison,” he said. According to Cannella, this type of synthesis using both the objective and subjective “reflects SI’s whole baseball approach.”

It turns out that approach is just as good as the wholly quantitative one. I evaluated the accuracy of Sports Illustrated’s divisional picks 58 5 using a ranked square-error method and found that, since 2005, Sports Illustrated’s forecasts have been — statistically speaking 59 6 — no less accurate about where a team finishes in its division than PECOTA or Las Vegas’ over/under win totals. 60 7 Cannella’s approach may not be the most rigorously mathematical, but it works.

Speaking of those Vegas over/unders, they’re the most crowdsourced projections we have. Every spring, various sportsbooks release baseline win totals for each team in the upcoming season, inviting bettors to place wagers on whether they believe a team will win more or less games than the oddsmakers think. These over/unders are not technically predictions — their aim is different from PECOTA’s or even Sports Illustrated’s — but they’re hugely predictive because of the stakes involved.

To get a sense of how Vegas sets these over/unders, I spoke with Ed Salmons, head oddsmaker for the Las Vegas Hotel & Casino, which cleared nearly 1 million sports bets in 2013. Like Cannella, Salmons doesn’t use a strictly algorithmic approach, although he is well-versed in sabermetrics. “I look at computerized [projections] before putting numbers out,” he said. “You’d be foolish not to.”

In addition to using public systems like PECOTA to audit its over/unders, the LVH also has several internal computer models dedicated to prediction. But Salmons isn’t necessarily trying to maximize pure predictive accuracy. “You can’t let the computer go crazy,” he said. “I also have to put out a number I think is good for the marketplace.”

Salmons’ pet example is the 2014 Houston Astros. He hinted that his in-house algorithms call for Houston to win between 65 and 70 games this year (a range also in accordance withPECOTA and Fangraphs’ Steamer), but the Astros are coming off a historically terrible season in which they won just 51 games. The casual bettor is much more likely to see the Astros as a god-awful team than a typically bad one. As Salmons put it, “The public doesn’t even know what ‘regression to the mean’ is.”

So Salmons undershot what he himself felt was most accurate and set the Astros’ over/under at 63.5 wins. That’s in keeping with his guiding philosophy of putting ”the highest number out there that the wise-guys will stay away from.” Salmons knew that if he made the projection as high as he wanted, he’d have the wrong balance of bettors. He never wants more than 30 percent of people who bet to be professional sports bettors — otherwise it’s possible he’ll lose too much money. So he had to drop Houston’s win total lower than his computers predicted. 61 8

Despite a professed goal that, in some cases, runs counter to maximizing predictive accuracy, Vegas’ forecasting track record is quite strong. In addition to the aforementioned study in predicting division placement since 2005, Vegas’ over/under win totals nearly matched PECOTA for accuracy over the same span — an RMSE of 9.1 wins, or 2.7 wins away from the best possible result of 6.4.

This means a purely statistical system, a market-based method and a hybrid approach each came to the same level of accuracy from three different directions. As Salmons noted when I asked him about the effect of sabermetrics on the betting game, “The market has gotten a lot stronger than it used to be.”

But no matter how strong they get, projections will never reach that 6.4 RMSE mark. Even if we somehow knew beyond a shadow of a doubt that a team had precisely 81-win talent, there would still be a 5 percent chance it would finish with a win total as low as 70 — or one as high as 92 — by pure randomness alone.

Chance is always a confounding factor. Perfection is impossible, even as it beckons forecasters to try to and reach it.

Read the whole story
sillygwailo
1 day ago
reply
"The Imperfect Pursuit of a Perfect [x]"
Toronto, ON
Share this story
Delete

A Programmer’s Introduction to Mathematics

1 Comment and 3 Shares

For the last four years I’ve been working on a book for programmers who want to learn mathematics. It’s finally done, and you can buy it today.

The website for the book is pimbook.org, which has purchase links—paperback and ebook—and a preview of the first pages. You can see more snippets later in the book on the Amazon listing’s “Look Inside” feature.

If you’re a programmer who wants to learn math, this book is written specifically for you! Why? Because programming and math are naturally complementary, and programmers have a leg up in learning math. Many of the underlying modes of thought in mathematics are present in programming, or are otherwise easy to explain by analogies and contrasts to familiar concepts in software. I leverage that in the book so that you can internalize the insights quickly, and appreciate the nuance more deeply than most books can allow. This book is a bridge from the world of programming to the world of math from the mathematician’s perspective. As far as I know, no other book provides this.

Programs make math more interesting and applicable than otherwise. Typical math writers often hold computation and algorithms at a healthy distance. Not us. We embrace computation as a prize and a principle worth fighting for. Each chapter of the book culminates in an exciting program that applies the mathematical insights from the chapter to an interesting application. The applications include cryptographic schemes, machine learning, drawing hyperbolic tessellations, and a Nobel-prize winning algorithm from economics.

The exercises of the book also push you beyond the book itself. There’s so much math out there that you can’t learn it from a single book. Perspectives and elaborations are spread throughout books, papers, blog posts, wikis, lecture notes, math magazines, and your own scratch paper. This book will prepare you to read a variety of sources by introducing you to the standard language of math, and also push you to engage with those resources.

Finally, this book includes a healthy dose of culture. Quotes and passages from the writings of famous mathematicians, contextual explanations of cultural attitudes, and a light dose of history will provide a peek into why mathematics is the way it is today, and why at times it can seem so confounding to an outsider. Through all this, I will show what progress means for math, what attitudes and patterns will help you along the way, and how to stay sane.

Of course, I couldn’t have written the book without the encouragement and support of you, my readers. Thank you for reading, commenting, and supporting me all these years.

Order the book today! I can’t wait to hear what you think 🙂



Read the whole story
samuel
29 days ago
reply
Starting reading this book and it's excellent. Worth the $20 if you're looking to pick up math and proofs. Covers polynomials, calculus, linear algebra, and sets/groups.
The Haight in San Francisco
sillygwailo
2 days ago
reply
Toronto, ON
Share this story
Delete

Icelandic name phenomena

1 Share

It is well-known that Icelandic surnames are special and unique in the world. The so-called patronyms are still in use here. Patronyms are “surnames” generated by the first name of your father adding an ending whether you are female (-dóttir for daughter) or male (-son), basically saying whose daughter or son you are. My name for example (if I would be Icelandic) would be Marie Údosdóttir, because my dad is called Udo. But I learned the other day that there are also other Icelandic name phenomena.

Icelandic nicknames (gælunöfn)

While I was chatting with my colleagues at the jólahlaðborð (Christmas buffet) the other day, which is a traditional way for companies celebrating the holiday season with loads of traditional Icelandic food, I found out that there are several ways of a first name to transform into a nickname. Of course, nicknames are generic and therefore you can not typically say “This first name ALWAYS transforms into this nickname”. It’s simply in the nature of nicknames and they come in as many variations as people and characters, I guess. Apparently I am by far not the only one who ever wondered about Icelandic nicknames – turns out there has been even a doctoral dissertation written on Icelandic nicknames (University of California by Kendra Wilson).

It seems they are most often sound similar to the formal name or its ground structure is still shining through. “What is also apparent in the formation of nicknames is that they are usually of a certain form, it is as if there was a certain ‘template’ that nicknames should fit into”, says Aðalsteinn Hákonarson from the Arnastofun. He goes on saying that they are usually dissyllabic even though the actual name may only have one syllable. For Example: Jón turns into Nonni or Björn into Bjössi.

Since Icelandic doesn’t really have any form of diminutive for personal names, like for example German (for example Lenchen) or Polish, nicknames or pet names are not based on any of that. “However there are some nicknames formed with the element –si, -sa which one can maybe consider a kind of diminutive suffix, e.g. Jónsi for Jón or Grímsi for Steingrímur”, says Aðalsteinn.

“A particularly interesting phenomenon is that nicknames seem often to be derived from the speech of small children”, Aðalsteinn Hákonarson

Icelandic names can be very long and truly difficult to pronounce – not only for foreigners, but presumbly also for kids. I think of names like Þjóðbjörn, Sigþrúður or Guðmundur. This Aðalsteinn connects also with the phenomenon that nicknames often sound like they were formed by child’s mouth: “A particularly interesting phenomenon is that nicknames seem often to be derived from speech of small children.

But nicknames aren’t the only name phenonemon I learned of recently and it is connected with surnames.

Icelandic surnames (ættarnöfn)

Icelandic family names
Source: arnastofnun.is

As I just mentioned patronyms are still in use in Iceland and are for most part of the population the applying surname system. What I didn’t know until recently was that there exist also Icelandic family last names. They usually appear to be Scandinavian, at least to my ears, which is why I haven’t figured out yet that they are meant to be Icelandic. There are some few in existence and how they came into being is rather amusing. Icelandic family names came into fashion rather late, probably in the 19th century. “The first surnames were adopted by Icelanders who were studying in Europe”, explains Kendra Wilson.

Back in those days everything foreign was en vogue, especially everything Danish due to the Danish occupation and influence. Therefore some rich families simply decided to give themselves Europeans sounding family names. “In the 19th century it became fashionable for the upwardly mobile to adopt surnames, often ‘Danishized’ versions of Icelandic place or personal names”, states Wilson. That is why some of them are retrieved from original place names, for example their home farm or place of activity. “Briem” for example originates from Brjánslækur, the place where the ferry from Stykkishólmur arrives in the wonderful Westfjords. “Hjaltalín” traces back from the farm Hjaltadalur in Skagafjörður, as well as “Blöndal” which comes from a family from Blöndudalshólar. However, family names were forbidden by law in 1925 and now the patronymic system is the only way to receive your “surname”.

Here a short list of Icelandic surnames:

  • Blöndal
  • Waage
  • Briem
  • Norðdahl
  • Hjaltalín
  • Hjaltested
  • Vídalín




Read the whole story
sillygwailo
12 days ago
reply
Toronto, ON
Share this story
Delete

Tanya Talaga: First let’s talk about basic Indigenous rights, then we’ll get to reconciliation

1 Share

As Ottawa continues to fall short of upholding Indigenous rights, the emptiness of its promises of reconciliation are laid bare, Tanya Talaga writes.
Read the whole story
sillygwailo
13 days ago
reply
Toronto, ON
Share this story
Delete

Homeland security: facial recognition in machines and crows

1 Share


Last October, I found myself at the Atlanta airport on the way out of the country for a conference. A Delta employee at the gate announced with surprising nonchalance that today they would be using facial recognition for the boarding process; she motioned toward an eye-level, oddly rounded screen atop of a gray plastic base. It looked more like something from a doctor’s office than an episode of Black Mirror. Next to it, a Delta sign read “ONE LOOK AND YOU’RE ON,” which, like the airline employee’s cheerful tone, framed this experience as a newfangled convenience.

Though no one protested, I sensed a familiar feeling in the air: that of a tired assent to some new, confusing, and vaguely dystopian terms. We lined up dutifully, following the instructions: “Please stand on the circle and look at the camera.” As my picture was being taken, I wondered what it was checking it against. Oh right, my passport photo is part of a Department of Homeland Security database. Then I became worried. In my passport photo, my hair is unusually big, I’m not wearing glasses, and personally, I think my eyes look weird. I wonder if — BEEP. A friendly checkmark appeared. Deep in the guts of some system, some measurements had matched some other measurements, and I had been deemed satisfactorily me.

It wasn’t my only time being recognized by a nonhuman entity. Starting in 2017, it was possible for Facebook to find me in photos even when no one had tagged me in them. There was the time I worked on a project that required me to enter a data center and thus have my eyes scanned to which the machine responded flatly, “Your identity… has been recorded.” But starting in 2016, my identity has also been recorded by something, or someone else: crows.

That was the year I read about John Marzluff, the go-to researcher for all things crow-related. He’d noticed that crows on the University of Washington campus would hold grudges against humans who had caught them for studies, so he had his researchers go about their business but with novelty Dick Cheney masks on. Sure enough, the crows thereafter mobbed anyone wearing the Cheney mask. Not only did it work if even the mask was upside down, but the crows appeared to transmit this information, so that the Cheneys acquired more and more haters. This made me wonder (with some delight) whether Marzluff’s plan was actually to make sure that crows would attack the real Dick Cheney if he ever dared to visit the University of Washington. But it also made me wonder whether I could get the neighborhood crows to recognize my face.

how to befriend a crow

My balcony, being one of many on the dark side of an apartment building, was not an intuitively interesting place for the neighborhood crows. It took a few months of leaving peanuts on the railing to get their attention, but eventually a family of three or four crows became regulars, arriving around the same time each day and forming a neat line on the telephone lines closest to the balcony.

crow-veillance

Once this mutual habit of ours had formed, I would watch the crows and their peculiar habits yawning, scooting up to one another hoping to get groomed, or pecking at a grotesque complex of mushrooms in the neighbor’s front yard. But I also hoped that I was being watched. Sitting on the balcony felt like hitting “record” on a mysterious device, since every minute that they peered over at me, I entreated them silently: Record this face!

My experiment soon yielded results. Sometimes if the crows were still on the wires when I left for the day, they’d spot me at the building entrance and follow me around the corner, gliding down the street and landing on telephone lines right over my head. Or they might land on the sidewalk and waddle behind me for an entire block. One time they followed me to the nearby bus stop and watched, perhaps puzzled, as I got onto a bus. These days, I’ll be walking near my apartment and they’ll glide down suddenly out of nowhere, landing right in front of me. Whenever I return from a long trip I wait on the balcony for them; they’ll swoop by especially close to my face, perhaps in greeting or to check that it’s really me.

my sometime-entourage

Animals and machines obviously use different mechanisms for recognizing faces. When my face was captured at the Atlanta airport, “recognition” had to do with a pattern of points around my eyes, nose, and mouth. In general, this means that face detection and recognition make certain demands of the image. Apple’s Face ID, for example, doesn’t work if your eyes are closed, and cameras that scan a crowd for faces have difficulty with faces tilted away.

By contrast, the crows have recognized me on the street when I’ve worn giant sunglasses or have bangs in my eyes, and even during the Camp Fire in November, when I wore an N95 mask that covered my mouth and nose. The crows’ image of me is rarely head-on, since they often spot me on the ground from the top of a tree, or on my balcony as they’re strolling along the sidewalk. I don’t know how they do this. The closest thing I could find to an answer was a 2017 study on macaques’ ability to recognize human faces, in which researchers at the California Institute of Technology identified groups of specialized “face cells” that together could encode a full fifty different aspects of a face. The aspects were things like distance between the eyes, face shape, and skin texture. I wonder whether the crows have memorized so many aspects of my face that even if they’re missing a few, they still know it’s me.

“are you home?”

Technical aspects aside, the biggest difference between facial recognition in machines and in crows has to do with the ends of recognition, and thus how it feels to be recognized. Back at the Atlanta airport, I’d found the Delta employee’s upbeat tone to be farcical given the inherent creepiness of facial recognition and how readily it’s weaponized. In 2016, a new Russian app called FindFace gave users the ability to take a photo of someone and find them on a popular Russian social network with 70 percent reliability something a Guardian headline speculated “may bring an end to public anonymity.” In 2017, the Dubai International Airport unveiled plans for a tunnel-shaped “virtual aquarium” whose virtual fish would encourage passengers to look around while eighty different cameras scanned their faces and irises. In 2018, the Chinese police used facial recognition to pinpoint a man amid a crowd of sixty thousand people at a concert.

But just as terrifying as a world without public anonymity is the idea of a world where we are not recognized at all. Anonymity, distance, and abstraction are hallmarks of contemporary existence, the cold night in which clusters of friends, family, lovers, acquaintances, pets, and not-quite-pets glow ever more vitally. To be recognized by a friend instantly, even after years apart is not just a wonder of neuroscience. It is evidence of a bond, a sometimes miraculous moment in which we are plucked out from the crowd and rendered more than a stranger. The airport is precisely the type of place where I find it most comforting to think back to my crows, and to that “database” in which they have encoded features of my face so specific and delicate that perhaps even I have not noticed them.

a familial fellow

In one of his lectures, Alan Watts said that the body doesn’t contain the soul, but rather the soul contains the body, since the soul for him was “the entire complex of relationships in whose context this organism exists.” To think of people and animals who recognize my face is a reminder that my image exists outside of me and dispersed in the minds of others. I like to think about the difference between this kind of image and a cold series of measurements. The Department of Homeland Security keeps a copy of my face, but it is the frozen face of a stranger, “recognized” by people and machines I will never see and am nonetheless asked to trust. By contrast, the version of my face that lives in the crows is born out of years of mutual attention attention that binds me to them, to my memory, and to my neighborhood. In that sense, the crows offer a very different kind of “homeland security.”

Read the whole story
sillygwailo
13 days ago
reply
Toronto, ON
Share this story
Delete
Next Page of Stories