What We Should Worry About When We Worry About Virtual Reality

My friend Cathy O’Neil just sent me an article she wrote for the NY Times reviewing two books by technologists about virtual reality (VR). Part of her take was that neither book talked enough about ways that VR could be abused, and she speculated that worrying about VR was still mostly the provenance of science fiction writers (think Star Trek) rather than technologists.

I’m pretty comfortable around both sci-fi and technology, but you really don’t need to be an expert in either to worry about how VR could upend our lives and civilization. Just some sense of recent history is enough. If you think that massive computational power, the internet, and smartphones might have turned out to be a bit more than we bargained for, maybe it’s time to consider how amazingly well-positioned VR is to amplify some of the most troublesome aspects of the technology revolution:

Personalization. We’ve learned over the last quarter century that we don’t mind being monitored (cookies, GPS, Fitbits), just as long as some benefits (recommendations, special offers, traffic advice, a tailored Facebook feed, the ability to broadcast our 5.4 mile running route to all our friends) come from crunching the resulting data. Never mind who might be storing all that data or what they might be doing with it.

Now think about VR, which massively scales up both the amount of data and the ability to collect it. On one hand, VR is an immersive experience, generated by high dimensional data sets (indeed, one of the uses of VR is as a tool to allow us to navigate data sets that are otherwise too complex to make sense of; see here or here or here). On the other, VR is delivered through a device, which can be used to track eye movements, and VR technology to monitor other biometrics like heart rate, pulse, and  electrical activity in the brain is already on the way (see here or here). You’ve probably heard of Google’s A/B tests, which enable web designers to vary individual aspects of a web page and track how people respond. Now imagine such tests in VR space, targeted at each individual user, and able both to vary all kinds of stimuli affecting all the senses, and to measure all kinds of response. In a contest between your family, friends, and VR set over who knows you better, it’s hard to see the humans having a chance.

Addictiveness. By now it’s sort of a cliche to hear a technologist speak thoughtfully about how they won’t let their children near smartphones or Instagram until they’re in high school, or to read articles about internet use sprinkled with multiple mentions of dopamine. Won’t this all seem quaint in a few years, when internet porn gives way to (personalized!) VR sex, and your social network can deliver a full VR simulation of your crush’s reaction to the cute photo you just posted, not just a stylized thumbs-up or heart. Um, yeah, VR is going to make the virtual world way more addictive. “Why go into the outside world at all, it’s such a fright,” as Black Flag sang, to their televisions, and that was at least two whole generations of technology ago!

Marketing. I was born in the Soviet Union, which had no ads, and it always felt strange to me that our entire media landscape (or, today, our entire information landscape) was driven by companies inserting little messages meant to sell you things. For one thing, I was always a bit skeptical that advertising was actually worth it. Well, with VR, there’ll be no question, because we’ll be able to track the outcomes of ads so precisely: eyeballs widen, heart rate rises just a bit, electrical activity heightens in the buying center of the brain (which by this time we will have effectively mapped, using — what else — VR technology). Advertisers will know exactly which ads worked (so the economy will make sense!), and, with predictive analytics and the heavy volumes of data attached to VR, they’ll also know which ads will work, for any given person. And lots of them will, because VR’s ability to virtually sample any product you might imagine might make it the most effective advertising medium ever. If today we think about ads as delivering eyeballs and clicks, in the age of VR, they might be delivering (virtual) wallets directly.

Will users object? One more thing we’ve learned in the internet age is that people don’t seem to mind being targeted with ads across their entire virtual experience. Ad-based media still dominate, while raising revenues via direct subscription works for a few niche publications at best. The internet is funded by advertising. Why wouldn’t VR be?

Though VR seems expensive today — the domain of rich NFL teams needing to train quarterbacks to have split second reactions to thousands of different stimuli, as Cathy writes in her book review — from another point of view, it might actually be quite cheap. In the non-virtual world, you have to be rich to sit in the front row at midfield at the Super Bowl, or swim with tortoises in the Galapagos Islands, or climb Mount Everest. But, mass adoption of VR could be a great leveler in a way, making virtual versions of all of these accessible to the masses. Being marketed to may seem like a small price to pay to have these experiences, especially if the income gap between rich and poor grows as technology makes more and more segments of the economy winner-take-all. Sure, VR might enable advertisers to fully exploit you economically, to optimize and control all of your purchasing power — but so what, we haven’t been troubled yet whenever our technology asks us to give up control to gain comfort.

The scariest thing about VR might be that it could be more of the same, but on steroids. If we’ve shown no societal ability so far to confront technology addiction, data collection and surveillance, or media manipulation, what happens when VR technology renders all of these ten times more powerful? Be afraid, be very afraid.

Advertisements

Math and Magic

The most inspiring math teacher I had in college was Persi Diaconis, who, before becoming a Harvard math professor, was… a card magician. This made him legendary around the math department, because his path into higher math (he started to learn calculus because it would help him invent even more awesome card tricks) was not exactly the most traditional way of getting into the subject. By the time I met him, Persi was already a leading mathematician and statistician (he’s the guy who proved that you need seven riffle shuffles to randomize a fifty two-card deck, among other things), but he still kept his interest in magic. And while I only got a rough idea of what math-driven magic tricks were like when I studied with him, I finally got a better sense a few years ago, when he published an absolutely, well, magical book devoted to the subject. It’s called Magical Mathematics.

I read the book soon after it came out, and showed my kids a few of the tricks I learned from it. They didn’t get the math, but they liked the tricks. And that was about it, until sometime last year, when my older son started learning card tricks too, via YouTube videos. For a while he was mostly doing traditional sleight-of-hand stuff, but this weekend he showed me a trick that’s very mathematical! It’s simpler than most of the tricks in Persi’s book, but very much in the same spirit. You should see it too. Here goes:

We start with a deck of cards, cut as many times as you like. I hand you the deck, asking you to cut it one last time. Then I show you the top and bottom card in the deck. I’m not going to know what they are, so you need to remember them. Follow along:

We’ll put the two cards back in the deck, say on top, and I ask you to cut the deck again, multiple times if you like, so we lose your cards in the deck:

The trick is to find them again. First, we deal the cards out into four piles:

Next, we combine these piles into two by joining together alternating piles (first and third into one single pile, second and fourth into another). Finally, we flip over one of the piles and riffle shuffle the two piles together. (My son’s been doing all the steps to this point, but I’m going to take over for this step because he hasn’t learned to riffle shuffle yet:)

Now let’s spread out our cards:

Notice that all the cards facing up have the same color… except one. That’s one of your cards!

Flip the deck over. Again, all the cards facing up have the same (other) color. There’s one exception: your other card!

How did we do that?

Unlike magic tricks based in sleight of hand, we didn’t hide anything: what you see is what you get. Except for one thing: you probably didn’t know that the original deck we started with looked like this:

The deck was set up to alternate red and black cards: one red, one black, one red, one black… Knowing that, stop for a moment and try to step through each step of the trick. Can you see how it works? If you can, congratulations! If not, let’s walk through it together:

We started by cutting the deck multiple times. That’s meant to make the audience think we’re making things random, but in fact it just cycles the deck around, and leaves the basic alternating red-black structure in place.

Now the key step: we took off the top and bottom cards, showed them to the audience, and put them back on top — almost, but not exactly, how we found them. That almost is the key to the trick. The point is that those two cards are now in opposite order to the rest of the deck. For example, say the top card was red. At that point, the deck must have been ordered as red (top), black, red, black, and so on, with black (the other card we picked up) on the bottom. After we take off the top and bottom card, the remainder of the deck is ordered as black (left on top), red, black, and so on, with red now on the bottom. And when we put our two cards on top, the order becomes red, black, black, red, black, red, black, red, and so on, with red still at the end. What’s special about our two cards is that they are out of phase with the others. And wherever they happen to go in the deck now, after we cut the cards, that’s how we’re going to find them again.

The way we find them by separating the reds from the blacks, which is what dealing the cards into piles was designed to do. For example, say that after we cut the cards, red was on top. Follow the cards into piles: red (1st pile), black (2nd pile), red (3rd pile), black (4th pile), red (back to the 1st pile), black (2nd pile), and so on. Each pile has all the same color cards — except the two out of phase cards! So piles 1 and 3 will be all red cards, with one exception, which is one of your cards. And piles 2 and 4 will be all black cards, also with one exception, which is your other card. Here’s how it looks under the hood:

Q (you might be asking): Why four piles? Since the colors alternate, wouldn’t two piles be enough?

A: Good math observation! Only it’s harder to keep track of what you’re doing with just two piles (I tried it). Since just one misdeal messes up the color separation, using four piles makes the trick more secure.

Now of course at this point we could just pick up the red pile and the black pile separately, find the off color card in each pile, and be done with things. But… that isn’t very theatrical, is it? So instead we flip one of the piles over, and shuffle them together. Which has exactly the same effect, but looks way cooler!

Again, this isn’t in Persi’s book, but it’s a good introduction to math-based tricks. If you like it, or know some kids who might, I’d very much encourage you to check Magical Mathematics! You can see a couple sample chapters online at the Princeton University Press site:

Chapter 1

Chapter 5

And if you know more tricks in this spirit, my son and I would love to see them!

 

How Value Added Models are Like Turds

“Why am I surrounded by statistical illiterates?” — Roger Mexico in Gravity’s Rainbow

Oops, they did it again. This weekend, the New York Times put out this profile of William Sanders, the originator of evaluating teachers using value-added models based on student standardized test results. It is statistically illiterate, uses math to mislead and intimidate, and is utterly infuriating.

Here’s the worst part:

When he began calculating value-added scores en masse, he immediately saw that the ratings fell into a “normal” distribution, or bell curve. A small number of teachers had unusually bad results, a small number had unusually good results, and most were somewhere in the middle.

And later:

Up until his death, Mr. Sanders never tired of pointing out that none of the critiques refuted the central insight of the value-added bell curve: Some teachers are much better than others, for reasons that conventional measures can’t explain.

The implication here is that value added models have scientific credibility because they look like math — they give you a bell curve, you know. That sounds sort of impressive until you remember that the bell curve is also the world’s most common model of random noise. Which is what value added models happen to be.

Just to replace the Times’s name dropping with some actual math, bell curves are ubiquitous because of the Central Limit Theorem, which says that any variable that depends on many similar-looking but independent factors looks like a bell curve, no matter what the unrelated factors are. For example, the number of heads you get in 100 coin flips. Each single flip is binary, but when you flip a coin over and over, one flip doesn’t affect the next, and out comes a bell curve. Or how about height? It depends on lots of factors: heredity, diet, environment, and so on, and you get a bell curve again. The central limit theorem is wonderful because it helps explain the world: it tells you why you see bell curves everywhere. It also tells you that random fluctuations that don’t mean anything tend to look like bell curves too.

So, just to take another example, if I decided to rate teachers by the size of the turds that come out of their ass, I could wave around a lovely bell-shaped distribution of teacher ratings, sit back, and wait for the Times article about how statistically insightful this is. Because back in the bad old days, we didn’t know how to distinguish between good and bad teachers, but the Turd Size Model™ produces a shiny, mathy-looking distribution — so it must be correct! — and shows us that teacher quality varies for reasons that conventional measures can’t explain.

Or maybe we should just rate news articles based on turd size, so this one could get a Pulitzer.

 

Why the Prado is the Best

I am in my mid-40’s and have been to a lot of art museums, so I didn’t really expect to walk into another one and think, “Oh my god, this is head and shoulders above any place I’ve been to” (the Met, the Louvre, the Uffizi, the Barnes, etc.). But then I had never been to Madrid and the Prado until this week. Here are three reasons why it really is the best:

1. The great stuff is so, so great. Within minutes of walking in, I had wandered into someplace called room 49, which might be the best single collection of Italian Renaissance art you’ll ever see in a single room. With a bunch of glorious large and mid-sized Raphaels, like this one:

img_3047

I mean, can you imagine seeing something like this just a couple minutes after you walk in? When I came across it, I didn’t know yet that they don’t let you take photos, so I snapped this one before the guards gently told me the rules.

And oh yeah, the Prado has a really interesting copy of the Mona Lisa too.

2. It’s not gigantic, but the quality is very high throughout. There’s something astonishing, often many things, in practically every room. Some highlights: likely the deepest Goya collection anywhere, from all across his career, including a bunch of large, airy, light-colored early paintings called tapestry cartoons up on the top floor. A magnificent set of El Grecos. And don’t get me started on Velazquez, Zurbaran, Ribera, Bosch (again, the deepest set of his works I’ve seen in one place), Fra Angelico’s Annunication, and Tintoretto. The main hall is anchored by a fantastic collection of Titian and Rubens, which… is absolutely great, but many of the other rooms are even better.

3. The way it just teems with life. So check out this graceful bit of Asian calligraphy, from a scroll I found in an out-of-the way, empty room, with no guards to tell me not to take pictures:

img_3058

I mean, those hands!

OK, the reason why you haven’t heard about the Asian calligraphy scrolls in the Prado is that there aren’t any. Our angel with the graceful hands is actually part of a Spanish altarpiece, circa 1200:

img_3057

Or maybe it didn’t look like an Asian scroll to you. But the magic of the Prado is that when you see so many amazing pictures together in one place, they amplify and animate each other, they transform each other, they dance with each other and with you, and before you know it, all the distinctions you’ve ever learned between different genres and periods and schools melt away. Then only the art is left, direct, pure, and alive, and it fills you with life, and keeps dancing with you long after you leave the building. The Prado is the most joyful art museum in the world.

Springtime for Donald*

What a beautiful spring day in the Northeast yesterday! The kind of day that makes you, right now, wherever you are, city folks or out in the country, snuggled in quilts or riding the bus, just turn to the nearest Real American around you, even your own reflection in the mirror and . . . just . . . sing:

It’s springtime for Donald and Vladimir!
Winter for NATO and Ukraine
Putin’s Manchurian Candidate
Is helping Make Russia Great Again!

Oh, it’s springtime for Donald and Vladimir,
Winter for Syrian refugees
Sorry if your doctor’s Iranian,
And judges belong on their knees

Yes, it’s springtime for Donald on Twitter
Please get your facts from Kellyanne
Congress had better just smile and nod
When Spicer says it’s not a ban

It’s springtime for Donald and the alt-right!
The Times and the Post are fake news
Read Breitbart so you’ll be the first to know
When Bannon puts a ban on the Jews!

Now ev’rybody –

*Apologies to Thomas Pynchon (from whom I stole the intro) and Mel Brooks (who gave us this):

Music For Marchers

1. The Opportunity

In mid-September 2001, about a week after terrorists destroyed the World Trade Center towers, Laurie Anderson played a concert at Town Hall in New York City. Here is what she said when she began:

We want to dedicate our music tonight

to the great opportunity that we all have

to begin to truly understand

the events of the past few days,

and to act upon them,

with courage and with compassion,

as we make our plans

to live in a completely new world.

Opportunities we didn’t ask for, opportunities that we fear, are still opportunities. We missed the one we faced back then, but let’s recognize the one we face as a country today — and act upon it.

2. Let’s Renew Ourselves Now

Leonard Cohen, improvising in front of perhaps 600,000 people at the Isle of Wight festival in 1970:

I know it’s been cold, and I know it’s been damp.

I know you’ve been sitting all night long.

But let’s renew ourselves now, let’s renew ourselves now, let’s renew ourselves now.

3. Marchons, Marchons

I don’t care how many times you’ve watched this movie or this scene, it’s impossible to see it without feeling something. So watch it again. And march on, march on.

Then They Came for my Job — and No One was Left to Speak for Me

Everybody loves Martin Niemoller. Wikipedia will tell you he is famous for his “provocative poem about the cowardice of German intellectuals following the Nazis’ rise to power and subsequent purging of their chosen targets, group after group.” Here’s the original form of that poem, in case it hasn’t come across your Facebook feed lately:

First they came for the Socialists, and I did not speak out —
Because I was not a Socialist.

Then they came for the Trade Unionists and I did not speak out —
Because I was not a Trade Unionist.

Then they came for the Jews, and I did not speak out —
Because I was not a Jew.

Then they came for me — and there was no one left to speak for me.

It’s a lovely piece to quote: it reckons with the past, signals enlightenment and moral clarity, and challenges us all to live up to its ideals. You pretty much can’t help feeling a little more ethical and upright when you read it. Still, before we get too complacent, I’d like to ask you to accompany me, and Niemoller, on a short tour of recent American economic and political history. I want to see if we’ve really learned to speak up for others as well as we think we have. Humor me, you don’t really mind, do you?

First they came for the Socialists — this feels like a quaint anachronism now, but the Socialist party was highly relevant politically in the United States in the early 1900’s. The Socialists had two representatives in Congress and high water marks of 6% of the popular vote in the 1912 presidential election for socialist Eugene Debs and almost 17% in 1924 for socialist-supported progressive Robert LaFollette. (By contrast, Gary Johnson won 3% and Jill Stein won 1% of the popular vote in 2016.) Within 25 years, discredited by American politics and world events, the Socialists were more or less irrelevant in the U.S., winning less than 0.1% of the presidential vote from the 1950’s on. Some people did speak out, but the tide was against them. You and I weren’t around then, in any case.

Then they came for the Trade Unionists —  much of this happened in our lifetimes. Labor unions were highly influential in the United States for most of the 20th century, especially during what’s known as the Great Compression, the period following the New Deal reforms when income inequality declined dramatically. From Wikipedia:

This “middle class society” of relatively low level of inequality remained fairly steady for about three decades ending in early 1970s, the product of relatively high wages for the US working class and political support for income leveling government policies.

This is clear in this chart that tracks income inequality over time. This income leveling correlates well to labor union membership, which grew steadily during the New Deal, and eventually reached almost 35% — more than 1 in 3! — of salaried workers in 1954. Union membership gradually declined from that point. The best statistics start in 1983, when around 20% of the work force still belonged to unions, and they show another 50% decline to just 11% of the work force in 2015. For my generation (I am in my mid-40’s), and for the so-called new economy, I think we can say that we’ve played out Niemoller’s script pretty much as written: few of us were unionists, and few of us have paid attention as union influence declined throughout our working lives. (Uber rides are cheap and convenient, so who cares if the drivers have no rights and the company can raise its take from fares at will? I bet a hundred years ago there would have been a strike.)

Then they came for the Industrial Cities. Here I am talking about both big (Detroit, MI, population 1.85M in 1950) and small cities (Youngstown, OH, population almost 170K in 1960). Since then, both cities have shrunk by about 60%! The history of these cities (and many more like them) in the 20th century is a complex mix of economics, sociology, and racism, but here’s the world’s shortest introduction:

U.S. cities and industry in the North and Midwest grew rapidly in the late 1800’s and early 1900’s, powered by a huge wave of  European immigrants. Then World War I created a labor shortage, and blacks from the South migrated north to fill the gap. This began the Great Migration of blacks to cities in the North and (eventually) the West. Apart from a short break during the Great Depression, the migration continued for the next 40-50 years, transforming the black population from 80% rural-20% urban to the exact opposite — 80% urban-20% rural — in little more than half a century.

As they came to the white-dominated cities, incoming blacks were steered to neighborhoods with lower economic opportunity and investment, away from whites, a morally bankrupt process known as redlining. (For a history of redlining that goes beyond Wikipedia, read Ta-Nehisi Coates’s mighty essay, The Case for Reparations.) Meanwhile, whites tended to concentrate in their own segregated parts of the same cities, and eventually to move away from the cities altogether as road networks improved (white flight). Some of the industrial jobs that brought blacks to the cities went to the suburbs with the white population, some began to go overseas, and some were lost to automation, with the latter trend showing no signs of abating. In just 20 years (1967-1987), Philadelphia, New York City, Detroit, and Chicago all lost over 50% of their manufacturing jobs. Some cities managed to reorient around the technology or financial sectors (Boston, New York), while others lost significant population and income that they’ve never regained. (For more on the relationship between lost industrial jobs and inner city poverty, you can read William Julius Wilson; here is a short summary paper.)

Who spoke out for these cities, for their minority populations and for their financially starved schools? Not those whites who moved to the suburbs to get away from blacks, and not the mainstream Democratic party of the 1980’s and 90’s, which decided it needed to distance itself from inner city concerns in order to win elections. Maybe you did?

Then they came for the Heartland. Or perhaps for the U.S. manufacturing economy, depending on whether you prefer to view things geographically or economically. This has been the subject of much debate since the election, because Trump’s margin of victory came from three “Rust Belt” states (PA, MI, WI) that he had been expected to lose, and because it’s believed that his margin of victory in those states came from dissatisfied white working class voters either seeking change or lashing out either at bullying elites or at bullied minorities or immigrants. (This is probably a lousy explanation of voter dynamics in Pennsylvania.) Frustratingly, these discussions have devolved into arguing over whether these voters are motivated by economic or social-cultural factors (as though it were easy to separate the two) and whether they deserve sympathy (as though we’ve never seen anyone bully others while being bullied at the same time).

No matter how you view the politics here, we should be able to come to some kind of agreement on the economic facts. Here’s a recent article suggesting that the Rust Belt is not a struggling region, because other parts of the country are worse off: the states we are talking about are all in the middle third of U.S. states by median income. Sure, they’re not Connecticut or California, but they’re not Arkansas or Mississippi either. This is true enough as far as it goes, but it fails to consider decline. As a quick back-of-the-envelope exercise, I pulled up the income data and sorted the states top to bottom by median income as of 2015 and as of 1995. Here’s what I found:

  • The state with the biggest decline in ranking over the last 20 years (from #5 to #28)? Wisconsin.
  • The state with the second biggest decline (from #14 to #31)? Michigan.
  • Five of the ten states with the biggest drop in ranking form a contiguous region at the heart of the Rust Belt: Wisconsin, Michigan, Illinois, Indiana, and Ohio. (There are no other clusters; the other five states are completely non-contiguous.)

At this point, these Rust Belt jobs are more likely lost to automation than to other states or countries (though there is some of the latter) or incoming immigrants from lower-income countries (virtually none coming into the Midwest). So it’s hardly clear what to do. Still, there’s not much sympathy for these people’s plight on my Facebook feed, where most people (1) have college or graduate degrees and professional occupations, (2) live in desirable areas, and (3) are frustrated and terrified, as I am, to see Trump in power.

Then they came for — OK, who’s next? Are you? What job will you have in twenty years, or what job will your children have, when:

  • Economic opportunity appears to be declining, not increasing, across generations.
  • The technology sector, which is not very labor intensive, and also finds it easy to move jobs around or offshore, is coming to dominate the (non-service) economy. (Just to give you an idea, Facebook is the 7th most valuable company in the world and employs only 13,000 people, a fraction of the workforce of the industrial behemoths of 50 years ago, whose employees numbered in the hundreds of thousands.)
  • Automation is increasing, and potentially applies to more and more jobs, both in and out of the tech sector, as technology itself becomes ever more capable.
  • The gig economy is rising, and there doesn’t appear to be an obvious limit on what kind of jobs can be gig-ized. (Gig workers have even less leverage than long-term employees.)
  • The financial industry, which has grown rapidly over the last several decades, provides relatively little economic value relative to the human capital invested in it, and might be ripe for contraction. You could speculate that the same might be true of marketing as well (another industry where too many of the well-educated settle), or at least that it could be done more cheaply. If our news stories can be replaced by crap written on the cheap in Eastern Europe, why not the ads?

Shifting into my ominous Rod Serling voice: with the manufacturing economy gone, both in the cities and out, with more and more people shifted into low-wage service jobs, with labor as powerless as it seems it’s ever been, with our economy sliced up into strata, and the humans in each one of those strata cut away in turn — who will be left to speak for you when they come for you? And how will you feel with Niemoller’s warning, turned into prophecy now, running on a loop over and over, not just on your Facebook feed but inside your head?