Posts
-
My First First-Author Paper
Comments
Today, the paper Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping went public on arXiv. I’ve been working on this project for the past 6 months, and this is the first time I’ve been first author on a research paper, so overall it’s a Pretty Big Deal for me. I share first authorship with Konstantinos Bousmalis and Paul Wohlhart, both of whom were great to work with. The link above will take you to a landing page with the arXiv link and a video that briefly explains our work.
If it wasn’t clear from the lengthy author list, a lot of people contributed to this project. Thanks to everyone for the mentorship and the engineering support that made this project possible.
-
The Friendship Paradox And You
CommentsThe friendship paradox is a cute rule of thumb. Unlike other rules of thumb, it actually has some mathematical justification behind it.
The paradox states that on average, your friends have more friends than you do. At first glance, this may seem strange, because it can’t be true for everybody. Someone has to be more popular than everybody else. And that’s true - somebody has to be on top. That’s why the statement says on average. A small fraction of people are more popular than their friends, and a large fraction are less popular than their friends.
To justify why this could be true, let’s model friendship as an undirected graph. In this graph, people are vertices, and an edge connects two people if they’re friends with one another.
Now let’s introduce some notation. \(V\) is the set of all vertices, \(n\) is the number of vertices, \(v\) is a single vertex, and \(d_v\) is the degree of vertex \(v\). The average number of friends a person has is
\[\text{Average number of friends} = \frac{\sum_{v \in V} d_v}{n}\]Alright. But what’s the average number of friends that someone’s friends have? To count this, let’s imagine that every person \(v\) creates \(d_v\) lists, one for each of their friends. Every list is titled with that friend’s name. Say their friend is named \(u\), for sake of example. On that list, \(v\) writes down all of \(u\)’s friends.
The average number of friends that \(v\)’s friends have is the average length of the lists that \(v\) created, not counting the title. The average number of friends that someone’s friends have is the average length of all these lists.
Each \(v\) creates \(d_v\) lists, giving a total of \(\sum_{v \in V} d_v\) lists. There are \(d_v\) lists titled \(v\), because we get one such list whenever a friend of \(v\) creates lists. Each of those lists has \(d_v\) names on it. Thus, each person \(v\) contributes \(d_v^2\) to the total length.
Overall, this gives
\[\text{Average number of friends of friends} = \frac{\sum_{v \in V} d_v^2}{\sum_{v \in V} d_v}\]Now, apply the Cauchy-Schwarz inequality. This inequality states that for any two vectors \(a\) and \(b\), their dot product is at most the product of their norms. We’ll use the version where we square both sides.
\[\langle a, b \rangle^2 \le \|a\|^2\|b\|^2\]Let \(a\) be the vector of all ones, and \(b\) be the vector of degrees \(d_v\). Since there are \(n\) vertices, we get
\[\left(\sum_{v \in V} d_v\right)^2 \le n \sum_{v \in V} d_v^2\]which rearranges to
\[\frac{\sum_{v \in V} d_v}{n} \le \frac{\sum_{v \in V} d_v^2}{\sum_{v \in V} d_v}\]The left hand side is the average number of friends, and the right hand side is the average number of friends of friends. That concludes the proof. \(\blacksquare\)
At a high level, the friendship paradox happens because the popularity of popular people spreads through the network - they have lots of friends, each of whom sees that some of their friends are popular.
Importantly, this only says something about the average. Arguing anything more requires making assumptions about how people interact and how friendships work.
A natural extension is to ask whether a similar result holds in directed graphs. A lot of relationships aren’t symmetric, so if a similar result holds, it makes the principle more applicable.
It turns out such a result does exist. Let’s model a directed edge as a producer-consumer relationship. There’s an edge from \(u\) to \(v\) if \(u\) produces something that \(v\) consumes.
People both produce things and consume things, represented by out-edges and in-edges respectively. Let \(d_{v,out}\) and \(d_{v,in}\) be the number of out-edges and in-edges for \(v\).
Let’s consider the average number of outgoing edges. This is the average number of things that people produce.
\[\frac{\sum_{v \in V} d_{v,out}}{n}\]For a given \(v\), let’s compare this to the number of things produced by content producers that \(v\) follows. For each incoming edge, create a list for the source of that edge, writing down every consumer of that source. (This list will be the endpoint of every outgoing edge from that source.)
From here we can apply similar logic. Each person creates one list for each outgoing edge, giving \(\sum_v d_{v,out}\) lists total. Each \(v\) is the title of \(d_{v,out}\) lists, one for each consumer. Each of those lists will have \(d_{v,out}\) items on it. All together, the average across all \(v\) is
\[\frac{\sum_{v \in V} d_{v,out}^2}{\sum_{v \in V} d_{v,out}}\]which we can once again apply Cauchy-Schwarz too. The conclusion?
On average, the content producers you follow make more things than you do.
I call this the producer view, because you’re always counting the edges that leave each vertex. We can also take the consumer view, counting the edges that are entering each vertex instead. By performing a similar argument, you get this conclusion instead.
On average, the people who follow your work follow more things than you do.
Both views are valid, and give different interpretations of the same graph.
Again, this argument only says something about the average, and you need assumptions about graph connectivity to argue anything stronger. In fact, despite its mathematical underpinnings, I would hesitate on treating the friendship paradox as a truth about the world. I see it more like a principle, that’s useful for flavoring different arguments, but not strong enough to hold an argument on its own.
* * *
In the derivation above, the only requirement was that we could model interactions as a graph.
There’s a branch of mathematics called category theory. I don’t know it very well, but the impression I get is that you let objects represent something, you let arrows represent some relation between objects, you draw arrows between different objects, and then you interpret all of mathematics as special cases of those objects and arrows. This lets you do things like explain finance by drawing a bunch of arrows.
For some reason I know \(\epsilon > 0\) fans of category theory are going to read this post, so as an homage to them, let’s make a bunch of wild claims about society by generating different interpretations of vertices and edges.
Let vertices be Twitter accounts. An edge connects \(u\) to \(v\) if \(u\) follows \(v\). In the producer view, on average the accounts you follow have more followers than you. In the consumer view, the people who follow you are more likely to follow more people than you do.
Let vertices be people. Instead of friendship, say there’s an edge from \(u\) to \(v\) if \(u\) has a crush on \(v\). To the disappointment of many people, crushes aren’t symmetric. In the producer view, on average the people you crush on have more admirers than you do. In the consumer view, on average people who have crushes on you have crushes on more people than you do. I don’t know if this makes anyone feel better about their love life, but there you go?
Again, let vertices be people. This time, there is an edge from \(u\) to \(v\) if \(u\) writes something that \(v\) reads. In the producer view, on average your readership is smaller than the readerships of writers you follow. In the consumer view, on average your readership reads more things than you do. Now, not everybody writes, but we could substitue writing with any form of communication. Blogs, articles, Facebook posts, speeches, Youtube videos, research papers, memes…
In general, any prolific person not only makes lots of things, they become well-known for making lots of things. Their reputation both precedes them and outgrows them.
* * *
I like the friendship paradox a lot. Why?
Well, for one, it’s great for addressing imposter syndrome issues. For example, sometimes I feel like I should be writing more. When I poke at the feeling, it often turns into this.
- I should write more.
- Why do I think that? It’s partly because I read cool things from people that write more than I do.
- But by friendship paradox, it’s expected that those writers are writing more than me.
- So hey, maybe I shouldn’t feel too bad.
More importantly, the friendship paradox touches on another, more important idea: the things you see don’t have to reflect reality. If you base your assumptions of popularity on the popularity of your friends, on average you’ll come up short. If you base your assumptions of productivity by things you read online, it’s easier to see evidence of productivity from people who are very productive. If you base your views of somebody by things you hear them say, it’s warped by the chances you would have heard their views in the first place. And so on down.
Not all of these are applications of the friendship paradox, but it’s easy to forget about these things, and thinking about the paradox is a nice reminder.
-
Two Years Later
CommentsI’ll be honest: I completely forgot to prepare a post for my two year blogging anniversary until today. I’m vaguely disgusted that “two year blogging anniversary” is a valid English phrase, but sometimes languages are silly like that.
Ironically this is because I’ve been working on a larger post that’s been taking up most of my writing time. Hopefully I get that done soon.
I do feel I’ve been less productive this year than last year, blogging wise. Let’s see if that’s backed up empirically. Now, word count is a pretty bad metric for writing time, because in my experience most of writing is rewriting. But it’s the best metric I’ve got. Here’s a list of all posts I wrote this year, along with their word count.
1244 2016-09-07-contradictions.markdown 1936 2016-10-18-politics.markdown 1750 2016-11-08-right-questions.markdown 895 2016-11-12-hybrid-argument.markdown 1384 2017-01-20-mh-2017.markdown 931 2017-01-25-research-comm.markdown 592 2017-02-12-default-arguments.markdown 4035 2017-02-22-wasserstein-gan.markdown 1827 2017-03-18-we-have-a-problem.markdown 834 2017-04-19-acss.markdown 1925 2017-04-26-perils-batch-norm.markdown 3443 2017-06-27-hyperparam-spectral.markdown 1613 2017-08-12-openai-dota-2.markdown 22409 totalHere’s the same list for last year.
542 2015-08-20-things-i-did-in-my-khan-academy-internship.markdown 1744 2015-08-23-simulating-a-biased-coin-with-a-fair-one.markdown 2592 2015-08-25-perfectly-intelligent-mafia.markdown 599 2015-09-01-optimizing-process.markdown 1580 2015-09-09-the-other-two-envelope-problem.markdown 1988 2015-09-24-how-an-audio-play-about-a-time-traveling-pony-turned-me-into-a-fanboy.markdown 1703 2015-10-11-humor-proposal.markdown 1871 2015-11-02-friendship-games-geometry.markdown 465 2015-11-29-mlp-season-5-review.markdown 6376 2016-01-03-grad-school.markdown 1231 2016-01-20-mysteryhunt.markdown 1784 2016-01-27-deepmind-go.markdown 3101 2016-02-11-secure-computation.markdown 2647 2016-02-20-half-year.markdown 4537 2016-03-17-alphago-lsd.markdown 1353 2016-03-22-primes-pi.markdown 1323 2016-06-13-hillary-google.markdown 817 2016-07-03-email-time.markdown 714 2016-07-04-fruit.markdown 1361 2016-07-17-ml-sleep.markdown 1016 2016-08-03-pokemon-go-equality.markdown 39344 totalSo, I wrote around 17,000 fewer words this year. What changed?
* * *
The most important change is that I’m been working full time the entire year. It’s been a lot harder for me to get side projects done than it was in college. This is a well-known experience with well-known explanations, but let me explain anyways.
In college, I didn’t have designated work times. I went to class, I had lunch and dinner, and the rest of the day was a blend between work and free time. It was very easy for me to work for 20 minutes, switch to playing Dustforce for 40 minutes, blog for 10 minutes, flowing back and forth depending on mood until 3 AM. Then I would wake up the next morning and start the process over again.
It was even easier for me to do this because I didn’t have many group projects, and I had very few homework groups or study groups. Overall there was no accountability on my schedule, and the blending was a natural consequence of that.
This wasn’t a bad thing. I wasn’t very focused, but at any time there was a chance I could switch to being productive, and the activation barrier between everything was much lower. Blogging slotted itself very naturally into my day. (Well, more like into my night.)
All that has changed. I have meetings, coworkers, managers. I’m working on a project with collaborators across three offices. Coordination is a lot easier if everybody’s working times overlap - one of my collaborators works from London, and he’s deliberately shifted his schedule forward because the only good meeting time is his evenings (which are our mornings in California).
It’s the same problem that most companies have, and there’s a shared solution: a homogenized schedule with similar working hours. When I was a student, I worked more, but the work was spread out over the entire week, like maple syrup on pancakes. Sure, sometimes I didn’t finish work until 2 AM. But I had relaxed enough during the afternoon to make this feel okay.
In a job, all the work gets compressed into a single block. I do take breaks, and I’m lucky enough to work at a company that doesn’t care how long your break is…as long as you get your work done. That nagging feeling of getting my work done means I don’t take a lot of breaks.
The end result is that my energy and motivation gets spent at work in one big burst. By the time I get home it’s hard to do much more than browse the Internet and play games.
I’ve thought a bit about whether I can reproduce the college working schedule, so to speak, but there are benefits to working similar hours to everybody else. Overall I don’t think there’s a lot of value in changing how I work.
* * *
Okay, time for more uplifting news! In last year’s post, I mentioned a few posts I said I would write this year! Let’s see how I did.
The post about Gunnerkrigg Court is going to happen. I swear it will.
Uhhhh, that post didn’t happen. Okay, next!
I wrote the start of a really, really stupid short story about dinosaurs. […] I hope that story sees the light of day, because the premise is so stupid that it might just wrap back around to brilliant.
That one didn’t happen either. Alright, what else did I say I was going to work on.
Oh, I said I was going to write a post about AI safety! That one is in progress! And not done! A bit far from done, in fact, but let’s round that up to a win. If failure is success rounded down, then surely success is failure rounded up.
To be serious for a bit: yes, I am a bit disappointed I haven’t written posts that I said I wanted to write. But I’m not actually that disappointed. It feels like I’m spent about as much time writing as I wanted to spend this year.
Well, maybe I’ve written a bit less than I wanted to.
Let’s say I’m more at peace and leave it at that.
In terms of life trajectory: I didn’t apply to PhD programs, mostly out of laziness. I should have applied, just to keep the option open, but I don’t see myself going to academia any time soon, unless my research interests or work drastically change.
I think I’ve changed less this year compared to last year. Again, this is a common sentiment. I found it easy to grow every year of college - it just happened naturally. I’ve been finding it harder to do so after leaving that environment.
When I think about what I would regret the most, it’s something like this: the worry that five years from now, I’ll look back on my life, and have no idea where five years of my life went. Something kind of like this.
Apple Bloom has sunk into a psychotic depression, because she knows this is how it all starts. Pretty soon they’ll all be living in different cities, and they’ll only see each other at New Year’s. […] Sure, they’ll start talking about things they did back in the day, like “Oh hey, remember that time we were happy and life felt like it had meaning?” And after a few drinks, it’ll feel like there’s still a connection between them, and they’ll share all their online handles and say they should get together sometime. […] Then the next day when they sober up, they’ll get right back to never getting around to it.
(Totally Legit Recap, “On Your Marks”. Warning: NSFW humor about My Little Pony.)
It’s that worry that pushes me to work on things, even when I have reasons not to.
Will pre-emptively thinking about quarter-life crises stop me from having a quarter-life crisis? Eh, maybe. We’ll find out.
I think I’ve rambled long enough. Two years down. Here’s to two more.