/param/

Notetaking for fun and profit.

Sun, 21 Mar 2021 00:00:00 +0000

If you read a book, how much of it do you retain a few months after finishing it? If you try to recall, can you recall more than the summary on the back? I’ve always found it really hard to retain anything more than broad strokes of any book I’ve read.

I was in San Francisco for a Stripe event where Andy Matuschak gave a talk about why books don’t work. While I’m not sure I agree with the premise that books just don’t work, I do agree that you need extra effort if you really want to internalize knowledge from books.

Enter notetaking. I’ve been taking notes on books I read for the last year and it’s been quite helpful. I started with just taking notes for books, but now I use dendron at work too.

A lot of the notes are interesting trivia that I know I’ll forget if I don’t jot down. Writing things down helps me remember things better because it means that I’m not endlessly consuming words, I’m also producing something, even if it is just a few sentences rephrasing the text I read. Once you produce something about a topic, you know it much better.

Another habit I’ve picked up is scrolling through the notes every so often, which leads to a kind of spaced repetition, helping me retain things better.

Notes can also be quite a good unit of knowledge to be gained. Instead of setting goals of the number of pages / chapters you want to read, you can set a goal of quantity of quality notes you want to take. Books can have different levels of density. Trying to read X pages or Y chapters every week is hard, and setting good values of X and Y for every different book can be quite difficult.

Another interesting benefit I’ve had with writing notes is that the amount of content I produce has quite dramatically increased. Now that I have a place to write stuff without it needing to be a blog post that others might find interesting, I write a lot more. Most of it will probably never be useful to anyone but me, but I have a feeling the archive will be quite valuable to me a few years down the line.

So yes, I hope this blog post convinces you to try out a workflow where you take notes. If you do, let me know how it goes!

Lessons learned by building things

Fri, 22 Jan 2021 00:00:00 +0000

2020 was not a normal year. With the pandemic and the lockdowns, I had a lot of time on my hands. Given that every plan I had made was cancelled by the virus, I put some time in on building products from scratch that people would want to use. Overall, I built two things with friends: soluchan.com, a website to help people preparing for competitive examinations in India, and funnel.fyi, a tool to help you organize your job search. Here are a few things I learned building these products.

A small feedback loop is essential.

Build it and they will ~~come~~ do absolutely nothing.

It’s tempting to take the easy path and spend weeks writing code, developing new features. However, if you do not have users and do not know for sure that your users want the thing that you’re writing, there is a very high probability that all that code will just sit in a lonely git repository unused. It’s best to write as little code as possible, talk to potential users, get feedback, and then write code with that feedback in mind. This feedback loop should ideally be as small as possible.

It’s hard to switch context.

Working a full-time job and trying to build things is not easy. Engineering is not a job you can clock out of. A lot of good ideas come to me in the shower or late in the night when I’m on the bed getting ready to sleep. If you focus on one thing for 40 hours a week, the side project will languish. It’s harder to focus on it, it’s harder to innovate with it. I think that to do this well, it needs to be a full-time focus.

Marketing is essential.

Being an engineer, I’ve been comfortable with writing code to build things for quite a while. However, those are not the only skills that are needed if you want to build something that creates value for others. People will not be aware of what you’re building if you do not push it. Pushing the thing you built in a way that’s engaging and fun is not easy or obvious, but it’s necessary. Nothing will happen without it. Things are made or broken by marketing. At some point, you will need to spend ALL your time on it.

Build stuff

Fri, 23 Oct 2020 00:00:00 +0000

I read a blog today that really resonated. The basic idea is that you should build stuff even if it doesn’t feel like the best thing that you could be building. It’s hard to come up with the best thing that you should be building. As such, it becomes very very easy to fall into a trap and feel like you need to postpone actually building your stuff for other things in favour of other stuff like “learning” and “gaining experience”.

The best way to gain experience is to build stuff. That way, you get to realize the problems you face and the skills that you’re missing. You also learn more stuff building actual stuff than you do reading books about building stuff.

In the year 2020, I’ve been putting this in action and as such I’ve built a few things that haven’t gotten much usage but have still been very useful learning experiences. I’m not gonna list them because they’re mostly embarrassing but it’s easy to find them, really. But overall, I’d definitely recommmend taking the strategy of building stuff that you think could be useful to your goals, and learning as you go.

One year at Stripe!

Fri, 25 Sep 2020 00:00:00 +0000

It’s a month and 20 days late but my 1st Stripeversary badge came in!

Write a design document

Fri, 18 Sep 2020 00:00:00 +0000

I read this blog post and spent some time thinking about how writing skills have helped me become a better engineer so far (and how they could help in the future!).

My first job was at a place where ALL communication happens in a public IRC channel. This basically meant that you need to be good at written communication to be able to have meaningful impact. So I understood the need to be able to write well implicitly very early. However, I didn’t really explicitly think of these benefits until I started working for Stripe which has a pretty big writing culture.

Here’s what I got: It is generally a good idea to write a design doc before starting a project. No matter how small the project is, if you think it’ll take you longer than a week, spend a few hours writing a design document. Try to list out the work that you’ll need to do. Think of how you’ll release the changes. Think of people who’d be interested in looking at your work and get those people to review it.

Why write words instead of code? There are a number of advantages:

Writing a design document down to the nitty gritty details helps you figure everything out yourself. I worked on a lot of projects willy nilly, without much planning and I can see now how good planning early on would have helped me execute on these projects better.
Writing a design document allows coworkers (or other opinionated people) to communicate their opinions to you early. That way, you can resolve disagreements and (more importantly) get better ideas early on, making it much much easier to execute. It’s not fun rewriting an entire pull request because code review brought up significant issues or because code review revealed a much easier way to do what you want to do.
Writing a design document keeps you honest. Once you have a design document fleshed out, you know what tasks you need to finish. If you’re the kind who puts deadlines or milestones into design docs, maybe you even know the dates when you should finish such stuff.
It helps you become better at estimating work. One of the skills that I’ve been trying to develop is being able to make realistic estimates on how long a project would take. It’s harder than it looks, I have a tendency of being overly optimistic, or just missing things that will need to be done. When you’re writing a story of how you’d finish a project, you’re made to go through ALL the things that you need to do, and making better estimates is easier.

Also, the future is remote and asynchronous, it’s probably best to brush up on those writing skills, they’re gonna be good skills to have when your teammates work in a completely different time zone.

A world that's designed to make you sad and angry

Fri, 05 Jun 2020 00:00:00 +0000

You wake up, brush your teeth and decide to check Instagram. There’s the friend that’s doing better than you at life. There’s the friend that’s having fun at the vacation while you work your job.

You decide that Instagram is too self-obsessed and decide to do something constructive. Maybe read the news, that’s a good thing, right? You should know what’s happening around the world. Well, another day of a bigoted government trying to bully a minority [¹][²]. Or another person that’s dead because of shitty systemic problems that no one is incentivized to solve [³][⁴][⁵].

I’ve recently realized that social media and news is designed to make you feel bad. If you feel bad, you react to stuff more.

Facebook has published an actual paper on their experiment of mass manipulation of around 700k people’s emotions [⁶]. So yeah, I’ve recently realized that I’ve been paying people (directly or indirectly) to make me sad.

What’s the way forward here? I don’t know.

I’m gonna stop reading the news, it’s too much stress for me. My brain is not designed to handle all of the world’s crises at the same time. I’m also gonna stop using social media websites with prior history of trying to manipulate my emotions and make me sad in the name of an experiment.

There has to be a better way to know what’s happening around you and what your friends are up to. There needs to be some de-incentivization of the fake news reality and the fake instagram reality that we’re all buying into.

References

https://www.bbc.com/news/world-us-canada-52932611 ↩
https://www.theguardian.com/world/2020/feb/06/feed-them-bullets-not-biriyani-bjp-uses-delhi-elections-to-stoke-religious-hatred ↩
https://en.wikipedia.org/wiki/2020_Palghar_mob_lynching ↩
https://en.wikipedia.org/wiki/2020_Delhi_riots ↩
https://en.wikipedia.org/wiki/Cow_vigilante_violence_in_India ↩
https://www.pnas.org/content/111/24/8788.full ↩

East India Company — India's encounters with unbridled capitalism

Tue, 11 Feb 2020 00:00:00 +0000

The East India Company started off innocently enough. In the 16th century, India accounted for 22% of the entire world’s GDP. The company decided to cash in on that and make some money trading with this exotic rich part of the world. The first century of their trade went relatively well, with everyone in the process getting rich. This is where corruption comes in. People in the company got rich in the first century of its operations. With money comes influence. They lobbied for complete monopoly in the British Parliament and got it.

Meanwhile, the Company also needed to keep a standing army to “safeguard” its assets. For almost a century, this meant a few hundred guards. By 1778, the army had grown to 67,000 people. An army of 67,000 in the hands of a company made solely for profit. One wonders if the British would have regulated the Company sooner if the army stood in Britain instead of far away in a remote nation, outside of their sphere of view. But it is what it is. India had in its presence an entity whose sole motive was profit and who had the power of 67,000 well trained soldiers. It goes without mentioning that most of these soldiers were Indian “sepoys”.

Now, what do you do when your neighbours are rich as fuck, you have a bigger stick than them and you want money? You beat the neighbour up with the stick and take their money (if you have no ethics). Most corporations have no real sense of social responsibilty. The East India Company definitely didn’t. Soon enough, the Company was fighting wars, marking almost all of India as territory of its own and exacting “taxes”. Now, the thing about taxes is that they’re supposed to be used to improve the lot of the people paying the taxes. In a democracy, the government has the incentive to do this because they’ll get kicked out in the next election otherwise. No money-making company has this incentive. The East India Company didn’t have it either. The money went to the company and the taxpayers would literally die without seeing a penny of it.

In the midst of Company rule, in 1770, the State of Bengal went through one of the worst famines in the history of the world. Around ten million people are estimated to have died in the famine. The rate of land tax however was increased by 10% in August 1770. Revenue earned in 1771, the peak of the famine, was higher than the revenue collected by the Company in 1768.

It wasn’t just the Company that was making money. Individuals “received” numerous “gifts” from locals too. Robert Clive, who has a pretty nicely whitewashed Wikipedia article, was the British general who led the first real war the East India Company fought. After his second trip to India, which lasted five years, Clive came back to Britain with 300,000 pounds in cash. Assuming a modest inflation rate of 2% in the last 300 years, that amounts to around 70 million pounds today. This is in cash, it’s not like the equity today’s billionaires have. And this is just one person. Clive then went on to become a British Member of Parliament.

This went on for an entire century before Indian people had had enough. In 1857, the first war of Indian Independence started. It went on for 1 year and 6 months before the British regained control. “Saner” heads prevailed, the company got dissolved. All the assets of the Company went to the Crown and India officially started being ruled in the name of the Queen.

And they all lived happily ever after. Well, not really. But that’s a whole another story.

Sidenote: I’ve started reading history, and it’s really hard to actually absorb history without reading it more than once. I’m starting to write up stuff that interests me, so that I have an incentive to read more than once, because writing up interesting stuff needs the actual facts to be verified.

Haven't won yet

Sun, 15 Dec 2019 00:00:00 +0000

It’s been almost 5 months since I joined Stripe and I was recently looking at some of the culture guiding principles. I really love these and a lot of them are things I’ve lived by long before I had heard of Stripe, see optimism, urgency and focus.

However, the one principle that really resonated with me was “we haven’t won yet”. Stripe is worth 35 billion dollars and processes billions of dollars in payments all over the world. However, we still realize that there’s so much left to do. Complacency is a thing to totally and completely avoid. There’s always a focus on how to keep getting better.

I’ve been thinking on these lines since I was in college. It wasn’t in these exact words. But “we haven’t won yet” is a really nice way to phrase it. No matter what happens, there’s always room to grow, always ways to improve. And if you’re not growing, what you’re doing is probably not what you should be doing.

This is very easy to say, but I’ve found that it’s much much harder to actually live. When you’re in school or in college, growth is very easy to measure. You’re working towards graduation! Getting to graduation is not exactly an easy task. However, growth is still well defined. It gets harder to figure out after you’ve graduated.

I haven’t been able to figure out how I really want to grow so far, however I do realize that being directionless for a long time won’t be good for me. The default in life in all things seems to be mediocrity and I do not want to feel mediocre. The first step here is to figure out the direction that I want to go now. And then start working to get there.

But, living by “We haven’t won yet” every moment of your life isn’t sustainable either. I believe in fleeting moments of celebration to cherish steps you’ve taken to get closer to winning. Without those fleeting moments, I’m not sure if “winning” would even be worth it. There’s a delicate balance here, it’s been hard for me to figure out. However, one thing I have figured out definitely, I haven’t won yet.

What do I do?

Sun, 22 Sep 2019 00:00:00 +0000

I watched Ad Astra today. It was a nice movie about how instead of looking at the stars and space for answers, we should maybe be looking in.

But that’s not the point of this post. There was a moment in the movie that made me think. Brad Pitt is sending a message to his long lost father and a sentence in the message really stood out to me.

“I’ve dedicated my life to the exploration of space”

Personally, I cannot define what I want to dedicate my life to… This is a problem. I do not want to think of myself as having done nothing of note five years in the future. A job at Stripe is a really amazing thing to have right now, but I’d be really disappointed if it was my life’s work product.

I don’t know what I need from myself right now. I know that Stripe is the biggest learning opportunity I’ve ever had in my life and I need to make the most of it as long as I can / need. However, my MetaBrainz work is the work I’m really proud of. I need to stop procrastinating and start working more on that too. However, most importantly, I need to find out what’ll make me feel proud of my work in life.

So yes, today made me realize that I need to think more about stuff I need to do before I turn complacent. I think this realization was actually pretty important, and I’m thankful that it doesn’t seem to have come too late.

Moving AcousticBrainz

Fri, 16 Nov 2018 00:00:00 +0000

Thoughts on migrating a terabyte scale database to a new server in production for the first time.

In the interest of making this blog an archive of interesting technical work that I do, I’m gonna write about the recent work I did while migrating the AcousticBrainz website from an old server to MetaBrainz’s relatively newer and shinier Hetzner infrastructure.

This was the first time I worked with a production database of this scale, and it was a real learning experience. It really felt like I had jumped off the deep end, but it was really fun!

Some backstory first, AcousticBrainz is a music technology project which crowd sources acoustic information for music recordings and is a collaboration between the Music Technology Group at Universitat Pompeu Fabra and the MetaBrainz Foundation. The project has collected information about 3.7 million unique recordings and has individual submissions from users for over 11 million recordings. All the data is stored in a single PostgreSQL database for now. The server that AcousticBrainz used to run on (we called it spike, after the Tom and Jerry character) had gotten old and started spitting out hard disk failure warnings, so we decided to move it to the central Hetzner infrastructure where other MetaBrainz projects are hosted.

We use docker for all services running in Hetzner and it has worked pretty well for us so far. So the first task was creating a production docker environment for AcousticBrainz. Consul is used to provide config values for the AcousticBrainz server which needed some new code and consul template files to be written. This is relatively simple stuff that did not take too long. We also have a repository to store all config values and scripts that need to be run on each of our servers. So I also wrote code to run the three different services that AcousticBrainz needs in different docker containers.

After that, I started work on creating data dumps of the AcousticBrainz data. There was already some code that dumped the entire database into a lzma compressed file. However, it was old code that hadn’t been run in a long time and the database had gotten biiig since then. The way the code worked was that it dumped each table as a file into a directory and then added the entire directory at once into a tar file. However, this approach doesn’t work now, because the table that stores the lowlevel json data that users submit to us has become too big to be stored in a single text file uncompressed. The lowlevel_json file has 11 million rows right now with each row containing a relatively large JSON document stored in a column of Postgres’ cool JSONB type. The table takes around 357 GB when stored inside Postgres and this balloons to much over the space we had on spike. So, I wrote code that dumped 500,000 rows into a file and compressed it before dumping the next 500,000 rows.

The compressed AcousticBrainz data dump was around 169 GB in size which seemed reasonable. Then, I realized that the server we were planning to run the webserver on (called boingo, after Oingo Boingo) did not have enough storage space or computational power to hold and work with the database. This led to us getting a new shiny server called frank (after Frank Ocean!) which has a pretty big 7200 RPM hard disk and over a 100 GB of RAM. We also decided to upgrade to Postgres 10 during the migration, which led me to creating a docker image for PostgreSQL 10 that we could use in production.

After this, I imported the data into the empty postgres server which worked pretty well. Everything seemed set for a small downtime for migration where we’d just create a small incremental data dump, move it to frank and import, bring spike down, bring the webserver up on frank and be done with it. The steps were written up and we were ready to go.

Things started, I brought the site down on spike, created an incremental dump, imported it to frank. Everything worked. We decided to do a integrity check of the new database once before bringing the new site up. This is where the trouble started. The number of rows in one of the the tables was 10 million when it should have been around a 100M, yikes. We realized that there had been a bug in the original data dump code that we’d written. It was a pretty small bug, the key we were using to dump the data was incorrect. One line fix. I thought that we need more tests for our data dumps code.

Well, at that point we decided to just go ahead and dump and import the table individually instead of stopping the whole process. The downtime was much longer than expected because of this, the table was pretty big, 100M rows is no joke, it took pg_dump hours to dump it. Then, I dropped the table on frank and began an import of the dumped file. We had decided to not drop constraints before importing for sanity reasons, but that turned out to be not that good an idea. It took the import 5-6 hours before it was even done halfway and the time to import new rows was increasing. We gave up, stopped the import and dropped all constraints before starting a new clean import. This worked much much faster and was done in around an hour. At that point, we did another sanity check of the database, before bringing the site back up.

Some static files like binaries and old dumps we linked to were still hosted on spike (another thing I missed!), so I had to whip up a quick pull request changing links temporarily. I was doing this at 3 in the morning and I had started working on this the previous day 11 in the morning. It was the longest, most intense production deployment I have ever done. Pretty fun, now that I think about it, but I was tired then.

Later, I set up an FTP server on frank and moved the static files we were hosting there.

There were a lot of things that I learned in this entire process. First thing was that we should really sanity check literally everything before bringing any production service down. Second thing was that importing data with constraints in a database (especially for large amounts of data) is not very feasible. Third is that this level of control is not something that I would ever get as a new grad in any big company. Being thrown off the deep end here at MetaBrainz was really awesome. Another thing that I forgot to mention was that the entire migration process was done remotely over IRC with me sitting in college in Hamirpur, India and my teammates in Barcelona. This really teaches efficient communication and teamwork.

All in all, working with production grade big data projects has been pretty awesome, and I hope I continue to learn as much as possible as early as possible.