Startup Life: Three Months In

Joining a startup after years in Big Tech has been a breath of fresh air.

Matt Welsh
9 min readJun 12, 2019

A little more than three months ago, I left a long-time position at Google (as head of the Chrome Mobile teams in Seattle and Kirkland) to join Xnor.ai, an early-stage startup developing AI for embedded devices. In an earlier post, I wrote about my reasons for jumping ship for a startup: the TL;DR was that I was ready for a big career change, and I was very excited about the technology and the team at Xnor.

A day in the life at Xnor. Pretty much the whole company fits in one room.

I have made an amazing discovery: working at a startup is not at all like working at a Big Tech company. I wanted to share some of my findings here.

So far, I’m having a blast at Xnor. I had forgotten how much fun it is to jump in to a totally new area and drink from the firehose. I am by no means an AI and ML expert, but even after a couple of weeks I was able to hold my own in conversations having to do with loss functions and learning rates and depthwise convolutional layers. As one of my first projects, I implemented a new neural network architecture and trained it against a large dataset to compare performance against some of our existing models, thereby unlocking this achievement:

Guess it’s time for my raise.

What does Xnor do?

We develop edge AI technology that runs on low-power, embedded devices — typically low-end microcontrollers. The company spun out of University of Washington and the Allen Institute for Artificial Intelligence back in 2016. Xnor’s key technology is a lightning-fast inference engine that is capable of running sophisticated ML models in a fraction of the memory and computational cost of conventional implementations, and a process for converting models to run on that engine. The basic business model is that we license our AI solutions to companies that embed it into their products. (You can try it out yourself using our free AI2GO SDK.)

It turns out there’s a huge market for this technology — especially from companies that have already deployed vast numbers of low-power CPUs, and aren’t able to retrofit a power-hungry, expensive GPU or TPU into their platforms. We work with partners across a large number of industries: home security, retail, appliances, automotive, and consumer electronics. So the applications are really exciting and really diverse.

Xnor is a technology-first company. Everything we do is bleeding-edge and demands that we continue innovating to stay a step ahead of the competition. The team is fantastic — a blend of deep ML expertise and embedded systems wizardry — and I am learning a ton from them.

As a young company, there is a lot of uncharted territory and plenty of open problems to tackle. It’s a big open playground where we get to focus on building cool stuff without all of the encumbrances of a big company: politics, massive amounts of legacy code, organizational overhead. When I left academia for Google, I went from spending about 2% of my time to around 80% of my time doing interesting technical work (though that number dropped a lot once I started managing a large org). Moving from Google to Xnor, the ratio is more like 95%.

Oh, and by the way, we’re hiring.

Rolling your own

The first big difference between the big company and the startup is the maturity of the technology stack and how much freedom it affords you to innovate.

At Google, pretty much all of the infrastructure and tooling is incredibly mature. For any problem you can think of there’s already at least one, and often more than one, solution (although whether any of the solutions are actually what you need is another matter). Take distributed storage. Google has so many cluster storage systems — dozens, I’d wager — that someone eventually wrote a lengthy guide on how to pick which one. It’s insane.

You also often find that someone else already solved your problem for you, which is nice. Case in point: I once spent a couple of days implementing a system to periodically pull data from a third-party website and dump it into a BigTable before someone told me that there was already an entire system that did exactly that. All I needed to do was write a configuration file for it. (Ironically for a search engine company, finding what technologies exist within Google is pretty hard. It’s a lot of word of mouth.)

Along with that maturity comes, I think, a certain degree of ossification. It is very hard to change the infrastructure at Google. Given the huge number of projects and teams that would be affected by, say, even a minor API change to BigTable (for example), there’s a strong tendency not to change that kind of stuff. So teams often end up going to great lengths to work around limitations in existing systems with which they need to interface. Ironically, all of the source code for all of Google’s internal systems is sitting in the same repository, but actually changing much of that code is not going to be easy. Big changes do happen, to be sure, but they are often multi-year efforts staffed by huge teams.

At Xnor, we can make big, fundamental changes to our (relatively tiny) software stack with ease. Nothing’s set in stone yet. For me, that is very satisfying.

As with most startups, Xnor cannot afford to invest huge engineering effort building homegrown infrastructure like code review systems, build systems, and testing frameworks of our own. So, we need to leverage all of the great work being done by open source projects as well as commercial SaaS vendors. Google builds nearly all of its infrastructure in house, and (fortunately for the rest of us) has open sourced a lot of it: we make heavy use of things like Bazel, Kubernetes, and protobufs. At the same time, I find that many commercial software offerings are better than what Google has built internally. Slack and Slab are just two examples where Google’s in-house stuff is pretty far behind what’s available in the real world.

Moving fast, but not breaking things

The second big difference is the sheer speed of execution at a startup: it’s way, way, way faster.

Big companies tend to move slow, and not just because of software complexity. At least in my corner of Google, even simple things could take months of negotiation, analysis, discussion, re-analysis, re-litigation before settling on a decision (that is, assuming a decision was ever reached — much of the time we would never converge). Much of this is due to the sheer number of stakeholders, and, I believe, a tendency to avoid committing to any decision, no matter how big or small, until everyone is satisfied with the answer. In part this is because making the “wrong” decision at Google is such a big friggin’ deal, and can often be hard to reverse. But I also think that many people eat Google are too skittish to disagree and commit.

True story: A major rollout of a new feature in Chrome (that my team was responsible for) was once blocked for more than a quarter because the implemented UI disagreed with the designer’s mocks by a few pixels of whitespace. In this case, it wasn’t that either side was digging in their heels — simply that all of the process involved in rolling out a new UI feature tended to get us wrapped around the axle.

At Xnor, we don’t have that problem. It is not an exaggeration to say that we typically make major decisions in five minutes of back-and-forth by the coffee machine. It is just so much easier to reach consensus when you have three people to talk to.

As an example, in the run-up to the launch of AI2GO, Xnor’s developer SDK, we were trying to decide the best way to structure some of the library and documentation components. Not rocket science, but I had grown accustomed to these kinds of things dragging on for months. Myself, the VP of engineering, and a couple of others settled on it in less time than it took to make an espresso. I didn’t know that such a thing was even possible.

Every day I’m hustlin’

Apart from the speed of execution, a major difference is the sheer energy level in the office and amongst the team. There’s a lot to be said for the energy level at a startup.

Google could be pretty relaxed at times: only about a third of the people were at their desks at any given time, and it was generally fairly quiet (well, depending on where you happened to sit in the office). I also didn’t feel that we were in any rush to get things done — the standard unit of time was quarters, not days or weeks.

At Xnor, it’s usually buzzing and full of activity (well, at least after around 10am when most of the folks roll in). There’s so much going on and I find it really energizing. Being a startup, we can’t afford to be too relaxed, and the pressure is higher. Deadlines matter, and we gotta move fast. There’s a sense of urgency around almost everything that we do.

At the same time, though, I don’t see people pulling all-nighters, nor regularly having to put in late nights or weekends — though it does happen from time to time. Enough of us at the company have families and kids and lives outside of work that we’re not willing to tolerate too much of that. Personally, I wouldn’t stick around if I had to work round the clock to keep up: no job is worth that to me.

Where’s the bowling alley?

Google’s campus and perks are famously awesome. At the Seattle campus alone, there are five full service cafes, serving breakfast, lunch, and dinner; full-time massage staff; in-house doctor; in-house barber; three separate barista bars; in-house gym; a music studio; a fully-stocked makerspace with multiple 3D printers, laser cutters, and all the tools you could ever need; and much more. The Mountain View headquarters is an order of magnitude more ridiculous — the on-campus bowling alley being my favorite example of this. People have been known to live on campus at Google, at least surreptitiously. Frankly, there is little reason to leave. It’s like a nerd wonderland.

At Xnor, we have … well, catered lunch three times a week. (But free snacks and drinks anytime!) And on the other days we heat up leftovers or go out to one of the local restaurants. So it’s a lot more scrappy and low-key.

At first I thought I was really going to miss all of the overwhelming diversity of perks at Google. Having been at Xnor a few months, I realized that I really don’t need all that stuff to be happy at work. 90% of the perks I never took advantage of, and what really matters is working with awesome people (which I did at Google, and do at Xnor); having a lot of freedom (Xnor wins there, by a long shot); and having fun (likewise).

No regrets

My eight-plus years at Google were fantastic, and I absolutely do not regret my decision to work there. I was so fortunate to work with so many incredible, talented people, and to work on products being used by billions of users. Those years really were the happiest and most productive of my career, and given the chance to do it all over again, I would absolutely do it.

I learned many things at Google that might be difficult to learn anywhere else: how to develop products that scale to a significant fraction of the world’s population; how to leverage Google’s tremendous technical stack; and much more. So while I’m loving the startup life right now, I think my time in Big Tech was very much worth it. And of course, it gives me the opportunity to translate the lessons I learned at Google into a very different kind of company working in a different domain.

--

--

Matt Welsh

AI and Systems hacker. Formerly at Fixie.ai, OctoML, Google, Apple, Harvard CS prof. I like big models and I cannot lie.