logging my thoughts on technology, security & management
I read a comment on HN that sparked this article: GPT is kind of like DevOps from the early 2000s.
Here’s the hot take: I don’t see the primary value of GPT being in its ability to help me develop novel use cases or features – at least not right now.
The primary value is that it MASSIVELY lowers the barrier of entry to machine learning features for startups.
What’s my line of reasoning? Well, here are some surprising things about how we use it:
Continue reading
My startup Truss (gettruss.io) released a few LLM-heavy features in the last six months, and the narrative around LLMs that I read on Hacker News is now starting to diverge from my reality, so I thought I’d share some of the more “surprising” lessons after churning through just north of 500 million tokens, by my estimate.
Some details first:
– we’re using the OpenAI models, see the Q&A at the bottom if you want my opinion of the others
– our usage is 85% GPT-4, and 15% GPT-3.5
Continue reading
One day, while The Manager was walking back from a morning coffee run, a group of frazzled engineers came near and spake unto him, saying: “Most Esteemed Boss, we are unable to hire Talent – and many of our candidates refuse to take our coding challenges! The labor market is tight and our staff are burning out, should we not ease our process and forgo coding challenges, so that we might secure butts-in-seats more quickly?”
The Manager turned to them, and upon seeing that they were truly desperate, sought to teach them them using this parable:
Continue reading
While I was at PKC, our team did upwards of twenty code audits, many of them for startups that were just around their Series A or B (that was usually when they had cash and realized that it’d be good to take a deeper look at their security, after the do-or-die focus on product market fit).
It was fascinating work – we dove deep on a great cross-section of stacks and architectures, across a wide variety of domains. We found all sorts of security issues, ranging from catastrophic to just plain interesting. And we also had a chance to chat with senior engineering leadership and CTOs more generally about the engineering and product challenges they were facing as they were just starting to scale.
It’s also been fascinating to see which of those startups have done well and which have faded, now that some of those audits are 7-8 years ago.
Continue readingThis is one in a series of deeper-dives into various Learnings from 5 years of tech startup code audits. In that article, I list several observations I had during the course of doing code audits fro 20-30 tech startups at or around the Series A / B mark.
Security seems to be on the up-and-up, despite all the bad news you hear in the media. What’s driven this improvement? Well, frameworks, cloud infrastructure, and the big cloud platforms have been hard at work creating the ”pit of success” — places where users just fall into being secure rather than having to fight to be secure — as a way to discourage the most severe security practices has by and large been an enormous success. This article is about our on-the-ground observations of these success stories.
We started doing code audits in 2014.
Continue readingThis is one in a series of deeper-dives into various Learnings from 5 years of tech startup code audits. In that article, I list several observations I had during the course of doing code audits fro 20-30 tech startups at or around the Series A / B mark.
We did several code audits for companies that rapidly scaled their engineering orgs relatively early on (we’re talking 50-100 engineers, maybe 10-35M annual revenue, series A/B). None of them are doing well right now, and some are out of business.
What made this observation interesting is how different it is from stories of the well-known counterexamples (Uber, FB, etc, etc, etc), where it seems like the ability to double headcount every 6 months is assumed to be a key component of their ability to scale rapidly with hypergrowth.
But I think we are taking the wrong lesson from these success stories. Here’s the real lesson:
You don’t grow engineering headcount like crazy in order to achieve hyprgrowth: these companies underwent hypergrowth first, and then are forced to grow headcount rapidly.
Typically the architectures we saw from these large engineering groups were not bad: they just were not really necessary. We often looked at the code and the infrastructure and kind of scratched our heads: why were they doing all this? It just didn’t seem to make sense. But then we talked to the very smart engineers they had, and it seemed like there were good explanations for most things. Looking back, I think a lot of the complexity was frankly busy work: you bring in a lot of engineers without enough truly business-essential work to do, and they will come up with things to do. This is not a crack on them, it’s just human nature.
A note before we begin: I’m arguing that technology ROI discussions are broken, not that ROI as a decision-making tool is broken. A solid understanding of how to calculate and use ROI is an essential skill for any tech executive, and when done right, it’s a powerful decision-making tool. This post is about how technology discussions that exclusively look at ROI often result in a one-eyed analysis that lacks depth.
Technical leaders need a wider range of tools for communicating the value of technology, and especially technology innovation. Communicating the value of technology is not a trivial task—and the point of this post is that exclusive reliance on the most commonly used tool for communicating value—Return on Investment (ROI)—will lead to broken discussions.
Continue reading
“In the brain of all brilliant minds resides in the corner a fool.”
Aristotle
Writing about “Best Practices” can get boring, so I thought I’d take a break this week, and write about some bad engineering practices that I’ve found the absolute hardest to undo once done. Real foot-guns, you could say.
Each of these is a bit controversial in its own way, and that’s how it should be—I’d welcome any counter-views. The prose in this post is a bit more irreverent than normal—in most cases, I’m poking fun at myself (both past and present!), as I’ve been guilty of each of these foot-guns—and a lot of them, frankly I still struggle with. Hopefully this post will generate some “motivation through transparency” 🙂
A few years ago, I was in one of my ruts. Everything I was working on seemed to be bogged down or low-leverage. What was so frustrating was that this had come on the heels of a few amazingly productive months, where I had gotten a lot done. Worse yet, this seemed to happen cyclically: periods of productivity and a sense of accomplishment were followed by periods of delays and a sense of frustration.
Coincidentally, around the same time, I had just heard of the Peter Principle, which goes something like this:
Continue reading“Every employee tends to rise to their level of incompetence”
Laurence J. Peter, THE Peter Principle (1969)
Hiring experienced engineers is one of the most difficult and important things that engineering leaders have to pull off. But it’s hard to gauge experience in a series of short interviews. I’ve definitely worked with some amazing engineers who probably wouldn’t have been hired in some of my previous hiring pipelines.
Here are some tips on things I’ve found that work and don’t work.
Continue readingI’ve been thinking recently about how to discover and hire great engineers in the hottest job market in decades. One of the biggest hurdles to hiring good engineers, and especially experienced engineers, is that they’re so. unbelievably. expensive.
Just take a look at some of the total compensation packages on levels.fyi:
There’s always a reason to rebuild your app. Always. But once you’ve been through a few rebuilds, you realize that talk of rebuilds, like talk of tax reform or anarchy, is just a tad bit dangerous—you never know what kind of danger you’ll end up in if you actually convince yourself to go through with it.
Continue readingThe core principle is this: fish nets over fishing lines. In the case of security monitoring, fish nets are alerting on anomalies, where anomalies are defined as universal constants that have been broken. Fishing lines are manual search procedures. Phrase this principle like this addresses the two seemingly intractable problems with security monitoring:
Continue readingThe more systems that are secure by default, the less twiddling your IT team has to do for each deployment. Less twiddling means fewer chances to make errors that lead to security breaches.
Continue readingThis core principle can be summed up by the famous Reagan Cold War quote: trust but verify. Transcendent CISOs trust their people with privileged access, but are simultaneously very stringent about authenticating them.
Continue readingThat first word—continuous—is the core of this control. “Continuous” has seen a bit of hype in tech circles in other contexts. In particular, I’m thinking of continuous integration and continuous delivery from the world of DevOps and continuous improvement from the world of Digital Transformation. Why not think of security the same way?
Continue readingThe same Golden Rule that applies to hardware applies to software: know what you have. No user on your systems should be able to install an executable onto a company device without the approval of security. This may seem like a draconian policy (and a short-circuit process does have to be in place for certain technology-heavy teams like R&D or the dev team), but it’s necessary.
Continue readingThere are only six controls in the Top 20 list that are designated “Basic,” and an inventory of your hardware is number one. I actually would like to rephrase this control slightly, so it better fits the core principle I wanted to highlight: if there was ever a Golden Rule in enterprise security, it’s this: know what you have.
Continue readingCISOs have an impossible job. When it comes to developing a roadmap for my company’s security program, where is the best place to start? That what this series is about.
Continue reading© 2024 Ken Kantzer's Blog
Theme by Anders Noren — Up ↑