Lawrence Kesteloot Reflects on Coding Machines
Books Covered
Book links are affiliate links. We earn from qualifying purchases.
Authors
Hosts & Guests
Transcript
This transcript was auto-generated by our recording software and may contain errors.
Carter Morgan (00:00)
nature an optimist.
Lawrence Kesteloot (00:00)
I've heard a long time ago that
like a programmer is like a short-term pessimist and a long-term optimist. So you have to be short-term pessimist. It's like, you got to believe that there's a bug in this routine and just like carefully look for it and don't just be like, sure. It's fine. You got to be like pessimistic and really go for it. But you've got to be long-term optimistic because like it's too hard to do any other.
Carter Morgan (00:03)
Yeah.
Nathan Toups (00:11)
I love that. I love that.
Carter Morgan (00:12)
Yeah. Yeah.
Carter Morgan (00:22)
Hey there, this is Book Overflows, the podcast for software engineers by software engineers where every week we read one of the best technical books in the world in an effort to improve our craft. I'm Carter Morgan and I'm joined here as always by my co-host, Nathan Toops. How you doing, Nathan?
Nathan Toups (00:33)
Doing great. Hey, everybody.
Carter Morgan (00:35)
Well, thanks for joining us, everyone. We have a special episode for you today. This is Lawrence Kesselut. He wrote a fantastic short story called Coding Machines. We have an episode on it. We'll link to it below. But for those who haven't read it, it is basically a short story about machines that learn how to program themselves and kind of talks about the machine uprising. We read it in conjunction with Ken Thompson's speech, Reflections on Trusting Trust.
And kind of the whole theme was this idea of like, we now live in a world where machines are increasingly writing more and more of our code via LLMs. And so we had a great discussion on the episode about that. We actually even talked to Carl Brown from Internet of Bugs about that. And then we were fortunate enough for Lawrence Kessler, the actual author of the story, to come on and talk about exactly that. Fantastic discussion with him. Nathan, you want to give them a sneak peek of what they're about to hear?
Nathan Toups (01:28)
Yeah, this one was super conversational. We just kind of went all over the map of just whatever was interesting to us. Spent a lot of time talking about AI, actually our own personal workflows and puzzling things or surprises, things that didn't surprise us. Just kind of, I don't know, it was a fun conversation.
Carter Morgan (01:47)
Very thoughtful guy. He had some questions, some ways of thinking about how LLMs might change the world, which I had never thought of it that way before. I'm like, you know what? I need to think about that more in my life. So stick around for the whole thing. Fantastic interview. This is Lawrence Kessalut, as he reflects on his short story, Coding Machines.
Nathan Toups (01:56)
Mm-hmm.
Carter Morgan (02:10)
Thanks for joining us, Laurence. It's such a pleasure to have you here today.
Lawrence Kesteloot (02:13)
It's great to be here.
Carter Morgan (02:14)
Well, give us a, we, well one, we really, really enjoyed your short story and Nathan, correct me if I'm wrong, but we've read essays in the past, but I don't think we've done a short story before, have we?
Nathan Toups (02:26)
No, and this was actually a recommendation from one of our listeners. They were like, you should read this story and the original Ken Thompson, the transcript of his speech. And it was a great episode. It was a lot of fun.
Carter Morgan (02:41)
Yeah, we've done very little fiction on the podcast. We've done basically this, the Unicorn Project, and then we did read Project Hail Mary, because I love Project Hail Mary. We read that when the movie came out. So yeah, very little fiction. So I guess talk to us just about that. mean, the story, when was it written?
Lawrence Kesteloot (03:00)
2009
Carter Morgan (03:02)
2009, okay, so maybe talk to us a bit about what motivated you to write the story. Was there anything in the industry at the time that kind of motivated it or was it just kind of, you you wanted to write a short story?
Lawrence Kesteloot (03:14)
Yeah, there were two things. There's a, do you know about NaNoWriMo? Have you heard of that? Okay. So it's, it's, it's short for National Novel Writing Month and it's November. And the idea is that there's a lot of people who want to write a novel. They can't get started and they get all tripped up with their ideas and stuff. So you sign up for NaNoWriMo. It's this website you go to and you're supposed to write 50,000 words in November. It's 50,000 is on the short side for a novel. It's like the size of, Mice and Men or something.
Carter Morgan (03:18)
I don't know, no.
Nathan Toups (03:19)
No.
Carter Morgan (03:25)
cool.
Lawrence Kesteloot (03:43)
And the idea is like, it's so many words, it's like thousands of words a day that you can't edit and you can't stop and you can't just get like writer's block. Like they tell you, like if you're writing, just if you get stuck, just a ninja comes in from the ceiling and solves your problem and you move on. then there's another month, in April, where you go back and you edit all that stuff away. And the idea is just to have a first draft. And once you have a first draft, editing it and polishing it is a lot easier than writing it if you're just stuck all the time. So my wife had done this a couple of times.
Carter Morgan (04:00)
Hahaha.
Lawrence Kesteloot (04:12)
And so she had a couple of short novels under her. I wanted to write one. And I wanted to do something, and I couldn't think of anything. And a friend of mine, Drew Ulbrich, had this sort of idea in his head. And all he had was this idea of there are a bunch of programmers, two or three programmers, who are sitting around, and they're thinking about the machine uprising or something. He's a little bit obsessed with the machine uprising, or he was even back then. And he had this scene in his head of they're wondering when it's going to happen in the future.
and they realized it's already happened. And that was it. And that was the entire premise of what he had in his head. And then when he told me that, like the whole rest of the story just like instantly just appeared in my head. And it was around November. So then I thought it was going to turn into a novel, but turns out there's nowhere near enough material. So in the end, it ended up being like 10,000 words or something like that, which was great. So it was a good opportunity to do it. And because it was shorter, I could actually polish it and publish it. And it was entirely based on that one little premise.
Carter Morgan (04:44)
Interesting.
Nathan Toups (04:52)
That's so cool.
Carter Morgan (04:58)
you
Nathan Toups (05:07)
I feel like when I talk to people and they find out that I've also read this story, it's like being in a secret club. It's like, oh yeah, you've read Coding Machines. I can also imagine that whoever was working on Stuxnet, which we all, guess the rest of the world found out like 18 months later after this book, was like, how did this guy figure out that we're working on this project? I mean, it's not exactly the same, but yeah, it's like out of control coding.
Carter Morgan (05:16)
You
I
Nathan Toups (05:37)
that ends up infecting a bunch of stuff, right?
Carter Morgan (05:41)
Embarrassingly, I did not know it was fiction until like the last 20%. And I was like, and then when I did, I know, right? I just thought we were reading like a blog post or something. And so like, then when it got to the, like the very end, was like, I'm like, I feel like if this had actually happened, I would have heard about this. And so then, but it was, but I think obviously like that's embarrassing that I know it was fiction. And then I was like, of course, once I found out.
Lawrence Kesteloot (05:46)
What?
Carter Morgan (06:08)
But I think one of the reasons that I didn't quite grock it as fiction right away is that it's obviously written by a software engineer, right? And like, it's very like day in the life of what it might be to work. Like at one point you describe like them all going out to get lunch and like what they're eating. And so to me, like, yeah, I mean, I don't know, it it read as very natural and it was gripping. And I mean, you say you have this friend who is kind of...
interested or concerned about the machine uprising. Is that something you personally are concerned about as well or just you thought it was an interesting premise?
Lawrence Kesteloot (06:47)
At the time I wasn't, and then five years later I started getting kind of doomy and now I'm not doomy anymore. But I'm in the Bay Area, right? So this is San Francisco Bay Area. So this is like, I'm surrounded by doomers. It's like, I think I know maybe two people who are not doomers. So it's pretty tough to sort of maintain the good attitude, but I'm generally not too worried about it.
Carter Morgan (06:54)
Okay.
Yeah. Yeah. Yeah.
Now that's interesting you say you're not worried because I feel like if anything the dooming is only increased lately with you know large language models. Why do you remain an optimist that there will be no machine uprising?
Lawrence Kesteloot (07:20)
Yeah, think the honest answer is that I just don't want to believe it. It just makes me happier to not believe it. But I also have this almost unbounded optimism in humanity's ability to be creative under duress. And we see that in wars and other things like that, like really horrible things happen and suddenly ordinary people just come up with incredible ideas. And so I'm just optimistic that we'll go through bumps that are hard, but I don't think we're going to go through this sort of like explosive, you know.
Carter Morgan (07:25)
Hahaha
Yeah.
Lawrence Kesteloot (07:50)
everybody dies sort of scenario. It doesn't feel plausible to me. And I think also when I compare it to just previous moral panics, like just like global warming or before that in the 70s when we had all the bio stuff and the peak oil and the food shortages and nuclear and there's especially when you're in the middle of all, if all your 50 closest friends are all just like maximally panicked about this, it's easy to sort of get wrapped up, but then you can sort of step back and
Carter Morgan (07:52)
That's it.
Lawrence Kesteloot (08:19)
say, okay, well, 10 years later, it doesn't feel like it was quite so as bad as that. And I think like, it's important for me that to shape match on that to be like, okay, like probably in 2005, if you were in the middle of the very worst of global warming, it really felt like, okay, everybody's saying, okay, it's a positive feedback loop. It's already started. We're all dead. Like everybody's dead. People were saying this right in 2005. And then 20 years later, we're like, no, that didn't happen. And so it's important to sort of shape match to them and be like, okay, I'm hearing the same stuff.
Carter Morgan (08:35)
Rev.
Lawrence Kesteloot (08:48)
And the arguments are different. And they're saying, OK, but this time it's different. But they also said that in 2005. So I just sort of like shape-matched that. I'm like, OK, this is probably another one of those. And it'll be bad in a lot of ways. And then we'll deal, like we do with a of stuff.
Carter Morgan (09:03)
I saw this interesting tweet where someone was basically saying, like, one of the funny things about getting older is when younger people will, like, almost try to fact check you about things you lived through, right? Like someone, like, they're at a trivia night. They're like, are you sure that the PlayStation 2 came out before the Wii? He's like, yes, I am 100 % sure, right? And I think that's a...
something very valuable in the software engineering industry is as you age and have been through those before, you're talking about this and kind of overall moral panics. And I agree, I think that framing is really useful too. But we had on, we're buddies with Carl Brown. He has an internet, a YouTube channel called Internet of Bugs. And he's older, he's like in his 50s. And he started the channel basically because like all of his daughter's friends in college were like,
nervous about large language models and kind of like how it was changing the industry. And so he just wanted to make some content to disseminate that. what he kind of has told us and has told us channels just like, this is not the first upheaval the tech industry has gone through. Right. And I was kind of saying like, well, I'm nervous about like a junior programmers and like, you know, the tax I was doing as a junior 10 years ago, like Claude can one shot that it now. And he was telling me, he's like, but Carter, the tasks I did as a junior in the nineties, like you could have solved in
10 minutes with Google and Stack Overflow and modern IDEs. He's like, it's a constantly changing industry. And if it's the kind of industry where you just get spooked anytime something new comes along, like, yeah, you know, it's not so much for you.
Lawrence Kesteloot (10:40)
Yeah, it's the same you can imagine when the Fortran compilers came out, right? And you were like, okay, it would take me three or four days to get this like numerical thing. And now it's like three lines of Fortran. What's the junior program we're going to learn? And every time somebody had a moral panic like that, actually like worked out. Now everybody says LLMs are different because they're actually like significantly stronger and it's possible. And I can't tell if that means it's going to be significantly better and there's going to be significantly more demand satisfied.
Carter Morgan (10:44)
I know, right?
Yeah.
Lawrence Kesteloot (11:08)
Because there's this like nearly infinite demand for software.
Carter Morgan (11:08)
Yeah.
Well, that's what I always say. I'm like, go back in to the nineties and explain Shopify to a developer back then. And like, you'd think the apocalypse had happened. You're like, you're telling me anyone anywhere can stand up an e-commerce site and like, and it works. But like, that's not what happened. Right. We moved. It's like you said, there's like a nearly infinite demand.
Nathan Toups (11:11)
Yeah.
Lawrence Kesteloot (11:27)
Yeah.
Nathan Toups (11:30)
I also, I sense like there's a strong like fellow contrarian here. It's like, too many people are feeling like it's doom time. Maybe I should take this sort of like counter stance. Cause I do this. I kind of be like, well, is that how I believe or should I be worried about it? And try to, that's how I like to think through problems. Not to just take the opposite opinion, but maybe the opposite opinion might be worthwhile thinking about.
One of the other, was actually doing some research before we got on the podcast with you and I'm a big Go programmer. And one of my favorite things to do is to read critiques of Go programming. And I actually didn't realize you wrote one of my favorite ones that came out in like 2016, I think of, and so, and again, like it just kind of put me in the right mindset of like, oh, this is...
Carter Morgan (12:11)
you
Nathan Toups (12:23)
this is exactly what you need to hear. As a Go programmer, I'm like, yeah, I can see why that would drive somebody crazy. I still like it for these other reasons, but like, yeah, this is not for everybody. That makes total sense. Yeah, yeah, yeah. Yeah, the, yeah, there's generics now, so you don't have to use the, yeah.
Lawrence Kesteloot (12:37)
know they fixed a bunch of that stuff too, so some of those things wouldn't apply to me.
Yeah, the interface, yeah, brace brace.
Nathan Toups (12:53)
What are there any other short stories or any of the other things that you've been kind of having in the back of your mind? were you surprised by the popularity of coding machines when it came out?
Lawrence Kesteloot (13:05)
Yeah,
for sure. Yeah, I tried to write another short story another year later and that went nowhere. they say, like, you know, you have your whole life to write your first book and then like two years to write your second book or something. it's like I felt it for sure here. Like I've heard so many things that I'd accumulated over the decades into this one story. And afterwards I had just nothing left. And it's fine. I don't I wanted to write a story that I would have liked to read. And you said earlier that it's very technical and goes into some of the details. And there's certain ways that I feel about software and like I
Carter Morgan (13:10)
Ha
Yeah, yeah.
Lawrence Kesteloot (13:34)
I write something in there about how lovely it is to write in assembly language. There's something just really interesting about actually knowing every little thing. And that was something somewhere in my head and I stuck it in there. It's like a lot of the ways that I think about software and developing software and managing software projects and stuff like that and how things are done that I just put in there. I think I just want to write a story that I want to hear and then that I want to read. And I don't think I have another one in me. I'm happy.
Carter Morgan (13:59)
Hahaha.
Nathan Toups (14:00)
Speaking of
programming, what's your language of choice these days? What are you working on?
Lawrence Kesteloot (14:07)
It's pretty much still Java. There's another funny blog post called Java for Everything that I have that shows up on Hacker News every couple years. And it's just a really good language. And it's just slowly transitioning to Kotlin. it's good on so many levels. It's just a really good language.
Nathan Toups (14:09)
Cool.
Yeah. That's
cool. That's cool. I have a buddy who's a, he went through a phase he's big into rest now, but he was, went through like a closure phase, but he was also sysadmin and he just, anything that was in the JVM world, he just would unlock. So it's like, he could switch between Java and Scala and closure and it didn't matter because they had the JVM under the hood. That was pretty neat.
Lawrence Kesteloot (14:33)
Mm-hmm.
Yep.
Carter Morgan (14:48)
We at Worker
Lawrence Kesteloot (14:48)
Yep.
Carter Morgan (14:50)
recently had to do kind of a big migration because long story short, like lot of tech deck caught up with us and our database. We basically were using MongoDB as like a relational database. then not using, yes, yes. Exactly, right. And you're doing like all these joins in memory and like one API call is making like 18 requests to the database. And so we got to the point where we're like, you know what? Like, what if we just did like a rewrite and.
Lawrence Kesteloot (15:02)
I have done that. Do not do that.
Nathan Toups (15:05)
you
Lawrence Kesteloot (15:08)
Yes. Yes. Yes.
Carter Morgan (15:17)
And we decided, like we went from Java to TypeScript because we all like TypeScript. We all have TypeScript, but throughout the whole time I was telling people, like, Java is not the issue here. We could do just fine on Java, right? It was like, we had some architectural, especially data design decisions. What was so funny about it at the end of the day, it was like, it was actually a pretty darn good relational schema. Like, you know, cause trying to take a non-relational schema and then like define it as relational, like we actually got it done very, very cleanly. So like it was pretty well designed. was just all.
Lawrence Kesteloot (15:21)
TextScript is super great.
Carter Morgan (15:46)
shoved into Mongo, right? Yeah, except basically, right?
Lawrence Kesteloot (15:48)
Yeah, you just re-implemented a lot of Postgres or whatever yourself. It's like you made an
Nathan Toups (15:51)
right
Lawrence Kesteloot (15:53)
ad hoc, slower, less well-tested version of Postgres. Speaking of migrations, just last week I took a 9,000 line C++ library that an intern had written. And I just had Cloud Code re-write it in Java. And it one-shotted it. It took an hour and a half and cost $36. it worked. And including, we had like 1,000.
Carter Morgan (15:55)
Yeah, that's what we, that's what we wound up doing.
Yeah. Yeah.
Yeah.
Lawrence Kesteloot (16:17)
regression tests and it passed them all and it's great.
Carter Morgan (16:20)
Well, and that's what we were doing with ours because we were just like, for I had insisted for a while, like, isn't it? What's the famous blog post? who wrote it? But it's like things that never do part one. And the whole article is just like, is it Joel Spolsky on software? Spolsky. Yeah, Joel Spolsky. Yeah. Yes, yes. He's talking about rewriting Netscape. He's like things that never do. And we had kind of floated around the idea of like a rewrite. And I was just like, I don't know, guys, like that is.
Lawrence Kesteloot (16:36)
Spolsky. Yeah, he's talking about rewriting Netscape or something, didn't he? I think that's what he's... Yeah, yeah, yeah, yeah.
Nathan Toups (16:37)
Yeah, sounds right.
Lawrence Kesteloot (16:49)
Mm-hmm.
Carter Morgan (16:49)
That's tough.
Lawrence Kesteloot (16:51)
Yeah.
Carter Morgan (16:51)
and so only once we kind of got like a pattern around it where we're like, okay, we can rewrite it in chunks. We don't have to quit feature development entirely. but then it was Claude. were like, now's the time we got to see if Claude can do it. And so the original code base wasn't as good at like having like a bunch of tech, like regression tests set up. And so it hasn't been able to like one shot it. I think there's some more complicated business logic in there. And so, but you know, in six weeks we got it.
almost all the way done. And so I keep talking about like, I don't know if you guys remember, but like, Oh, it must've like eight years ago or so, like the company movie pass came out where like their whole thing, was like subscriptions to the movies. Like you paid movie pass. Like it was something ridiculous, like $15 a month. And then you could go to the movies as much as you wanted, but movie pass was not owned by the chains. And so movie passes business models literally, they would just mail you a debit card and like you would just use it to buy.
movie tickets. so like, they were just losing tons and tons of money. And I think they had like grand ambitions that like, eventually advertise or whatever. It was like a really, really bad business idea. But people still talk about like, Hey, do you remember movie pass? Like how many movie pass or movies did you see when you had movie pass? And like, there's a part of me that wonders right now in the golden age of like LLM costs when they're so subsidized, like, will we look back on these days? Maybe, you know, just the marginal cost of inference will get a lot cheaper. But if there is a future where there are more expensive
What will we look back on in like 2025 and 2026 and be like, Hey, what did you do when, you know, everything was so subsidized? And I feel proud that our answer will be like, yeah, you know, we, we wrote the whole code base. Like that feels like a good use of subsidized tokens.
Lawrence Kesteloot (18:29)
Do they subsidize?
Like when I paid $36 for an hour and a half, I don't, it was like over an hour of thinking. I don't know how subsidized that is. It might be.
Carter Morgan (18:35)
Yeah, I don't know. I
know that the Claude code like plans, like if you get like the $200 plan, I know that's crazy subsidized, but as far as like the API cost, I genuinely have no idea. Yeah, if it's a...
Lawrence Kesteloot (18:42)
Oh, yeah. Okay. Yeah, the per token. I don't know either. And I
mean, they could be 10 times more expensive and I would be thrilled to pay, right? This is like totally fine to me. Like to rewrite 9,000 lines of C++ into Java, like I'll pay $360 for this. It's fine.
Carter Morgan (18:52)
Right,
Yeah. Yeah.
Well, I want to hear more about that from you because I feel like with AI and large language models, we get a lot of people who are really excited about them who are younger. And then I also feel like that a lot of the more skeptical voices, out of all the ones I've at least heard, tend to be a little more older, a little more established. And I feel like I
I know they're out there, but I don't hear as much. sounds like you're pretty bullish on them. You're really enjoying working with them. mean, you know, as someone who's obviously a very accomplished software professional, mean, talk to us a little bit about, you know, like, how has your workflow changed since using them? And I guess why, why are you enjoying them so much?
Lawrence Kesteloot (19:41)
Yeah, I don't use them nearly as much as some of my friends. Like I have a friend of mine who has a newborn and he will literally just have the newborn and the thing and he's driving and he's literally using like cloud code on his phone while driving and instructing it to like check out this bug one, two, three and implement it and tell me how you're doing. And then he's got another session. He's doing all this on the voice. So I'm not there, but I use it. Okay. I use a little bit. I've noticed two things. One is it writes code a little bit like a junior programmer, which is solvable. Like what's an example? Like.
Carter Morgan (19:44)
Okay.
Yeah
my gosh.
Hahaha.
Lawrence Kesteloot (20:12)
Like I just reviewed an intern's code this morning and she was like duplicating some strings, right? It's like you're sticking in a constant and then cloud code will do that too. They'll just duplicate a bunch of strings. So that's solvable maybe with a prompt or maybe just like as the models improve, they'll just do less junior level stuff. Another example of that is like the comments tend to be kind of juvenile or like they'll describe the code. Like the code will be like, you know, equals B, C equals D and then the comment will be like.
Carter Morgan (20:31)
Yeah.
Lawrence Kesteloot (20:37)
Assign B to A and D to A. That's not what the comment should say. It should say, like, back up the user data or something, right? So it's doing junior stuff like that. That's fine. That's solvable. That's going to get better. What I notice as a user is that I implemented OAuth the other day. I had it just implemented OAuth from scratch. Because I don't want to read through the OAuth spec and figure all that stuff out. It can do that. But what I notice is that my relationship to that module is very different from the relationship to every other module in my code. I don't understand it. don't really, despite
Carter Morgan (20:38)
Yeah.
Yeah.
Right, right.
Lawrence Kesteloot (21:07)
code reviewed it, I still don't, I didn't internalize it. It's like reading code versus writing it. It's just very different. And when I went to make another decision later about it, I had a hard time making a decision about it because I hadn't really understood it. And so like at a, at at a senior, as a, as a code lead, as a programming lead, I had a hard time making a decision that I normally would have no problem with. And that's just one module. And I can imagine if I had like 40 modules that I never re didn't really understand how they worked or how they connected despite having done the code reviews, I would feel bad about it. I would feel very insecure.
Carter Morgan (21:18)
Right.
Lawrence Kesteloot (21:37)
And I don't know how, do I just let go of that eventually? Is it like assembly language where people used to review the assembly language that the Fortran programmer made and the compiler made and at some point you just let go of that and you're like, I don't care, it's fine. And I don't understand what it's doing under the hood and I don't have to because I can just prompted some more to do another thing. It felt, I felt disconnected from the code in a way that felt bad to me. And I don't know how to solve that because it did save me a bunch of days. But ultimately is it a long-term good strategy where two years from now I'm going to look back and not really understand.
Carter Morgan (21:58)
Right.
Lawrence Kesteloot (22:08)
product. It felt weird. It was a weird feeling to not be connected that way.
Nathan Toups (22:09)
Yeah.
Carter Morgan (22:09)
Right.
Nathan Toups (22:15)
I've been running into that with, I have some like little personal projects. I've been using Claude more for projects that I've like started working on and then stopped and kind of put on the shelf for a long time. And then I've come back and go, maybe it can help me get through this like sticky point. And I've had the same sort of feeling where like, you know, I'm really glad I fixed this UDP server that I was writing from scratch. Like it fixed some weird problem that I was working on, but I also did not feel as connected to it. And while I did spend time and I was like,
There was a couple parts of it where I was like, okay, that's where it got stuck. But I've also had this situation where...
I've allowed, I'm pretty careful about not vibe coding too much, but if I do vibe code on certain areas, I'm like, I do, come back and I feel like in the moment that it was actually pretty cool. And then I come back and have this like sober response. like, this is terrible. This is really not good. I've like slept on this and any bug that I'll have, it's like the architecture of it. like, I'll like, well, it's the, are you familiar with the gall man? It's like,
Lawrence Kesteloot (23:13)
mean, it's not good, like the code is not good or the, what's not good about it.
Nathan Toups (23:24)
I think it's the Gallman, not theorem, but Gallman. sorry, sorry, Gelman, Gelman amnesia. Thank you. Yeah, I will butcher some words. yeah, Gelman amnesia, It's, you know, again, for the folks listening in. If you ever had a newspaper article or something written about you, you'll find factual errors like pretty quickly and everybody makes a little errors. Most of them are not that big of a deal, but you notice. But in general, you read the newspaper for...
Lawrence Kesteloot (23:27)
It's called Gelman amnesia. Gelman amnesia, yeah.
Nathan Toups (23:52)
source of news and you're like, okay, there's all this great stuff in here and you don't really scrutinize it. You're like, this is where the truthful reporting is in place. And I think that this is the same thing with like scrutiny in our code. Like if it's some go server backend piece, I have a lot of scrutiny that I can very easily put in it. But if it's some TypeScript, I'm not going to scrutinize it quite as well. And if it's like some next JS, know, middleware thing, I might accept what it
Lawrence Kesteloot (23:56)
Mm-hmm.
Nathan Toups (24:22)
gives me and then I realized later after reading some documents, I'm like, wait a minute, that's like the old way of doing it. And there's this way better tool and it's solved all these weird things that I've run into and I should have used that. And then I have to go back and like, re-implement it with this new insight that I have. And of course, if I had read the documents, like I always did with every other language I've ever been in, I would have just avoided that categorically, I guess.
Carter Morgan (24:49)
You know, I had that happen. well.
Lawrence Kesteloot (24:49)
say that because I I was gonna say two weeks ago
I just tweeted like, uh, gal man amnesia for AI. I just tweeted that. It was like, same idea. was like, it was like, ask, I think somebody else had tweeted something like, every once in a while, ask Claude something you know a lot about just so you realize that it's, it's, it's kind of off about a lot of stuff. It doesn't, it doesn't get a lot of the nuance.
Carter Morgan (25:04)
Yes.
Yes.
Lawrence Kesteloot (25:10)
it's sort of getting some median thing off the internet, which isn't exactly where the correctness is. So everyone's, and we'll just calibrate on cloud by just asking it something you really know about, which is retweeted that by saying, come in and.
Nathan Toups (25:16)
Yes.
Yeah.
Carter Morgan (25:20)
Hahaha
Nathan Toups (25:21)
And I think the opposite of that is, and I've said this before, it's, so there's the gel man amnesia. And then there is the Dunning-Kruger powered tech debt machine, which is like the unstoppable person who doesn't care. Like they throw all, all, you know, caution to the wind. And it's just the Dunning-Kruger of not knowing how bad you are at something and just developing, you know, a big ball of mud at a rate faster than humanly possible before.
Carter Morgan (25:48)
Well, you know, I had this happen to me where I was where we're doing we're doing a basically right like where no one was going on. Okay. Yeah. So we're I'm building out like a notifications, just a simple notification system in the app. Right. I spend a lot of time because I'm more back end than front end. Right. That's kind of how I lean. And so I spend a lot of time.
Nathan Toups (25:54)
Your little miniature psychosis.
Carter Morgan (26:13)
really pouring over the data structure for the notifications, because I wanted to be extensible enough to support all these different kinds of notifications we're going to do, but at the same time, we needed to be fast on query. And so I'm spending a lot of time. And Claude keeps coming back with API and schema designs, like, no, no, almost, almost. And then I lock in, like, OK, that's it. Let's do it. But I had to guide Claude through it a lot. So then I'm back on the front end. And with our home page,
I need things like, for example, like if you like a post and then click into it and then go back, I want your heart to still be there without doing a fresh query. Right. and so I'm asking Claude, like, well, how do I do that? And it writes like this custom react provider component and like whips it up. And then I tested it it's working and I just felt so bad. I was just like, my gosh, like I never in a million years would I have been able to write this kind of custom provider component.
Claude is such a good engineer. I submitted it for PR and then the engineer I work with who is more front end focused, he's like, why on earth did you write a custom provider for this? He's like, we should be using something like, it was like some library that I wasn't familiar with. And so it's just so funny because I saw that exactly. Like something I knew pretty well, I knew kind of how to coax and guide Claude. And then the moment I passed it over to something that I didn't know as well, I'm just like, my gosh, this guy's a genius and I can't do anything.
Lawrence Kesteloot (27:37)
weird thing here is that Stack Overflow is becoming less and less useful for answering these kinds of questions. And the other day I needed to do something in Objective-C, I can't remember, it was like I needed to figure out how to write protected, how to have a protected field in the class that's only visible to itself and to subclasses. And I did a search on Google three or four different ways and the front page did not help at all. And finally I just went to Cloud and it just immediately told me. And it's just weird that like, I just more and more just want to go straight to Cloud to ask questions. It's just a weird shift that happened. felt
Carter Morgan (27:41)
It is weird, right?
Yeah.
Lawrence Kesteloot (28:07)
Very quickly.
Nathan Toups (28:09)
Yeah. Yeah, I would love to see the statistics and like, you know, Google's back end of what types of questions, like I would imagine the shape of question that people bring to Google has shifted because there's a whole class of question that I don't go to Google immediately over anymore. I think it was a while back. I actually got like one foot in like the pin test community world.
Carter Morgan (28:22)
Right.
Nathan Toups (28:35)
just because my background is a systems administrator, so I'm always just thinking about creative ways that people might try to use our systems. And the first time I'd ever really paid attention to using these chat tools as a way of searching through knowledge was that this pen tester was really trying to get a better understanding of like this... Oh, sorry, Pen tester is a penetration... It's like an ethical hacker, right? The folks who are trying to make sure your systems are...
Lawrence Kesteloot (28:56)
What's up, what's up, Pentaster?
That's just okay. Okay.
Nathan Toups (29:05)
safe and will help you scrutinize that. And he was looking at the C implementation of this TCP stack. And it was just way faster for him to ask ChadGPT or Claude to get this thing. And then he asked it for sources. And of course, used it to kind of skip right to where he needed in this very large technical document that it came from. he didn't just accept the answer.
But to Google that, we've been really, you had to do some digging, right? To get into some deep technical documentation. And that was like where it clicked, where I was like, this is like a yes and, where I can ask it this very particular kind of obscure thing. And if you think of it as this amazing indexer across human knowledge, if you're asking that category of questions, it's incredible. Like a next token guesser is really good if you're asking it something that's within a bounded.
domain of knowledge.
we have. Yeah.
Carter Morgan (30:07)
It's funny
to me how often I like ask a question is like, and I'm as I'm typing it out, I'm like, how much do I care if this answer is 100 % correct? And then I'm like, not a ton, right? I just need the general shape, right? but I have noticed like it's, it's a muscle. You kind of got to keep strong because I find myself going to Google less and less. And there are some things where I'm like, you know, I should, I should know this. Like I was curious the other day, I was asking it about like,
And it's tough because you need to kind of discern. It can be great for kind of fuzzier questions. If I want to know what year Abraham Lincoln was born, I'll just go to Google. Ironically, Claude would probably know the answer to that. But then I was curious about the differences between the American and French revolutions and what ideas kind of, I'd heard a claim growing up in America, obviously you think the American Revolution is most significant.
but I had heard a claim that the French Revolution is actually considered globally more significant. I was like, oh, I never heard that before. And so I wanted to know, OK, what ideas came out of the French Revolution that are considered more globally significant than the American ideals? And it gave me an answer, and it was interesting. But then I was like, I don't know if any of that's true. Exactly.
Lawrence Kesteloot (31:23)
Yeah, right. Yeah,
exactly. Yeah, it can get stuck. It can get stuck like it will start some idea and it'll just feel like it has to flush it out even though it may not actually be true. There's a good book if you're interested in that topic, The Anatomy Revolution, which is, yeah, it compares a bunch of revolutions. I don't remember all of them. read it in high school, but it's like American and French, I think. I don't remember, Russian probably.
Carter Morgan (31:32)
Yeah.
that sounds great.
that sounds fantastic. I'd love to read that.
Lawrence Kesteloot (31:48)
Yeah,
I think maybe five different revolutions and it tries to see like what's in common, they're different, stuff like that. I remember liking it.
Carter Morgan (31:51)
Okay
Okay, awesome. I've been on a bit of a history kick lately. I try to at lunch, like, you know, I'm usually watching a video or something and like on YouTube and I was like, you know what, I bought the movie Lincoln by Steven Spielberg like a couple of days ago. I'm like, I'm to watch this at lunch. I'm going to watch this in 15 minute chunks. ever since watching that I've been like on a bit of a history kick. So I'll have to pick that book up. It sounds really interesting.
Lawrence Kesteloot (32:18)
Do you think that software is going to get like crap here? There's a lot of like guessing about this where like the other day I asked Claude to write some code for me and it was a race condition inside. It wasn't very serious, but it was a race. And so I talked about it and it immediately understood it. Look, found another problem I hadn't even seen and then fixed it and it was all fine. But I had to point it out and I think a junior programmer wouldn't have seen it. so small races aren't a big deal, but when you have a thousand of them, they do eventually trigger. then so I can't tell if like,
Carter Morgan (32:21)
yeah.
Yeah, Brian.
Lawrence Kesteloot (32:48)
that kind of stuff will get through. So software is going to get worse and web search are going to be down more or whatever. Or if cloud is actually better than the average already. So the average programmer is already would have written that race and cloud is less likely to write the race than the average programmers. So programming. So like our websites are software going to get actually better. I know in the long term, they'll be fine. But in the next five years, I could, I could argue myself either way that like a bunch of slop is getting through.
Carter Morgan (32:56)
Yeah, right.
Nathan Toups (33:15)
I've pendulum swung on this. I think that both are probably true. I'll read these weird blog posts where someone's writing formal verification specs and then getting these to write Rust or assembly or something. And I'm like, that is fascinating and weird and cool. And I hope there's more projects that go that direction.
I listen to folks like Mitchell Hashimoto, who's been working on Ghosty in Zig. And he's got this, he was on pragmatic engineering recently and he's like talking about his relationship with agent AI and like what types of tasks he put on there. And I'm like, okay, I respect him. I respect his perspective. Actually what he's describing sounds delightful. Like it sounds like a wonderful way to write programming and he's still getting to like work on the fun, juicy parts.
Carter Morgan (33:45)
Right.
Nathan Toups (34:09)
And then I see the unhinged stuff where they're like, if you're not dealing, you know, if you're not like orchestrating 10 agents from your phone 24 seven, you're going to get left behind. I'm like,
Most of those folks, I've never seen the code that they're writing or the output of that code. And I'm really skeptical. I'm not saying that you can't. I would imagine there's probably prolific people who are doing amazing things. But I think there's also just like, it reminds me a lot of like the NFT bros and crypto like five years ago.
Carter Morgan (34:25)
Yeah.
My hope is that, no, you go ahead, you go ahead. I was just gonna say my hope is that it gets better in that I think the natural inclination is to get worse. I have long-term skepticism of the large language model, the transformer architecture at like a fundamental level in the sense that like, as I understand it, just because of the way the softmax equation works where all the tokens, every token must attend to the...
Lawrence Kesteloot (34:39)
Yeah, friend of mine, pardon, a guide party.
Carter Morgan (35:08)
every other token and then all of those about they're all given a weight which then sums to one. But because it has to sum to one, that just means by definition as your context window grows, every token can is less meaningful. And we already know that they have like, like what do they call it? The lost middle or the messy middle or a needle in the haystack problems, which is that like it's, it tends to remember stuff at, at the end of the context window really well. And at the beginning really well.
And it'll forget things in the middle. And it's not something you can solve, but just like, even if you grow the context window, like, it's a one trillion token context window. It's like, well, there's the chance that that 90 % of the middle is kind of functionally useless. And so if they always remain somewhat context bounded, then. You know, like I think they'll continually get better at coding in general, because we can just generate so much training data for them because.
code is independently verifiable, right? If you generate the code and it compiles, and I'm sure they have very sophisticated generation mechanisms to, know, at the same time, probably generating, you know, regression tests on that training data. Anyhow. But I don't know, just kind of this, they're stateless. They are stateless machines that feel no long-term pain about the decisions they're making. And so this idea, you know, this idea that we can just like keep, that we can just
completely let go and generate tons and tons of stuff. like, I don't know. I don't ever see that happening. What I do see and hopefully happens is, you know, just in a world where like, if code generation is faster and we can move faster, does that free us up to attend to all of the other kind of minor pain points that all software has? Like I've been talking about that. Like if everyone is zigging because they say, hey, let's move 10 times faster. like, should we zag a little?
Should we still move faster than we did before? But instead, should we be looking at all the other parts of our application and saying, hey, you know what? This is kind of a little quirk or bug that we've put up with for a year or two because we just haven't had time to fix it. But why? Why do we have any patience for that anymore? So I hope that's what we see software become. I hope with this kind of increased bandwidth, we instead really devote ourselves to higher quality. But yeah, a lot of the hype cycle right now is just
completely let go, like Nathan said, run 10 agents at once and, you know, Claude take the wheel.
Nathan Toups (37:39)
I would love to hear both of your pendulum swing sides of what you think, Lawrence.
Lawrence Kesteloot (37:45)
this particularly. The thing I see most, the thing I noticed most is not like software being written faster by engineers. It's software being written by non-engineers. And that's the things I've noticed amongst my friends. A friend of mine misses the days of OkCupid. Remember this dating site from 15 years ago? It supposed to be like the best dating site ever. It was bought out and kind of shut down. so he wants to remake OkCupid. And so he did. just is by quoting the entire thing. He's not looked at a single line.
Nathan Toups (38:01)
Yeah, yes.
Lawrence Kesteloot (38:12)
And it looks great and it's incredible what he's done in just a few weeks. You wouldn't even believe how many features it has. If he came to me and said, OK, how much should I pay you to do this? I'd be like, I don't know. This is like two years of work. It's insane. The config page is full of options and it's all dynamic and it all works really well. It's all animated and it looks gorgeous. Another friend of mine wanted to make a social network because she got tired of Facebook. So she just made her own social network on scratch. And she's not a programmer.
Carter Morgan (38:25)
Right, right.
Yeah.
Lawrence Kesteloot (38:41)
But if I look at those websites, you know they're down a lot. I think there's just a lot of like, ask Claude to make a change and you push right to production and then I just broke something. But I think the zero to one is the interesting one to me. Like the stuff that exists. We had our basement redone a couple years ago and the general contractor asked me at the end.
Carter Morgan (38:44)
Yeah.
Nathan Toups (38:51)
it.
Lawrence Kesteloot (39:03)
He has a computer science degree, actually. And he said, like, I need an app to coordinate my teams. have two teams. have the Asian team and the Hispanic team. And they can talk to each other. And I want to organize everything. Can I swap that for, like, let's say a bathroom remodel for you? I'm like, absolutely not. Like, are you kidding me? A bathroom model is like, what, $15,000? This would take me way more than that in labor to do. But now he could. He could probably just do it. And he could have his own. And I have my own app, right? My app is a grocery list app.
Carter Morgan (39:22)
Right. Yeah.
Nathan Toups (39:22)
Right.
Great.
Lawrence Kesteloot (39:29)
And people ask me questions like, could you add this feature? Could you add that feature? A lot of them is like fussy pantry management. Like I want to record when I put bananas and then record the date and then add a banana, remove banana. I'm like, I'm not adding that feature for you, but like just write your own grocery list app or just ask Claude to like keep track of your grocery list altogether. That's the other thing is like people are writing apps to do stuff. They're asking Claude to write apps to do stuff instead of just asking Claude to do stuff. It's like if...
Nathan Toups (39:49)
Yeah.
Carter Morgan (40:00)
Well, that was something Carl... go ahead, go ahead.
Lawrence Kesteloot (40:00)
If you
had a really smart, extremely cheap assistant who's generally competent, you wouldn't be like, write a grocery list app for me. You would be like, keep tracking my grocery list. So I think there's stuff you could just do more directly.
Carter Morgan (40:07)
Rhyme.
That was something Carl Brown pointed out when we interviewed him with again, worries about how is it gonna affect the job market? he's like, one thing people aren't talking about is that finding product market fit is very expensive and very hard for businesses. if you were kind of, previously, if you're a non-technical founder, you've got good business instinct, You either need to hope you can find a technical founder and hope that they're willing to generate a lot of code for you, right?
You're going to pay cheap contractors or something like that. He's like, you know, there might be a future where like, you have a lot of these non-technical founders who can kind of vibe code the, the MVP of how this should work. Right. You get it out, you find product market fit, you get some traction, you get some funding, right. Or, just generating revenue. And then you call in a software engineer at that point and say, okay, you know, now, let's make it real. which I think is a very interesting.
possibility. I do wonder how that kind of affects like for a while, you know, if you wanted like VC funding, like being a technical founder was very, very valuable, but just this idea that like, you don't need anyone to write your code for you. can just write your code yourself. I think that'll always still be useful, but I have been wondering, I'm like, is that, is that as useful anymore? Or if you're a non-technical founder, you probably don't feel the strong pull at the very beginning to go find your technical founder.
Maybe you're kind of willing to just go out alone until it gets some traction and then you can save some equity for yourself. I think that's probably what I'll have.
Lawrence Kesteloot (41:55)
for think it's been it's been
a meme forever. You find all these like business people who has they have an idea, right? And then they they're like, Oh, I can't find my technical founder, but this brilliant idea would be so successful if I only had a technical founder. And I think a lot of these guys are going to find out that, you know, just because you have an idea doesn't mean it's gonna like work. Just go ahead and find code and you'll probably find that like, man, nobody shows up. It's it's a lot harder to come up with an idea than people think. And but like before they never even got to find out that it was a bad idea.
Carter Morgan (41:59)
Yes. Yeah.
Nathan Toups (42:24)
Right. Yeah, it's been interesting because I even even with our podcast, like I was we had a kind of like a landing page. It wasn't much there. And I had some idea that I was actually got laid off. I was like starting up a new business. I had a little extra time on my hands. So I was like, oh, play with these. And we I wanted to build a little episode tracker where we could put transcripts and just had a very like a laundry list of stuff.
And I didn't want to conform this to like a CRM or a content management system or something. I didn't want to do that. I just like, I just need this. I just need these like few things. I knew the structure of it. I kind of in my mind had a very clear idea of the URLs. And I was, I mean, I am the audience for this is from a cloud code perspective. So I just built enough of it, put Postgres behind it and actually had something that we could like share with the audience on our Discord within a couple of weekends. And
It's been fun. It's just like, yes, I own it. Yes, if it breaks, it's like up to me. I don't get any support or help, right? I mean, like, yes, it's all the things, but it's also been really rewarding. Like I've actually really enjoyed thinking about what we need as a podcast. And it's been kind of fun along the way. So yeah, these little projects that are grounding, it is empowering actually to have like an idea and be like, can I actually execute on this? Which again is the hard part. Can I execute?
always.
Carter Morgan (43:50)
Yeah, I
mean, and that just kind of shows the shifting landscape because when we started the podcast about two years ago next month, I wanted a website for it, but I can operate somewhat confidently in a front end if an existing one, I just, do not have a, I can tell if a design is bad and I can tell if a design is good generally, but I cannot produce like, just, I can't create, do any sort of web design, right? And,
And so when I wanted to make the website, I went and bought like a $15 React template and then just kind of edited it to serve the website. then, you know, a year and a half later, Nathan, you just vibe code basically the whole new bookoverflow.io. Once again, it's like low stakes, right? It's just our little website.
Nathan Toups (44:36)
How
has it shifted your, like with your grocery list app, like what has it shifted in your business?
Lawrence Kesteloot (44:44)
Business-wise, not yet. So I'm kind of waiting for people to basically say, like, yeah, I'm using my own app that I've live-coded from scratch because none of the existing ones did the right thing. That hasn't happened yet. So nothing is showing up on my radar business-wise. Yeah.
Nathan Toups (44:59)
Okay, cool.
Are there any?
Lawrence Kesteloot (45:02)
I mean, our biggest
competition is actually just pen and paper, right? So it's like, that's actually what people are, that's what, we're not trying to bring people from other apps, we're trying to get people to stop using pen and paper, then like they write a whole list at home and they forget it when they go to the store. So that's the user I'm trying to get. So that's the...
Carter Morgan (45:05)
Yeah.
Is this your first time
Nathan Toups (45:19)
school.
Carter Morgan (45:22)
running your own company? That's really cool.
Lawrence Kesteloot (45:24)
Yeah.
Yeah, it was a side project a friend of mine did for the same year I was destroyed 2009. We were both new parents and we just wanted a shopping list app that synchronized across the family and you could add stuff and keep track of recipes and stuff. So we just wrote it and it was a side project for like four years and then went full time on it.
Carter Morgan (45:46)
That's fantastic. I am jealous. I want to do my own thing one day. Everyone always says like the idea is the hard part, or the idea is the easy part. Execution is the hard part. And we're like, man, I don't have any good ideas. I know, right? Yeah, I guess it's,
Lawrence Kesteloot (46:02)
It turns out the answer is both, right? Yeah, Sorry. Yeah, we got lucky.
Nathan Toups (46:04)
You
You
If you could go back and tell yourself in 2009 either something to persevere or something to abandon, what would you go back and tell yourself?
Lawrence Kesteloot (46:22)
think it would be fun to just drop all that stuff and just study AI, right? And just ride the wave as a programmer, as an individual. Yeah, I mean, yeah, like it would have been incredible. Like those things don't happen very often. And it would have been great to actually know what's going on and to just be there for all that.
Nathan Toups (46:29)
Yeah, yeah, yeah, yeah, yeah.
Yeah. Have you? I was just watching. I don't know if you've ever seen Welch Labs. He's like, and again, I'm probably just YouTube brained. so Welch Labs is a really good educator on math and some other stuff. But he's recently been doing some interesting interviews. And there's a guy, I'm going to probably butcher his name. It's in French, but it's Jan. I think it's Jan LeCun. He I think he was one of the original. He worked at Bell Labs. He was working on convolutional.
neural networks, and was big part of the visual AI stuff that was kind of the wave that happened right before Transformers and large language models came out. And he's actually taking like a billion dollar bet against large language models on something I think it's called JEPA. And it's using embeddings in a different way. And I'm just starting to wrap my head around it. But it's one of these things where it feels like a first glimpse of like,
someone who's been in this industry a long time has looked at the success of large-image models, but it's like, you know what? Tokens can't be everything. This is kind of his whole thing. These neural networks can actually do all of this stuff, and we've constrained it to token guessing. And what if we went back and took this old stuff and these really new ideas and took this new... And his argument is there's this paradigm shift that's about to happen. I know enough to be a fanboy from the side.
But I want to experience that. That's the part I'm like, yeah. I hope that we get to experience these things that are kind of like, we look back and go, that was adorable, large language models. Or that they're really good at this one domain and we overextended them. And really, when they're in this constraint, they're amazing. And they can run on my phone. Remember when data centers did it, and now it's on this. I think that's going to be cool to see in the next 10 years, probably.
Lawrence Kesteloot (48:19)
Yeah.
talking to friend of mine who's professionally he does interpretability, which is like trying to figure out like if we can look at the weights and just try to guess something from that. And he's really been disappointed at how little progress has been with LLMs. And he says like, from the user's perspective, they're getting better. But if you learned the theory like eight years ago, there's not much like you could catch up on the theory pretty quickly. And he's like, he was really disappointed. He's, you know, he's like a math PhD or whatever. And he's just like, looking at this stuff. He's like,
Carter Morgan (48:58)
Yes.
Nathan Toups (48:59)
Interesting, yeah.
Lawrence Kesteloot (49:07)
You could probably learn all that stuff. You could learn transformers, and that would take a while. And then you could catch up after transformers with all this nothing. So that surprised me. didn't realize it was actually like a lot of the work is at the very high level where they just take these things and they feed them back into each other. And there's not a lot of low-level AI progress.
Carter Morgan (49:15)
Yeah.
I've been trying to, yeah, that surprised me too, because we read and actually interviewed Stephen Wolf from his book, What is Chat GPT doing and Why Does It Work? Which Sam Altman actually endorsed and said, like, this is kind of the best high level overview of how the LLM architecture works. But it's a, yeah, but so we read that, it's like 150 pages. And I was like, okay, this is good. I feel like this was useful to really understand like.
Again, what's going on here? And then as like LLMs kept like advancing, I was like, well, are they inventing something new? Have they done something different? And like, the answer is no. Like they're, just, it's, it's training. And I've heard that a lot of the gains lately have come from reinforced learning. And that's actually one of my, I don't even know if it's a hot take at this point, but like when kind of, you know, it like back in February or so there was another big like, oh my gosh, everything's changing because I think that's when cloud code really started to gain steam and.
Uh, but, but I was like, but that's not necessarily like an LLM improvement. Like Claude code is a very sophisticated harness for an LLM, but like the model is still the same underneath. And there's a part of me that feels a little like, you know, and every technology goes through this, right? Like, you know, they've been the iPhone and for the first, you know, the first four or five, six iterations are all these big advances. And then guess what? The past 15 iPhones have all been functionally identical to each other. And I kind of felt that same way about Claude code and the agentic harness. I was like,
You can only invent the agent to harness once and then you can improve on top of that. like how much better does the harness gets? I don't know. And like we see things like, for example, with open code, like open code is just another harness and you can run that with, you know, deep seek or what is it? Quinn is a Q W E N right. And like, can get good results out of it. Not as good as like running it with Opus 4.7, but still good. So I don't know. It's like how much.
You know, most technologies fall like that sigmoid, right? Where it's a exponential growth up until some resource is consumed and then it kind of plateaus and it would shock me if LLMs don't follow the same trajectory that every technology humans have invented has ever followed. But I don't know, maybe it's the first time for everything, right?
Lawrence Kesteloot (51:43)
Yeah, sometimes I think back at mobile and like if in 2007 the iPhone came out, like what would you have absolutely not predicted? And I think it's something like Uber, right? It's like, I would have imagined like the desktop but mobile, but I would not have imagined is that it completely replaces taxis or something. And that's the kind of stuff that I'm waiting for for these LLMs. It's like, what am I absolutely not predicting? Because I'm just thinking in terms of like, I'm programming, okay, I'm programming faster. It's like overflow, but better.
Nathan Toups (52:02)
Okay.
Carter Morgan (52:02)
Right,
right.
Right.
Lawrence Kesteloot (52:10)
and autocomplete, even better autocomplete. And I'm just thinking more in terms of like more of what I have now. But in a couple of years, we're going to have something which is like utterly new, like Uber was, which is like that is is piecing together a bunch of things that happen all at once. And that's the thing that is.
Carter Morgan (52:11)
Yeah.
Yeah.
Airbnb is a good example,
too. Like, you know, that's really buoyed by the mobile revolution. And yeah, just this idea that like, you know, growing up, would you have thought like staying in some random stranger's home is just as viable an option as, you know, booking a hotel or, know, I remember even in college thinking like, why does only pizza and Chinese deliver? Right. And what if you could orchestrate some sort of third party delivery infrastructure, which then restaurants could contract out with you? And I
I like that framing, this idea of like, you know, it's like, yeah, you get a mobile phone and, and, you know, the first iPhone comes out and like, would you have been shocked to find out that YouTubing would become more of a legitimate profession after the release of the first iPhone? No, not really. Cause it kind of flows naturally that like, now that I can consume YouTube easier, I'll, you know, that might become a more lucrative industry, but yeah, would you have thought of Uber, Airbnb, DoorDash? Not really.
Nathan Toups (53:13)
All
Lawrence Kesteloot (53:24)
The GPS there was pretty key, right? Having a GPS that just reports your location and locked all sorts of stuff.
Carter Morgan (53:24)
Are you going to say? Yeah, yeah.
Nathan Toups (53:26)
Right.
And the comfort, the other thing is that some of it is changing social norms, right? Because if you told people in the 90s, you're going to have this tracking device in your pocket and it's going to know your location all the time. Like people would have, like that's something that would have been very unsettling. And you want me to pay for it and I'm going to, you know, like all of these things. But you kind of realize like the usefulness of it. And then you kind of just like whittles away at, not that uncomfortable or, I think that Apple's,
Carter Morgan (53:31)
Yeah, I love that.
Nathan Toups (54:00)
privacy stance is good enough or whatever. That is interesting.
Lawrence Kesteloot (54:04)
say that
about so many things like if I told you right now let's let's introduce a technology that's going to kill 30,000 Americans a year mostly 20 year olds you would have been like no but we have cars and we totally accept them and and they're just grandfathered in but we would never never allow them to come on the market right now but we're like okay well we don't want to take them away they're pretty useful so let's go ahead and kill 30,000 people
Nathan Toups (54:14)
Yeah.
Carter Morgan (54:16)
Yeah.
Nathan Toups (54:16)
Right.
Right. No, that's a...
I will say there's a few things that I think I've been in a few different industries, something that I'm fascinated with it. I don't think it's been solved yet. There's these algorithms called stable matching algorithms that are very interesting. And you can you can do like two and three party markets and they can actually prove that the best outcome for all parties are involved. And you have to play the game in a certain way. It's like a game theory thing.
You could imagine if you a way of easily describing your intent, like I would like to participate in the market this way. And that market could be anything like my kid gets a voucher for a choice school that we're trying to get to, right? It doesn't have to be like monetary stuff. So the stable matching and some sort of intent and agentic stuff. Like you can imagine a world upon which I can actually like...
Carter Morgan (55:23)
Bye.
Nathan Toups (55:24)
give something a narrow scope of advocating for my view that's going to go work on something 24 seven, kind of like looking for some way to do solve the puzzle and let the let these sort of like, you know, I guess they call them like thick markets or like some some way of like settling that you don't need a central broker on potentially. Like you have an algorithm that decides what a fair stable matches. I have my agent, which is my intention in it. And then we all participate. I think that like
there may be something there where I give it enough creativity or enough of its ability to kind of try to problem solve that operates within those constraints. And then it's not, it's not, I'm not asking it to do the algorithm in the best way. The matching algorithm is this like external thing. I don't know. Again, some of that seems like pretty futuristic stuff, but I feel like that could be some, that's what Uber kind of touched that, right? Which is this idea that I have.
drivers who want fares and I have me who wants to get from point A to point B and they made like a much nicer marketplace to make it super simple and know, payment settled and safety settled.
Carter Morgan (56:33)
But it's
interesting, like the changing social norms. I saw a clip of someone, like, they were kind of like videoing like how their mom uses chat GPT, but she was texting it like a person, like, hey, we just made it on the flight, like, pray for us. And chat GPT was like, absolutely. Let's say a prayer right now. And like, it started generating a prayer. And I was like, this is spooky, right? And like,
But there are even things like, I don't know, like personally, I am not like a short form vertical video guy. Like I don't have TikTok and I can't stand how often YouTube tries to foist it on me. But like, I've just kind of decided for myself and like, that's not something I really want to do. But obviously that has kind of massively consumed the world. I always say this with like parenting too, which is like anything that kind of existed when I was a kid, like I, cause I'm a pretty young dad. And so I'm not as
removed from when I was a kid. so like, can kind of grok it a little easier and be like, okay, I kind of remember how I dealt with this. And I feel like I can help my son navigate through this. It's the new stuff that's challenging where I'm like, this didn't really exist when I was a kid. So how do you navigate that? Like YouTube is a big one. We're like, we have a lockdown YouTube. have a couple white listed channels that he's allowed to view, but then like Netflix has started putting on more like actual YouTuber content. it's basically like,
Kid reality TV and like everything in my bones kind of says like, don't watch this. Like what is it? But then there's also part of me is like, I don't know. Is this just how the world is now? Like, and so like, I feel that strong with LLMs too, which is like, I see someone texting it, like it's a person and it generating a prayer and everything about me screams like, this is bad. This is wrong. We shouldn't do it. But then I'm like, can you even stop it? Right. And it's like a very meta, very meta similarity.
to your short story, right? Which is like, maybe it's already happening and maybe there's nothing you can do at this point. I don't know.
Lawrence Kesteloot (58:39)
I remember when I was a kid, I would see my parents getting behind technology. And I say, when I'm going to get older, I'm not going to do that. I'm going to keep up with technology. And what I find is not like it's hard to keep up with technology is that I don't understand why you would do it. Like, why would you do stuff? It's like when Twitter came out, everybody's like, I had eggs for breakfast. It's like, why? Like, don't, it's not like I don't understand Twitter. It's I don't understand why you would. And that's, I missed, I missed that. I think when I was a kid, it's like, it's not like it's hard to understand. It's like, just don't know. There's some motivation there that other people have. Like,
Carter Morgan (58:50)
Yeah
You
Lawrence Kesteloot (59:08)
talking to ChaiTPT about being nervous about getting on a flight that I'm not understanding the use case or something.
Carter Morgan (59:10)
Yeah.
Nathan Toups (59:14)
Ram.
Carter Morgan (59:15)
Yeah, it's funny. I was a missionary for my church and it was two years and during those two years, you don't, you're very focused on the mission. And so you don't like really watch, you don't watch movies or TV or anything. And, and you kind of, I remember a man in my life who had served a mission when he was younger and he was saying, he's like, you think you're going to like catch up on all that stuff when you get back. Right. And, and as a missionary, you're just making like this list of like all these movies you're going to see. He's like,
He's like, you never really catch up. He's like, you kind of just have a permanent two year gap in terms of pop culture. And like, I saw like what I most wanted to see, but like there are some movies that like I totally would have seen when they came out and I just never did and don't really have them. and I remember Snapchat had gone in big when I was a missionary and it came and I came back and someone was trying to explain it to me. I'm just like, the world has passed me by. Like, I don't get this. I'm not, I don't like kind of like you're saying, like, I don't understand the use case. Like, yeah, guess it's not for me.
Lawrence Kesteloot (1:00:10)
Yeah, I actually
almost brought up Snapchat as another example. that's another one.
Carter Morgan (1:00:12)
Yeah, yeah, right, right.
I remember a girl at college telling me she was like, she's you send a picture. like, well, what if I don't have a picture of anything to take a picture of? Like, almost people just take a picture of their shoes. I'm like, that's it. I don't get this. Like,
Nathan Toups (1:00:22)
Thank
Did
Lawrence Kesteloot (1:00:26)
That's actually not true with Snapchat, Was with the sexy pictures? think that's the whole point.
Carter Morgan (1:00:31)
Yeah, yeah, well, and that, that,
and then, but I wish someone had just told that to me. If they had just said like, yeah, this is for sending sexy pictures, I'd be like, well, I understand the point of this. Anyhow, we don't.
Nathan Toups (1:00:39)
I don't know if you saw
the recent post on OpenAI's blog about goblins. Did you see this? Yeah, so, but again, it got me, I was thinking about this actually in light of, we knew that this conversation was coming up. I was like, this almost reminds me of this like,
Carter Morgan (1:00:48)
Yeah, yeah, I developed like a goblin fixation.
Nathan Toups (1:01:04)
of trusting the compiler piece, which is that we're training on the data set. And you could imagine, just imagine a world, a Goblin's is a silly one, but imagine a world in which somebody kind of preempted that these training data sets are going to have things that I could sort of backdoor. I could give a nefarious instruction set that's told to hide itself so that it's not manifested normally. And you did this five years before everybody figured it out, right? You've given it some sort of,
you know, what is that called? it's not cryptography. It's the other one, steganography, right? Right, so I'm hiding messages in plain sight. And you can imagine like this steganographic message set, even generated from the large language model itself in its output. And then when it's on the internet, it's actually just like bootstrapping itself, even if it's been told not to do certain things.
Lawrence Kesteloot (1:01:41)
7.0 feet.
Nathan Toups (1:02:01)
The Goblin thing made me think of this. And of course, your story made me think of this. have you thought about this? Have you thought about what happens if large language models are given a back door?
Lawrence Kesteloot (1:02:12)
No, I did not, no.
There's also questions about biases, right? Like people worry whether Grok has a conservative bias or specifically, or has specifically been told stuff about South Africa and that sort of thing. But yeah, and I don't know if it's true and more and more people are going to be like, okay, you and I are having an argument about something. Okay, let's adjudicate it. We can't just go to Google, that's too complicated. So I'm just going to go to one of these agents and have it tell us which one of us is right and like what biases it have here.
Nathan Toups (1:02:16)
Right.
Carter Morgan (1:02:17)
Right, right.
Nathan Toups (1:02:20)
Right.
Carter Morgan (1:02:21)
Right, right.
Nathan Toups (1:02:25)
Right, right, right.
Carter Morgan (1:02:26)
Right,
right.
Nathan Toups (1:02:38)
Yeah, it goes back to like Noam Chomsky, you know, manufacturing consent stuff from the 90s. Like it's applicable all of a sudden.
Carter Morgan (1:02:39)
That's,
Right.
And that's been a concern with the internet at large, right? You know, these large social media platforms, like are they enforcing a bias and, but then you get down to like the model itself, right? Like it's one thing to like suppress viewpoints you disagree with. It's another thing to just encode in the all-knowing answer machine. The thing I believe is the correct thing, right? Well, we could talk for hours and.
And, you know, it's all, you I wish we could leave off on like a more positive note, I guess. And that's so funny. We start this with like, I'm an optimist. Like I'm very optimistic about the future. And then we just talk for an hour about all the ways that it could possibly go wrong. But I'm with you. Yeah. Yeah. I'm, I'm by nature an optimist.
Lawrence Kesteloot (1:03:28)
Yeah, I am still an optimist. Yeah, I'm still quite an optimist. Like they say, like I've heard a long time ago that
like a programmer is like a short-term pessimist and a long-term optimist. So you have to be short-term pessimist. It's like, you got to believe that there's a bug in this routine and just like carefully look for it and don't just be like, sure. It's fine. You got to be like pessimistic and really go for it. But you've got to be long-term optimistic because like it's too hard to do any other.
Carter Morgan (1:03:37)
Yeah.
Nathan Toups (1:03:45)
I love that. I love that.
Carter Morgan (1:03:46)
Yeah. Yeah.
Nathan Toups (1:03:49)
Right? Amazing.
Carter Morgan (1:03:50)
Unironically,
that's what I loved about Project Hail Mary. It's just Andy Weir takes a very like, and Ryan Gosling actually said this on the press tour. He said he's like, he thought it was a story that the future is not something to be scared of. It's just a problem to be figured out, right? And Andy Weir, can tell, is a very optimistic author too, and that like his belief in humanity, his ability to come together and to confront hard things. And so I agree. What do they say? Like it's better to be an optimist and...
proven wrong than a pessimist and proven right. I'm optimistic that everything will work out just fine. But Lawrence, it's such a pleasure to have you on. And again, we always love talking to any of our authors, but especially when we get to talk to some of our fiction authors, that's such a fun treat for us. Anything you'd recommend for our listeners to read? Technical, non-technical?
Lawrence Kesteloot (1:04:42)
Yeah, so the two books by David Deutsch are really good. I was reminded of that. You just said like the Andy Weir attitude of like, they're the only problems that you can just solve and just go solve them. That's fine. And I think that's, that's all the, his two books are dripping with, with that, especially the beginning of infinity is super good. And in terms of like topical stuff, I don't know if you guys know Scott Alexander. Do you know who he is? Yeah.
Carter Morgan (1:04:46)
Okay.
Nathan Toups (1:05:01)
cool.
Carter Morgan (1:05:07)
Yeah, yeah, SlateStar Codex
Lawrence Kesteloot (1:05:09)
Yeah, that's the old name.
Carter Morgan (1:05:09)
or yeah.
Lawrence Kesteloot (1:05:10)
It's now called Astral Codex 10. And yeah, he's great. And I think he's just really well balanced. Do you remember like in the old days, you were talking about Joel Swolsky, right? Like in 2005, like people just refer to Joel and everybody would know who Joel is. And then this is sort of the same. People just talk about Scott and he writes really good long form content and it's very thorough and well thought out and pretty well balanced. And I mean, if you've not discovered Scott, think it's great.
Carter Morgan (1:05:21)
Yeah, yeah.
Nathan Toups (1:05:25)
Mm-hmm.
Carter Morgan (1:05:26)
Right, right.
Great.
Nathan Toups (1:05:37)
I can't wait. I can't wait.
Carter Morgan (1:05:37)
Yeah,
don't, I don't, he pops up on my Twitter, you know, he's always, he's good at like getting into the discourse. Like, and not even like himself necessarily, but he'll always, he's writing thought provoking stuff enough that I'll see other people be like, Hey, did you see what Scott Alexander said? And so, yeah, I, I've everything, anything I've read from him, I've really enjoyed. Well, thanks so much for joining us, Lawrence. thanks everyone for listening. you can always, see all of our episodes at bookoraflo.io. You can contact us at contact at bookoraflo.io.
The podcast is on Twitter at BookOverloadPod. I'm on Twitter at Carter Morgan. And Nathan and his work with his consulting agency are rohoroboto.com and his newsletter is there at rohoroboto.com slash newsletter. Thanks again, Lawrence, and we'll see you around.
Lawrence Kesteloot (1:06:19)
Have a great time. you guys.
Carter Morgan (1:06:21)
Thanks.