Hey there, welcome to Book Overflow, this is the podcast for software engineers by software engineers, where every week we read one of the best technical books in the world in an effort to improve our craft. I am Carter Morgan and I'm joined here, as always, by my co host, Nathan Tupes. How you doing, Nathan?

Nathan Toups (00:30)

Doing great. Hey everybody.

Carter Morgan (00:32)

Well, we have a fantastic episode for you today. This is Mark Richards and Neil Ford reflecting on software architecture at the hard parts. Fun fact, we had actually scheduled them to be separate and then they were just chatting with each other. They're actually working on a new book right now. They're like, Hey, you're you're doing book overflow, so am I. Like, let's just do it at the same time. so glad we did because we just wrapped up and fantastic interview, fantastic discussion with both of these gentlemen. We're a huge fan of their work. Neil Ford's been on the podcast now three times, and Mark Richards this second time.

and yeah, just a great discussion. Nathan, give them a sneak peek of what they're about to hear.

Nathan Toups (01:07)

I mean, we we we just effortlessly sort of did a deep dive into our favorite parts about the hard parts and just kind of riffed on a bunch of topics, including AI, which of course was not in the book because this book was written in twenty twenty-one. I think Carter, you say something about it being sort of frozen in amber, which I thought was a sort of apropos. it's it's also interesting. I I looked real quick

Do you know when they actually were guests on the podcast last? It's been more time than I want to admit.

Carter Morgan (01:37)

Well, Mark Richards was our n first guest ever, so that must have been in like June of twenty twenty four.

Nathan Toups (01:40)

Yeah, so yeah,

yeah, I think it may it is either June or July that it came out. But yes, it was June, I think it was June or July 2024, both for Neil Ford's first appearance and Mark Richards. And then Neil Ford came back in September. But it's been since the year 2024 that we've had either of them on on the podcast. So it's been really cool. They've actually had a second edition of Fundamentals come out since then that we haven't read. Actually, Team Topologies and this should but

Carter Morgan (01:45)

Right.

There we go. Okay.

Nathan Toups (02:10)

I don't know. It's really exciting. I I think we have an open invite for more conversations in the future. And I just come away from all these conversations like so energized. I don't know about you.

Carter Morgan (02:19)

Well, absolutely. I mean, there's a reason we've read so much of their work because they're fantastic authors and they're fantastic interviewees as well. We're excited for you guys to hear it. So stay tuned for the whole episode. This is Mark Richards and Neil Ford reflecting on their book, Software Architecture, The Hard Parts.

Carter Morgan (02:37)

Well, Neil Mark, it's such a pleasure to have you on the podcast again.

Neal (02:40)

it's great to be here again.

Mark Richards (02:41)

Yeah, indeed it is. Thank you.

Carter Morgan (02:43)

Well, I yeah, we were talking about before we recorded, but this is I think only our second ever joint interview. The first one was a a strange combination. Uncle Bob and John Osterhote, who are almost a little adversarial with each other. We were we're it was fun to get on the podcast together. and we're

Neal (02:59)

I I'll bet I'll bet Uncle

Bob sp s spoke seventy percent of the time, right?

Carter Morgan (03:03)

Yeah.

Nathan Toups (03:05)

Y actually, you know,

they had a good back and forth. Well, what's funny is I and I'll give a little context for for folks who are not aware, we tried to get them to debate. I think Osterhout was very against T D. He's very big into unit tests and other coverage, but just different philosophy, right? Of of how to do testing. and then Uncle Bob sort of representing that the sort of agile, you know, consulting community side of this and his sort of approach with clean code.

And they couldn't agree to the terms of the debate, which was just hilarious itself. And it turns into this email back and forth that John Osterhout eventually publishes on GitHub of like all the stuff that they had. and we were like, wait a second, I think we were the catalyst for this. and and yeah, to the point that I think the update to clean code actually had references to influences that on John Osterhout had had on him through these debates.

Neal (03:37)

Ha ha ha.

Ha ha

Carter Morgan (03:50)

Yeah.

Nathan Toups (04:01)

And then we had them on were like, come talk to us and they were actually incredibly cordial and both had like really good back and forth. I mean, I was pleasantly surprised about the whole thing.

Carter Morgan (04:09)

Yeah, yeah.

I know I I saw it in like hacker news. It was like, Hey, like John Ostroho and Uncle Bob had like this debate and I was like, Wait, didn't we introduce them to each other? Yeah. well we wanted to congratulate both of you. we were talking to Mark about this before we recorded, but Neil I maybe sent this over in the email when we scheduled, but yeah, we we did before last year we said, you know what, let's put yeah.

Neal (04:19)

Mm-hmm.

Nathan Toups (04:28)

Yeah.

Yep. This December of 2025.

I actually have the the statistics here. So fundamentals of software architecture was the winner. So what we did is we actually we read through, I think it was all 32 books or something that we had read since the beginning of the podcast. We're quickly approaching two years on the podcast. I think we've read almost 40 books at this point. and we could we did a, you know, sort of like a bracket, a March Madness bracket. And so

Carter Morgan (04:37)

Okay, give give it to us, Nathan.

Like a march madness bracket, yeah.

Nathan Toups (05:00)

Philosophy of software design versus fundamentals of software architecture became the final two in our our audience actually was the tiebreaker. And fundamentals of software architecture was the the ultimate winner of this March Madness. Yeah. And one of the things I wanted to bring up though is that these are the rounds, and they're I mean, you had some good adversaries here. round one, you were up against software engineer the software engineer's guidebook, which is the Gergé Oraz, the pragmatic engineer book.

Neal (05:11)

All right. That's awesome.

Mark Richards (05:12)

Wow!

Carter Morgan (05:13)

So yeah.

And

Neal (05:20)

Mm-hmm.

Nathan Toups (05:30)

And

then then round two, you're against mastering open telemetry, which is Steve Flanders, which you know very domain specific. So I think it was an easy winner there. Steven Wolfram's What is Chat GPT doing? that you won that round. Then it was up against team topologies, which you rose about I know, which we we're huge fan. and then and then you went up against refactoring by Martin Fowler.

Neal (05:47)

wow.

Mark Richards (05:47)

Mm.

Neal (05:49)

Yeah, we are too.

Ha ha ha.

Nathan Toups (05:56)

And then you went up against John Osterhouse of Philosophy of Software Design. I mean, these are all heavy hitters.

Mark Richards (06:00)

Wow.

Neal (06:00)

Wow.

Carter Morgan (06:01)

Yeah. So it wasn't an easy path. and I I've recommended Fundamentals for a long time. I've said, like, look, if you're a software engineer and you just want the best bang for your buck book to, you know, kind of level up your career to get beyond kind of just the coding, I merge to main, I'm not sure what happens after that. Right. Like Fundamentals is is the best book out there. and Software Architecture, the hard parts, which is what we're discussing today, excellent follow-up. We really, really enjoyed it. I guess we wanted to ask, go ahead.

Neal (06:02)

Well

there's a great story about how that book came about. Literally the name and and everything in the content. I'll let Mark tell it 'cause he's the one that came up with the name.

Carter Morgan (06:33)

Yes, wh why don't you tell us?

Ha ha.

Mark Richards (06:37)

Well,

that's true, but we had some challenges writing the fundamentals of software architecture on that first edition because we had so much material that we kept having to skip things out. And that could have been a 200 page book or a 20,000 page book. And we're of course limited on page count. So we kept coming up with all these things we wanted to talk about. And we're like, we're going to have to...

Carter Morgan (06:58)

Ha ha.

Mark Richards (07:06)

We have to push this one off. Now, we have to talk about distributed transactions. I mean, it's part of the whole game. It's like, no, no, we got to push that off. And ended up we had this big pile of all these great topics. And I think I made the comment, it's like, well, these are all the of the hard parts of architecture. And Neil said, I think that's our title.

Neal (07:28)

Well, in in fact I I remember distinctly. We were on one of the, you know, mean Zoom calls you get on when you're trying to write a book, and not in the same physical space. And we were ejecting yet another perfectly good example, and Mark said, Well, maybe we should call the next book the hard parts and that that was just the the spark is, Hey, that's a great name. 'Cause of course it it sort of is a little riff on JavaScript the good parts, but software architects the hard parts. So we were hoping enough people understood the connection there to sort of get the the joke.

Carter Morgan (07:56)

One thing we really enjoyed about reading software architecture, the hard parts, is this was published in was it twenty twenty one or twenty twenty two? During the pandemic, okay.

Neal (08:04)

During the pandemic. So there are actually tons

of Easter eggs down within that text because we were trapped at home during the pandemic writing a book and so we ended up putting a bunch of Easter eggs in there. Some of which we can tell you.

Mark Richards (08:09)

you

Carter Morgan (08:14)

That's fun.

Mark Richards (08:16)

Yeah.

Carter Morgan (08:17)

well well, what are some of the Easter eggs?

Neal (08:19)

So we'll talk about some of the Easter eggs. One of things that we wanted to do, just sort of randomly, is to see if we could write a technical book that had not a single gendered pronoun in it. And so if you look at software auction, there's not a single gendered pronoun in it. And we had characters in the hard parts book, the Sisop Squad. and we chose very carefully chose gender neutral names for every single character in our Sisop Squad.

Carter Morgan (08:32)

Okay.

I did notice

that.

Neal (08:46)

And y we were just trying, you know, just kind of an experiment to see. You know, I mean it's a it's a pandemic. We're at home writing a book, so you know, it seems like an interesting thing to do. Well, so when you publish a book, the first thing you always wait for is the first Amazon review to see what the world thinks of it. And we got our first Amazon review and it was five stars, but the comment under it said, I thought this is a great book. The only thing I didn't like about it was the one character I didn't like was a woman.

Mark Richards (09:13)

And we still don't know.

Carter Morgan (09:12)

what?

Neal (09:12)

We have no idea who he's talking about because he didn't say the character name.

Carter Morgan (09:14)

Yeah.

Neal (09:17)

We still don't know. And and we suspect

it's more about him. I guarantee it's a him. Then he again so that that's one of the little Easter eggs. But and we actually wrote, so you're familiar with my previous work, this idea of architectural fitness functions. We actually had a book fitness function that ran every time we checked in to see if we had gendered pronouns, because it's easy to fall into. We also have a fitness function that we put in every book that

Nathan Toups (09:22)

Right. It was an imaginary character.

Carter Morgan (09:37)

interesting.

Neal (09:42)

You never start a sentence there are 'cause it's a weak start of a sentence and so we it's easy to fall into. So we we actually have some stuff that scans our text to see if we're doing bad writing.

Carter Morgan (09:44)

Okay.

Nathan Toups (09:54)

speaking from a technical standpoint, if I remember correctly, and please roast me if I'm wrong, it y'all use is it ASCII doc or what is the okay.

Neal (10:00)

ASCII Doc. Well, and

Mark Richards (10:00)

Yes. Yep.

Neal (10:01)

and one of the brilliant

things about the use of ASCII Doc in this book is so one of the things that we realize in the hard parts book, even more so than the fundamentals book, is that when you talk about software architecture in the abstract, it it's kind of boring. Because the only things you can reach conclusions are are really obvious things. But real software architecture is all about the nuance of trade-offs of a particular situation or scenario, et cetera.

And so we realized that this is gonna be the dullest book ever if we just hand wave, you know, all this architecture stuff for the entire book. And so we invented the Sysop Squad as a way of unifying that story. Almost like, and this is funny because we had this idea and wrote the whole book with all that, and then we were talking to the publisher and and we were like, Yeah, you know, it's it's sort of like a very lightweight version of the Phoenix project. And our editor said to us, you know, since the Fiji project came out, we get approximately three book proposals a week.

To do that for some other tech stack. It's like, okay, good. It's good we didn't ask because we just barreled ahead and did it. But one of the things that we wanted to do, and this is actually one of the other Easter eggs in the book, all the meetings for the Sisop Squad. So every Sysop Squad meeting, there's a time progression for those, for the summer solstice through the winter solstice and back to the spring. And so when you get toward the end and things are getting better, it's springtime. And so there's a but, but it's a one unified story, but it's all scattered around through the book. And so here's the beauty of ASCII Doc.

Mark Richards (10:56)

Right.

Carter Morgan (10:56)

Yeah.

Mark Richards (10:59)

You

Nathan Toups (11:11)

No way.

Neal (11:22)

I wrote a little bit of magic that created a synthetic chapter. So we can do includes an ASCII doc. And so all the sysop stories were includes. And we built a synthetic chapter at the end, which was nothing but the Sysop star sysop squad stories top to bottom. So we could go to that chapter and read to make sure it all made sense linearly. But then when we actually produced the book, it scattered them all throughout the contents of the book.

Nathan Toups (11:42)

Interesting.

Mark Richards (11:44)

Yeah.

Well, and as a matter of fact, another interesting, well, I'm not sure if I call it an Easter egg, but a large majority of those stories actually came from Neil and I's personal experience at consultants sites or client sites as consultants. And so it's I think one of the reasons we've gotten some feedback that it really rang true.

is because most of those scenarios we wrote about were. So we didn't have to make up a whole lot to actually stitch that story together. But I do think it was one of the pieces that I thought was most fun was actually writing those Sysop Squad narratives because...

Carter Morgan (12:13)

Yeah.

Neal (12:28)

We had to write dialogue and neither of us

Carter Morgan (12:29)

Yeah.

Neal (12:30)

had

ever written dialogue and we sucked at it when we started. In fact, the good thing is that Mark's wife is a novelist and so we leaned heavily on her to please make this not suck so much in the the dialogue 'cause it's a completely different kind of writing that, you know, we'd never really thought of because we'd never done it, 'cause we write technical books, but you all of a sudden here we are.

Carter Morgan (12:37)

there you go.

Mark Richards (12:41)

Yeah.

Carter Morgan (12:42)

Ha ha ha.

I really enjoyed it as a writing structure. I think, you know, there's a reason the Phoenix Project and the Unicorn Project are so popular. And I I actually w we read the Unicorn Project. We haven't actually haven't read the Phoenix Project. but Yeah, yeah. Yeah. but I I think

Mark Richards (12:57)

Hmm.

Nathan Toups (13:04)

Yeah, I read it years ago, but I'm I'm in DevOps background, so that's like required reading, right?

Mark Richards (13:08)

Yeah, yeah.

Carter Morgan (13:12)

I I wish there were more books like what you guys did, where it's like half of it is just kind of purely technical and then you you take almost a bit of a breather to help that stick. I mean, I found that to be a very, very helpful writing construct. I I

Neal (13:27)

Well, and for us too,

it was critical because we could talk about specific trade offs, not generic ones. And that's when it gets interesting. My former colleague Birgitta Burkela said that when she read the hard parts book, she said, Normally I skip all those stories about s about people. She said, But about the second chapter I realized, no, I'd better go back and read those. It seems like there's some important stuff in there And yeah, sure enough

Carter Morgan (13:40)

Ha ha ha.

Yeah.

Mark Richards (13:47)

You

Carter Morgan (13:47)

And this was

another one where because I remember when we interviewed it might have been you, Mark, but you said with fundamentals, you're like, this totally shocked us. The audiobook sold surprisingly well, right? Yeah. Yeah.

Mark Richards (13:58)

my gosh, yes. when an audio... Well, so when

Neal (13:59)

Yeah. I still can't imagine it.

Mark Richards (14:03)

the

audio book from the fundamentals first came out, I said, are you serious? We've got all these technical diagrams and architecture diagrams and pictures. And I'm like, I just, I don't get it. So of course, Neil and I get copies of these and I said, okay, I'm gonna just listen to this whole thing. And I was actually shocked. I do remember this conversation we had in a...

Carter Morgan (14:08)

Yeah.

Mark Richards (14:27)

prior podcast. Yeah, I was absolutely shocked at how well it conveyed all of the diagrams, the explanation of the diagrams that flowed. And I'm like, wow, that shocked me.

Carter Morgan (14:42)

Well, we it it came through with the hard parts too. We read the hard parts, or I I mean I I listened to an audiobook and I was pleasantly surprised.

Neal (14:42)

It's an un

Mark Richards (14:45)

Yeah.

Nathan Toups (14:49)

Well, I I will say though,

I I had the audiobook and I started with it and I was like, okay, actually I need to go back. So I ended up buying the the the Kindle version and the audiobook. So, you know, great for y'all. And and two things end up happening. Either I would listen to a section and then go back and read well, there was a PDF that comes along with it, but still the book is nice to have the context with it. But then I ended up finding as I got further into some of the technical stuff, I would actually listen to the audiobook and read it at the same time.

Neal (15:00)

Yeah, thank you.

Carter Morgan (15:01)

Ha ha

ha.

Nathan Toups (15:18)

which I think just further sort of solidified some of these more complex topics. And so we've been big advocates for books that are executed this way. That are a lot of our listeners have been surprised that we're like pro audiobook when they're available. as a supplement. And again, it changes the way you consume it. You have to think about it. if these are new topics, I would say like entirely new topics, you probably should read, like physically read it.

Carter Morgan (15:44)

I had that

problem with designing data intensive architecture or applications. I was like, this is this is too much. I should have read this.

Nathan Toups (15:48)

Yeah. Yes. Yeah.

Neal (15:48)

Yeah.

Mm-hmm.

Mark Richards (15:51)

Hahaha

Nathan Toups (15:52)

No, that one that one definitely is that one you have to s let it s let it stew for a bit.

Mark Richards (15:56)

You

Neal (15:57)

Mm-hmm. Mm-hmm.

Carter Morgan (15:58)

Yeah.

Well

Neal (16:00)

Well, one of our goals when we write books, so we were also surprised that paper copies of the Fundamentals book sold better than electronic copies. But one of one of our mantras, so you it takes an enormous amount of time and effort to write a book. And the last thing you want to do is write a book that just kind of comes and goes instantly after you wrote all that spent all that time doing it. And I know that from experience because my first few books did exactly that.

Carter Morgan (16:08)

really?

Yeah.

Neal (16:21)

started thinking, okay,

how can I prevent that from happening? And so one of Mark and my goals, and I was actually just reflecting on this recently. I didn't tell Mark about this, is one of our goals for technical books is to create IP that will last at least five years, but hopefully 10 years. Because it's going to take you two years to write the book anyway. And so you really want to create IP that lasts for a long time. And sure enough, fundamentals is 10 years old now.

Mark Richards (16:24)

haha

Carter Morgan (16:44)

There you go.

Neal (16:44)

we've we've managed to create some IP that is still relevant because in the second edition we mostly added stuff. We didn't change hardly anything in it because it's still relevant. And so that I I think was shows that that's part of our goal. one of one of the drivers for writing a book, we have this little list of things why you would write a book in the twenty-first century. Because hardly anybody reads. Hats off to you guys because you're actually reading and gonna encouraging people to read. How can we clone you guys and spread you all over the world? I would love to do that.

Mark Richards (16:51)

Yeah.

Carter Morgan (17:06)

Ha ha ha.

Mark Richards (17:07)

I know.

You

Neal (17:14)

But it's really hard to get people to read. And so one of our other mantras for this, and this goes back to the hard parts book, is that a technical book should have a narrative arc. It shouldn't just be a dry collection of facts. By the time you get to the end of it, you should feel like you've had this entire story put together for you. And and what's interesting about the hard parts is that it actually has three narrative arcs overlaid on top of one another.

Because it has the technical narrative arc, which is all the solutions, and then it has the Sisop Squad, which is a story that reaches a culmination and a and a finale and a climax. And then the very last chapter of the book, this is another Easter egg, by the way. The first 14 chapters of the book are written in first and third person. We're showing you all these trade-off techniques. But the last chapter is called Build Your Own Trade-Off Analysis.

We switched to second person on purpose for the last chapter because we're trying to drag the reader in. Hey, we've been talking at you for this entire book. Now let's do it together and drag you in. And so that's actually the the the narrative arc for all the trade-off analysis stuff wraps up in that last chapter that pushes it back on the reader to to to take on take up the mantle.

Nathan Toups (18:24)

I didn't catch that consciously, but I did I think it spoke to me that way. And so it's a really kind of great from a call to action standpoint.

Neal (18:31)

We we really love call to actions at the end of the book. So it's a it's a great way to wrap up a narrative arc, you know, and and really stitch the thing together. So that's one of the things that Mark and I are always adamant about is our technical books need to read like a real book and you know, by the if you do read the entire thing by the time you get to the end of it, you feel like it's it's better than the sum of the parts. And we're doing the exact same thing right now. Mark and I and Raju Gandhi are writing a patterns book, Software Architecture Patterns, Anti Patterns and Pitfalls. And this is the first book as as far as I know ever

Mark Richards (18:33)

Yeah.

Neal (19:01)

To focus not on individual patterns, but comparing patterns. Because in the real world, to choose one, you have to compare them. Well, what kind of basis do you have for comparison? And so that's the narrative arc of our patterns book, is this idea of none of these things live in isolation. They all influence each other. In fact, one of the sections we have for each pattern are in external influences that can change the trade-offs for this pattern. And so it's a much more holistic view of patterns that we've seen in the past because it almost always is just.

Carter Morgan (19:05)

Neal (19:29)

discrete pattern, discrete pattern, discrete pattern, and no thinking about how they interact with each other. So that's the narrative arc of the the patterns book.

Carter Morgan (19:37)

We say any time we read a book we love a good pitfalls and anti pattern section. So you guys are doing a whole book on that.

Neal (19:43)

Mark Richards (19:42)

hahahahah

Neal (19:44)

are doing a whole book on that.

Nathan Toups (19:44)

Well that's what

I that's what I always learn from. I mean, that I've had enough war stories and enough bad decisions I've made in my career that I'm like, okay, I wanna see where things go sideways or what are the trade offs, right?

Mark Richards (19:54)

Right, yeah,

Neal (19:55)

Mm-hmm.

Mark Richards (19:56)

yeah, yeah, we all are pretty familiar with what we're right or well, but sometimes not what we're actually doing wrong. And so that's why this just becomes really interesting. Anytime, we go way back with doing software anti-patterns and pitfalls, development anti-patterns. Now, the book we're writing here is on architecture, but.

Neal (19:56)

That's often the most interesting bits.

Nathan Toups (19:58)

Mm-hmm.

Carter Morgan (20:07)

Okay.

Mark Richards (20:22)

Boy, those were some of my most popular talks at conferences, those software developer anti-patterns and pitfalls. And yeah, those were fun to talk about.

Neal (20:35)

Yeah, we we distinguish we did this in our the second edition fundamentals book, but in this book as well, distinguish between an anti pattern and a pitfall. because an anti pattern is the classic kind of example. It looks like a pattern, but over time it turns into something bad. But we have another category of things which are pitfalls, which basically just burn you right away. It's like the a tiger pit, that's the pitfall where, you know, it looks like a nice grassy meadow and now I'm stuck with spikes in me. So

Carter Morgan (20:53)

Yeah.

Mark Richards (20:59)

You're in.

Neal (21:01)

Those

are pitfalls which are things that may initially seem like a good idea but almost instantly lead to something.

Nathan Toups (21:07)

I loved in the saga s section, where you're really diving in. And once as soon as I got to horror story saga, I was just like, man, I've seen this more times than I want to admit.

Mark Richards (21:17)

Nathan, we've got a great story about that. So this actually was one of Neil's discoveries about, what would you say, maybe three quarters the way through the writing of the book. And so Neil pinged me on a text and said, hey, do you have a little bit of time for a Zoom call? And I said, sure. So we set up a Zoom meeting and...

Neal (21:31)

extremely late in the process, much later than we wanted it to be.

Carter Morgan (21:32)

really?

Mark Richards (21:43)

talked and he said, I think I've come up with a discovery here. You know how we got all these chapters and we talk about communication and coordination and transactions and all this. And I said, yeah, yeah. He said, I think these are interconnected. And I said, what do you mean? And he said, well, well, take a look at this. And he showed me a few things and he said, you know, I've been thinking about this for a while and it seems like we treat all of these as individual.

factors within an architecture, but if you change one of them, the other two are influenced, almost like little dials that you're turning. And so what ended up being supposedly a quick 10 minute Zoom call, we were on for almost two hours. I said, Neil, said, I think you're right. mean, you've demonstrated it to me. We need to add this to the book. And we're almost like done with the book.

Neal (22:35)

Well, and and part of the problem too was that

Carter Morgan (22:36)

Ha ha ha.

Neal (22:38)

I had I had taken on the the say something interesting about transactional sagas and we wrote all the other stuff and I didn't have anything interesting to say and we wrote all the other stuff and I didn't really have any and so it was starting to get really dire. It's like, okay, am I going to be able to say anything interesting about this? And it it was actually a great relief to figure that out 'cause I was actually listening to Mark teach this material for, I don't know, the five hundred and third time or something online, and it that's when it kind of hit me. And

What's frustrating about that is six months earlier I had started down that path of that taxonomy and then kind of backed off of it for some other reason and then came back to it and finally figured it out.

Mark Richards (23:15)

Well,

and so I want to continue the story because Nathan, you mentioned one of them. So one of the things that was a lot of fun, these are transactional sagas. And a saga is a story that you tell. And it's basically the story of the request flow as a transaction through the system. So we got a chance because these were our own IP to be able to name these things. And that was a fun part.

Carter Morgan (23:43)

Ha ha.

Mark Richards (23:44)

Because

we all want you know, they needed to be related to some sort of book type of book or a story well Turns out we named all these and we looked at the whole catalog of all eight of them It's like, know what? No one would do this one and especially the horror story was the only one Neil and I had not seen in the wild all the others. It's like yeah. Yeah. Yeah I remember one one project I was on at a client we did this and but horror story Yeah, that was one of the exceptions and it's like well

I don't know, let's put it in there for now, but we may pull it out because generally, like the mistake we made with waterfall methodology, generally you don't want to write about something that is wrong, not to do. Well, guess what? Two months later, I get a ping from Neil and he says, guess what? I say, what? And he says, I've come across the horror story. In the wild.

Carter Morgan (24:26)

Right, right.

Neal (24:37)

In the wild. Yep.

Mark Richards (24:41)

So we in fact did see all of those, we hence included it in the book.

Neal (24:46)

Well and

part of our idea of naming the things and giving them sort of whimsical names was we realized that nobody would remember the positions of the sliders because it's all different combinations of the three sliders. And so giving it a name like that, you know, creates an association that you can kind of carry. It's sort of like design patterns. So it's it's sort of the same flavor, but it it's much easier to think about them by their names than the than the the three attributes.

Nathan Toups (24:53)

Right.

Carter Morgan (24:53)

Right,

right. Yeah.

One thing we found really interesting about this book is because of when it was released, right, during the pandemic, it's almost a little frozen in amber because it's before the release of LLMs, at least in any sort of large commercial capacity, right? and so I think the book obviously is evergreen because it's it's focusing on architecture, which is as much as LLMs have helped us, haven't really, you know, they're not really gonna solve that problem for us quite yet. but

Mark Richards (25:18)

Mm-hmm. Yeah.

Neal (25:18)

Yep. Absolutely. Yep.

Or ever.

Carter Morgan (25:38)

Or ever, right.

And so I guess maybe I I want to talk to you guys about that, right? Which is, you know, what are your takes on LLMs and development? How do you use them? and and you point out, Neil, I mean, it these may never solve our kind of architectural problems. that's a position I'm I am inclined to hold as well, but it's always nicer when someone a little more experienced than me holds it too.

Neal (25:57)

Well and and well and

here's why. So this is a little bit of a lengthy explanation, but it's very useful. And in fact I wrote about this first in my book from two thousand eight called The Productive Programmer. So this is an idea. This idea actually came from the sixties or seventies in the United States nursing profession. It's this thing called the the Dreyfus model of knowledge acquisition. Sounds very fancy, but it turns out that the nursing profession is very much like software development in that you can go to school to do it, but you don't really become a nurse until you do it for real.

Carter Morgan (26:17)

Interesting.

Neal (26:26)

Same for software development. You don't really until you've done it for real, you can't really call yourself a software developer. And so as an aid to training nurses, they came up with a scale. And I'll give you the scale briefly. The first is novice. A novice can only apply recipes, and if something breaks, they're lost because they don't understand why things work. An advanced beginner has done dozens or hundreds of recipes and they can do limited improvisation now. If a recipe breaks, they can kind of substitute some steps and get it working again. So

Carter Morgan (26:26)

Interesting.

Neal (26:53)

Still don't understand how it works, but can kind of make it work by manipulating it. Competent is the next one which understands how things work. Proficient is the next one up from that that understands all the layers. And then expert is they know it so well they can't even explain it to you anymore because it's basically muscle memory. The reason this is pertinent is because LLMs are perpetually advanced beginners. They are good recipe finders, but they're not reasoning.

Carter Morgan (27:16)

Interesting.

Neal (27:21)

You can still trip up virtually every LLM with Johnny picked five mangoes, Susie picked three mangoes, two of them are smaller than the others. How many mangoes do we have? They all subtract because they're not doing the math. They say a recipe that says smaller means subtract, and they subtract. This is the fundamental problem with software architecture. In our first edition of the fundamentals book, and the second two, the first law of software architecture is that everything in software architecture is a trade-off. But to do a trade-off, you have to reason about.

Carter Morgan (27:30)

Right.

Mark Richards (27:30)

You

Neal (27:56)

And so to be able to get past, they can't brute force their way past advanced beginner, but what we need what needs to happen now is actually different kinds of AI, modeling, reasoning engines, inference, neural networks, and that kind of stuff need to be plugged into it. But that's a completely different breakthrough that we haven't seen yet.

Nathan Toups (28:13)

Mm-hmm.

Right.

Neal (28:14)

And

so we are at a limit to how much LLMs can do, particularly for things like trade-off analysis, which requires humans and it's so nuanced that we can't even do it just one time and forget about it. That's this idea of a best practice where anytime this happens, just turn your brain off and do that. Well, that never happens in software architecture because it's always different values of trade-offs that get tweaked, et cetera. So that's the the real essence of software architecture.

Carter Morgan (28:27)

Right.

Mark Richards (28:39)

Well, and there's two other aspects of this as well. Do you had a question?

Carter Morgan (28:39)

Mind

no, I

just gonna say Martin Fowler called it. he said he didn't like the term best practices. He liked the term sensible defaults. Right.

Neal (28:49)

Yep. I

Mark Richards (28:49)

Yeah, yeah,

Neal (28:50)

think that's a much better idea.

Mark Richards (28:50)

yeah, exactly. There's two other aspects though that both Neil and I frequently talk about with regards to LLMs and such. And that is first, when we talk about architecture and what does it even mean, it's really that distinguishing factor between behavior versus the capabilities. And I will have to admit, LLMs are extremely good now with behavior in terms of the overall design.

Neal (28:57)

Okay.

Carter Morgan (29:15)

Right.

Mark Richards (29:17)

does it function the way it's supposed to? And yeah, with some work back and forth and some spec-driven development, we can actually really model the behavior fairly well. But what the LLMs currently lack are the capabilities. And capabilities are the architectural aspects, not only the structure of the system and what its internal structure looks like.

but also all of those kind of illities, what we call architecture characteristics. Too many times we hear, oh yeah, I got this thing done in like two days. And it's like, oh great, does it scale? Huh? Is it available? What do you mean? And all of a sudden you realize, oh yeah, those things. And John Gall way back when wrote Systematics and he coined Gall's law, which is a law he found which

Carter Morgan (29:54)

Right, yeah.

Neal (29:55)

Mm-hmm.

Mark Richards (30:11)

applies here, meaning that you can't just kind of add or bolt on architecture. It requires a lot of planning. And so consequently, that's the other big challenge with LLMs and architecture. But there is a third one, and that is the fact that when we make a decision, any of the mirror decisions we make regarding

Should this be async or sync? Should this be orchestration choreography? Should I use a library or services? All of these dozens of questions require an incredible amount of context. And right now, what do you call it, Neil? The tension span, I think is...

Carter Morgan (30:49)

Yes,

Neal (30:54)

Yeah, the attention

the we missed such an opportunity. We call it the context window. We should have called it attention span, 'cause that's what it is. That's its attention span. So

Carter Morgan (30:59)

Yeah, yeah.

Mark Richards (30:59)

Finch and span. I mean,

Nathan Toups (31:00)

Right.

Mark Richards (31:01)

you can feed all of this context. Well, I don't think you can ever feed all of the context in, but you feed in so much context that the context window overflows and the LLMs is like, yeah, I'm just not listening anymore. Look, let me just get going on this. And so consequently, that's the other big challenge with trying to teach LLMs about architecture. We've made some pretty strong

head roads, head ways into this with another book that we're currently writing that we've been working on for over two years now about trying to be able to teach LLMs about architecture in the most concise way possible. And we've come up with some pretty neat languages and ideas for doing this, but.

Carter Morgan (31:37)

Mm-hmm.

Neal (31:49)

Well,

in fact, a good example of that is that one of the things you care about from a capability standpoint is producing good quality code, taking t ad advantage of the things that Uncle Bob talks about and clean architecture, et cetera. It turns out that agents can produce high quality code, but they don't do that by default. They're lazy. And so if you just tell them produce code, they're gonna produce slot, but if you start constraining them,

Carter Morgan (32:08)

Right. Yes, yes.

Neal (32:13)

in the way they produce code, they will produce much higher quality code in terms of coupling and cohesion and other metrics and so. that's the real trick is to build constraints that that in that guide it to building proper code, not just let it run wild.

Mark Richards (32:27)

Neil and I were using Claude last month, and we wanted to add some of these constraints in from an engineering practices standpoint. So we said, generate this particular spec for us, but make sure the cyclomatic complexity is less than 10. OK. And so chug, chug, chug, chug, chug, chug, chug, chug, chug, chug, chug, couple of tens of thousands of tokens later. Here's your code. And it's like, OK. And we're looking at it. It's like, this is not.

a cyclomatic complexity lesson 10. And so we asked it, what is the cyclomatic complexity of this code? Came back like 56. And it was like, we told you to do a 10. yes, you're so correct. Let me rewrite that code. And it's like, so the lesson we learned, which we already knew, but we needed to actually demonstrate, was that you instruct the LLM.

Carter Morgan (33:04)

Mm-hmm.

Mark Richards (33:21)

to do certain things. And it comes back with all the confidence in the world to say, yeah, here you go, it's all done. But if we hadn't verified that through even a fitness function and some sort of guardrail, then we're just using blind trust that, yeah, all the tests pass. Really? It looks like they all say assert true on them.

Carter Morgan (33:42)

Yeah.

Neal (33:43)

Well, and and and that's the thing, if they're if they're recipe

finders, if you put enough constraints on them, they'll eventually realize, the way to get this pesky unit test to pass, it won't pass, is just replace the assertion with assert true. Success. We can move on. So but one of the things I've realized recently is that so human developers won't do either of those things. Assert true or just ignore your instruction. But the problem is for human developers, you can either shame them publicly or fire them. And you can't do either of those things to an agent.

Carter Morgan (33:54)

Right, right. Yeah.

Right, right.

Neal (34:14)

Ha ha ha

Carter Morgan (34:15)

Well yeah, I

I w I was going back and forth with Claude one time and it it you know, I it got all tripped up and I it said like, sorry, I got it mixed up in my head and I was so annoyed. I'm like, You don't have a head like yeah.

Neal (34:27)

You know, the the

the anthropomorphism drives me crazy. And you know that's that's a feature, not a bug. They found that the more it sucks up to you, the more you'll interact with it and spend tokens. And so even though you're paying for it, they're still treating you like the product, which is really annoying. I hate the obsequiousness of L L Ms.

Carter Morgan (34:30)

Yeah. Right, right.

Right, right.

Nathan Toups (34:41)

It it it is weird

how like defensive I have to like I've built a lot of tools to defend myself against the defaults. And so I I actually am a huge fan of Matt Pocock's work on this. I don't know if you've heard of this the grill me skill. if yeah, it's phenomenal, which is literally because of this context thing. It's like grill me until I can explain the situation clearly enough, and then it helps you draft up a markdown file.

Neal (34:57)

Have heard of yet?

Nathan Toups (35:08)

So that you really are like, you're not just dialing stuff in. It's like actively pushing you on stuff, which I find really helpful for just being a rubber ducky, right? It's a and it also gets back to I think we just recently finally read Mythical Man Month and realized that these sort of like these very clear clarity of thought structural ideas are actually like more prescient than ever. Like we real like reading long form stuff, really understanding architecture, really thinking about how to think through a system.

Mark Richards (35:24)

Mmm.

Neal (35:31)

yeah.

Nathan Toups (35:38)

is great when you it it's incredibly important now that we do have this little army of advanced beginners, right? It because if you can leverage them as advanced beginners and let them sort of like operate in their wheelhouse, but you d what you don't want to do is let them design the system for you. Cause all of a sudden, yeah, they'll they'll lie to you or they'll do these things. so I end up finding myself building a lot of sharp tools to help me as well. So my pattern has basically been like grill me, build some small Unix philosophy sort of command line tools.

Neal (35:44)

Right. Yep.

Mark Richards (36:00)

Mm-hmm.

Nathan Toups (36:08)

And then build a skill around that. And then this way g it gives me like a lot of ways of just being like, don't dial it in on on the complexity piece, like build a little tool that runs and in you know, which part do I need some creativity around the large language model? And then where's this like set of tools that I'm doing, you know, local fitness functions or sort of like pre pre-work? And I it's still evolving. I'm not gonna sit here and pretend I have some like that figured it all out.

Neal (36:34)

Well, it is for everybody

still because I mean part of the problem is we're trying to hit a moving target. The LMs are changing and reving and you know the the way we approach things. I mean two months ago everybody was talking about, skills are the way to go. And now people are saying stop with all the skills. You're you're you're destroying the context window of the LM. So I mean it's it's really all over the map. and it's gonna be for a while until things settle.

Carter Morgan (36:38)

Right, right.

Right.

Nathan Toups (36:53)

Right.

Carter Morgan (36:56)

And when you talk about the context windows, I mean, you know, obviously there's billions of dollars writing on this and maybe they'll figure it out. But from what I understand about the transformer architecture, like it's somewhat of an intractable problem. Just that, you know, every token has to attend to every other token. And right, like it it also summed to one. Right. Right.

Neal (37:07)

It is, yeah.

Yep. There's yeah, there's a yeah. Yeah, it it it there's a scale problem that that's hard

Mark Richards (37:11)

Mm-hmm.

Neal (37:17)

to brute force past there. One of the things you mentioned Mythical Man Month. I I remember reading that years ago. And the thing that struck me so much in Mythical Man Month, I still remember that to this day, is the difference in advice on projects versus technology stuff. Because the project stuff is like don't add late peop people to a late software project, we'll make it later. And it's like I still see people doing that.

Carter Morgan (37:38)

Right.

Neal (37:39)

which is still stupid

Carter Morgan (37:39)

Yeah.

Neal (37:41)

since since the seventies. But then you listen to the technical advice and it's things like, consider using something higher level than mainframe assembly language to build your projects. And it's like the the the distance between those two things, we haven't touched mainframes in decades, but the process stuff is still right up in our face every day, which is why that's still a valuable book because we'd never learn the lessons.

Carter Morgan (37:49)

Yeah.

Mark Richards (37:50)

Ha!

Nathan Toups (37:58)

Isn't that funny?

Carter Morgan (38:03)

Well, and that was interesting reading Mythical Man Month. And I I think I had we had Carl Brown, who he has this YouTube channel called Internet of Bugs on the podcast. we like him. He's like our YouTube Obi-Wan Kenobi. We we get a lot of advice from him. he's but I I was telling him, I was saying, like, look, I'm I'm not super concerned about me as like a you know, a a senior principal level engineer. I'm like, I think I'm gonna have a job for a long time. I think all all everything we've talked about today, like

Their context windows are limited. They're, you know, advanced beginners. like I think I'm gonna be okay. I said, I'm really worried about junior programmers though. Cause, you know, when I think about, I said, when I think about what LLMs can do, I said, my first job at my my first task at as a full-time software engineer was to translate a library from like Angular to, you know, AngularJS to Angular 2, right? And like it was a decent amount of work for a junior engineer. And I I'm like,

Neal (38:39)

Yeah.

Mark Richards (38:39)

Mmm.

Carter Morgan (38:56)

Yeah, yeah. I'm like, Claude could one-shot that. And he told me he said, Yeah, but Carter, the thing is, like when you were beginning in the 2010s, he's like, the kind of stuff you could do in an hour would have taken me a day in 1990, right? Because he's like, you had Stack Overflow, you had Google, right? You had more advanced IDEs than I had. He's like, and and Mythical Man Month really boils that idea down to like when you're building software, there is accidental complexity and essential complexity.

Mark Richards (39:08)

Hmm.

Mm-hmm.

Carter Morgan (39:24)

And essential

complexity is just hard because anytime you're trying to model something that's in the real world in software, there's a lot of trade-offs and decisions you're gonna have to make about how you model that. Accidental complexity is the pro I think Fred Brooks calls it like the problems we invent, right? And and what you're talking about, like use use something more, you know, high level than mainframe. and all of software has just been getting rid of that accidental complexity, right? And getting us closer to the essential complexity. I think one of the

Challenges is, and I'm sure you guys have seen this, there are a lot of engineers who have made their living solely in the accidental complexity layer, right? Just as the implementers. which is why I think, yeah, right, right.

Mark Richards (40:01)

Mm-hmm. Yep.

Neal (40:02)

yeah, absolutely. Yep. I remember talking to to

a a programmer at one point and I was complaining about how arcane a bunch of the stuff in C is. You know, the const keyword, it can appear four different places and it means a completely different thing every time it appears a different place. And I was complaining to him about that, and he said, I love that about C plus plus. And I said, Really? And he said, Yeah, because I understand that that makes me part of the priesthood. And I know a bunch of stuff nobody else knows, and I can hold that over him. It's like, man, this is a

Carter Morgan (40:15)

Right.

Yeah

Mark Richards (40:26)

Yeah.

Nathan Toups (40:27)

boy.

Carter Morgan (40:30)

Right, right.

Neal (40:31)

Bad grad attitude about things.

Carter Morgan (40:32)

Yeah.

Neal (40:35)

Complexity for complexity's sake. But that actually goes back. You you bring up a really important point there is that we have very blunt tools to defend against LLMs. One of the problems with cyclomatic complexity, which Mark brought up earlier, is that it's dumb. It cannot distinguish between essential versus accidental complexity. It still takes a human to determine is this complexity really necessary, or we just accidentally made it a lot more complicated than it was before.

Nathan Toups (40:52)

Mm-hmm.

Carter Morgan (40:52)

Interesting.

Mark Richards (40:55)

Yeah.

You know, Carter, though, I want to come back to your comment about the junior developer, because if we're hiring less junior developers, my question then is, where do we get senior developers from? And this, I have heard this question a number of times from a number of people. And so I think we, I mean, maybe in IT, are seeing, well, if you're not bringing anybody in, then how are they going to become senior?

Carter Morgan (41:12)

Right, right.

Neal (41:13)

Well, that's

Mark Richards (41:29)

But I'm not sure a lot of companies are seeing that right now and that longevity piece or sustainability, I should say.

Neal (41:36)

Well and and here's the other scary thing about that is that we're replacing junior developers with LLMs, but how do LLMs figure stuff out? They slurp up all the information that's out on the internet, increasingly that information being generated by LLMs. And so you're getting this dilution effect. So even our replacement junior engineers are getting dumber and dumber and dumber over time because we may actually be at the very peak of LLM functionality because more and more

Carter Morgan (41:48)

Right.

Mark Richards (41:52)

Heh.

Carter Morgan (41:53)

Yeah, yeah.

Neal (42:06)

it's going to be sifting through slop and I think the level of capability is going to sort of settle down into sort of swampy middle ground at some

Nathan Toups (42:12)

Yeah.

listeners and it's been such a rewarding experience just to hear how people feel about stuff and what they're apprehensive about. And I think this reversion to the mean of Stephen Wolfram, when we had him on talking about LLMs, I asked this in a general context of like, what happens when all the blog posts on the internet are LLM written and then it's being trained on this and it's no longer even imitating human behavior any longer. And he's like, yeah, there's a real risk that we revert to the mean and that, you know, yes, the pinnacle of

Mark Richards (42:41)

Mm-hmm.

Nathan Toups (42:49)

its abilities stop. We might have to like freeze at a moment in time as far as like where the training data can come from.

Neal (42:56)

Or start making a better distinction about what you know, where what's the origin, what's the provenance of this data. 'Cause bump both Mark and I are part of the Anthropic lawsuit settlement because they slurped up the hard parts book and software architecture fundamentals and building evolution architecture. So they illegally slurped up all of those into anthropic. So they're going to eventually pay us money for the royalties they never paid us for stealing our IP.

Nathan Toups (43:02)

The other thing that I

Well

i I I also worry that we we're gonna freeze the like advancement of tooling in the sense that you know, is React just gonna be the default, you know, JavaScript framework because that's what all the th stuff was trained on and it's just really easy for it to write good enough React. which is unfortunate because like I I liked, you know, I I don't know, maybe maybe Go, Rust, and Zig are like languages that only could have

Carter Morgan (43:30)

Yeah.

Nathan Toups (43:45)

evolved right at the edge right before L LMs kicked in, you know, which are three languages that I think are beautiful languages in their own way.

Neal (43:52)

Well, f

ish. Go has some some some serious issues around you know, ig ignoring what we've learned for the past twenty years about threading and building a brand new language that still has deadlocks is kind of stupid as a rock. But, you know, outside of that, I mean syntactically it's very pretty, I guess.

Carter Morgan (43:53)

Yeah.

Mark Richards (43:53)

It is.

Yeah.

Well,

Neil has this great, we both teach a class on the architecture as code stuff that we're currently writing about. And Neil has this great slide showing the evolution of abstraction. And I just love it because coming back to the Mythical Man Month, yes, if we're on Honeywell assembler language, how, or doing IBM assembler,

It was so low level, we abstracted all of that low level pops and pulls and pushes and all this kind of stuff and peaks and registers over to an actual programming language like COBOL that we can actually use human words and paragraphs and sentences and periods. And that really advanced us because of that level of abstraction. And then we got C, which was a little more sophisticated, but man.

mallocs and memory allocation and storage was just so buggy. And it took pretty good level of sophistication. And so then we have Java and C Sharp, which abstracts us from a lot of that memory management. It's like, things got easier. And then we have 4GLs, which abstracted of saying, look, I don't even need to write this code. Look, all I need to do is specify it and it gets written. And then we saw what happened with that.

But this evolution continues to the point where what we're seeing or predicting now is just based on our prior conversation. I really don't care what technology frameworks or languages you use. Use them all if you want. I need this thing to work. And I need to work by next week. Well, should we use Go? I don't care. And so if you think about these boxes we're surrounding it in,

Carter Morgan (45:54)

Ha ha.

Mark Richards (45:58)

Now we're at almost the business level of abstraction where the business says, this is the behavior I want, make it work and generate the code. I don't care what you use. And we're starting to get to that level of abstraction now. So Nathan, when you were talking about using Go or Python, it's just like, does it really matter anymore? I mean, I was...

Neal (46:21)

matters less

now than it used to. And in fact, every environment's polyglot now because it's just easier to to do that way in in a lot of cases. It's just it's just go polyglot.

Mark Richards (46:26)

It is.

Carter Morgan (46:27)

Right, right.

Mark Richards (46:31)

Well, and in some

ways, that's actually fairly powerful because certain languages, the reason languages exist is because they have certain superpowers. we select a language and maybe it's for that particular superpower of maybe scalability with Scala, for example, and it's parallelism. Yet that is only one fifth of the entire system.

The rest of it we have to try to use Scala for and it's not really meant for that kind of high level language stuff. Whereas an LLM may be able to say, well, let's see for this area here, I'm going to be, it needs to be maintained. I'm just going to use mainstream language like Java. Here, I'm going to add some Scala. I'm going to put some go over here to get some threading. And anyways, that would be pretty cool if it got to that level of sophistication.

Nathan Toups (47:01)

Right.

Carter Morgan (47:01)

Right.

Neal (47:18)

The

the big caveat from what Mark is talking about is one of our new favorite characteristics of code, meta characteristics of code, which is ephemerality. How ephemeral is this code? If it really is just gonna last for two months and get thrown away and replaced by something else, I literally don't care what it is. I don't care about unit tests, I care about functional tests, I don't care about the structure, I don't care what it's written in. But if I'm trying to build a foundation for something I'm gonna build on for 10 years, I gotta care about all that stuff.

Carter Morgan (47:33)

Right, right.

Neal (47:47)

There was another interesting thing. So Mark and I recently got to hang out with a whole bunch of the members of the closure community. This gathering we'll go to once a year. And there was an interesting perspective on closure, which of course is a the Lisp hosted by the JBM. There was an argument being made, and I can see the the merits of it, that LLMs would produce better closure code than Java code because the amount of closure code that's already out on the internet is

Pretty high quality, and you cannot say that about the Java code that's out there. And the JavaScript code is absolutely terrifying. and so having a much smaller platformer tech stack may give you richer results because it's not, you know, talk about going to the mean, reverting to the mean. The mean quality of JavaScript code, do you really want that in your project?

Carter Morgan (48:39)

Right.

Mark Richards (48:39)

Yeah.

Nathan Toups (48:39)

Yeah.

Carter Morgan (48:44)

it's yeah, and I think that's what makes you all of your work in kind of architecture really even more pertinent these days. You know, when we talk about like junior programmers, I I I don't know if I'm just really lucky. We have exceptionally bright juniors at my company, just you know, the best of the best. And they were asking me, they were saying, Okay, so you know, you started ten years ago, right? Like what was different pre L LMs? I said, I just remember being stuck a lot more.

Like I I remember like sitting down at my job and being like, I feel like if I were, I was explaining to my wife, I was like, I feel like if I were a journalist, I might be like, Well, how do I best format this article or what's the best lead here? But I don't think I would be like, how do I open Microsoft Word? Right. And and I remember that a lot as a junior engineer, just being like, I don't even know what I'm supposed to do. I remember like following YouTube tutorials and being like the moment it diverged from what I was doing, like, well, I'm lost, right? And so LLMs, I think.

Mark Richards (49:28)

Yep.

Carter Morgan (49:41)

And again, maybe our juniors are very, very bright. I love that for them because they're not stuck nearly as often as I was as a junior engineer. And I I don't know, like I I think when you talk about like people, you know, well, how are we gonna have seniors if we don't have juniors? To me, I think people have it a little backwards because I think if I were starting a a software company, you know, 20 years ago, fifty 10 years ago, when would I start hiring a junior?

Mark Richards (49:48)

Mm-hmm.

Carter Morgan (50:07)

You know, by the time I I I'm raised in a series A, a series B, maybe, because juniors have just been net drains on productivity for a long time. Again, and maybe our juniors are just very talented, but I've been very, very impressed with what they've been able to accomplish with Claude, because they've just got a, you know, a pair programmer with them all the time, right? And as long as they're kind of pointed in the right direction by their their senior colleagues, like I've been really impressed at how effectively they've been able to move. And so I'm like, I don't know.

Part of me thinks that the economics of hiring a junior engineer have never made more sense because, you know, you just equip them with, you know, a couple hundred bucks of tokens each month. And then all of a sudden they've got someone who can kind of help steer them, you know, when they get stuck.

Neal (50:48)

Yes, but so I always like to say that LLMs are a magnifier. So let's say you're already five X and the average developer and and the LLM is a ten X magnifier, that makes you fifty X. But if you're zero, zero times ten is zero. And if you're negative fifty, negative fifty times ten is negative five hundred.

Carter Morgan (50:50)

Yeah, right.

Right, right.

Yeah.

Mark Richards (51:07)

So.

Carter Morgan (51:09)

Right, right.

Neal (51:13)

So that's

the problem you run into is that there has to be at least some knowledge there to start to know what to do and then get guidance rather than just, you know, throwing spaghetti at the wall and seeing what sticks. So and I I don't think that's what you were suggesting, but I think a lot of people, when they think about this, is I can just hire any bartender off the street and you know, get the and give them a claw license and some tokens and you know, we'll build the next enterprise software stack. And that's

Carter Morgan (51:26)

Yes.

Yes, right, right.

Mark Richards (51:36)

Yeah,

Nathan Toups (51:36)

Mark Richards (51:36)

yeah.

Nathan Toups (51:36)

I I think this ties into two things though, with like the the the great thinking and systems book, right, which talks about feedback loops. And I think that it is that amplifier there. The other one I think about a lot, and we've it's come up a couple of times on the podcast, is the the Gell Man amnesia, right? Are are you familiar with this? right. The the idea is that you read a newspaper article about yourself and you realize they get a few details wrong, right? That it's not not not the end of the world, but yeah, that you know, I did I didn't actually graduate that year or whatever.

and then you read the newspaper about something you're less aware of and you're like, this is great source of news. Like I, you know, I I just learned about this event. And I think this is like a really good reminder to ask Claude a question about something you have deep knowledge about, and then realize you're like, that's almost right, or that's crazy because it sounds so convincing, but it's not quite. And then you do it something on like, you for me, I'm not a JavaScript developer and I'm like, what's this pattern that I should use for this? And it gives me an answer. I'm like, that sounds awesome. Like, I, you I'll just like, I'll, you know, look at it.

Carter Morgan (52:11)

Yeah.

Right.

Neal (52:23)

Exactly. Yep.

Mark Richards (52:25)

Yeah.

Mm-hmm.

Nathan Toups (52:34)

And

I think a junior dev maybe, you know, aesthetically doesn't understand what they don't know. And they look they have this like what feels like a super tool and they very confidently make a bunch of decisions. and so I do think that like the organization itself has to have good feedback loops so that they can like handle the fact that Gale Man amnesia can actually like be exacerbated with a junior dev.

Carter Morgan (52:41)

Right, right.

Mark Richards (52:57)

Yeah.

Well, it's interesting. We talk about a junior dev and one of the things we haven't done in this podcast is really define what a junior dev is because we can take a junior dev right out of college that can probably code and not probably probably can code in any language better than I can. And so in our, as you remember from our fundamentals, software architecture book, we defined the knowledge triangle.

Carter Morgan (53:16)

Right, right.

Mark Richards (53:25)

where the peak of that pyramid or triangle was the stuff you know. That's the stuff you're an expert in, is what you're getting hired to do. And then the second level was the stuff you know you don't know. In other words, things you're familiar with. And then that third level, things you've never heard of before, the stuff you don't know that you don't know. We define a junior developer as having that peak of stuff you know. And they know it very well. mean, junior developers, I would say I know

very many junior developers that I would consider experts in certain programming languages. But the problem is they've got very, very, little area in that second one, the stuff you know you don't know, and that third area is even bigger. So if you visualize their triangle, it's only at the top. So really, that junior developer is exactly as you guys are describing is

Nathan Toups (53:58)

Mm-hmm.

Carter Morgan (54:08)

Right, right. Yeah.

Mark Richards (54:22)

It's the stuff I don't know that I don't know and the stuff that I'm not familiar that's out there that starts transitioning you into that senior developer where you start accentuating or building those levels of your triangle. So I think there's a misnomer here that we tend to think about a junior developer as being somebody who doesn't know very much about coding or know very much about software. And in fact, they do.

Carter Morgan (54:50)

Interesting.

Mark Richards (54:52)

but they don't know about their surrounding environment. That's the big difference. And Neil and I commonly refer to the quote, the adult in the room. And regardless of what level we're using LLMs, you still need the adult in the room. And that could be a junior developer that's initially guiding the LLM, but they're saying, okay, we got to get started here. I kind of know the algorithms I want to use and the ideas. Let's get it going.

That could be an adult in the room or the architect who is now saying, yeah, it's not all about behavior. It's also about the capabilities. If you don't need any capabilities, go wild. Vibe, spec, do whatever you want because the ephemerality is probably not much or it's just a single user product or system or a timesheet approval app. No, you don't need an architecture for that.

Carter Morgan (55:44)

Right, right.

Neal (55:47)

The the one of my favorite stories of the

the turn of the year, somebody was talking about yeah, I vibe coded this awesome fitness app for myself last year and then I realized when it turned twenty twenty six that it had hard coded the year twenty twenty five into the entire

Mark Richards (56:00)

Hahaha

Neal (56:03)

That's

not having an adult in the room looking at what you're producing. It's like, okay, let's let's make a fitness app that's better than just one year.

Carter Morgan (56:05)

Right, right.

Mark Richards (56:06)

Right. Yeah.

Carter Morgan (56:10)

Right.

Mark Richards (56:10)

But

coming back to the very beginning of the podcast, was kind of one of the drivers also for the Hard Parts book. It wasn't just all the leftover stuff from the fundamentals that we couldn't fit in. It was really more about, so in the fundamentals, we kind of taught you about all the various aspects of architecture, even all the way through the soft skills. But now,

Now let's dive a little deeper and teach you how to do trade-off analysis. And that was really the key point of that book. Through all the various examples, whether it be the SysOps or the topics we chose that were hard decisions to make, it was really, how do you make those decisions? And that's what the book primarily was about. It was about proper modern trade-off analysis or analyses.

Neal (57:03)

Well that and

that's what we figured out the narrative arc ultimately of that book was when we looked at the big pile of problems, it's like what is the common thread? And it was the the difficulty of doing trade off analysis.

Mark Richards (57:06)

Yeah.

Well, one of the other amazing things, I was hoping we'd talk about, but we can't because I know we've got another minute or two, was part of the whole narrative arc that we also put into that was pulling things apart, which was part one, and then part two, putting them back together. And that's really clever when you think about dismantling systems and moving to levels of modularity, whether it be through migrations or just kind of teasing apart larger systems.

that have gotten too big, we realized there's techniques in ways, which was part one, for actually pulling things apart and teasing apart these things. But then it's so clever because once you pull them apart, you got to stitch them back together. And if you think about just blowing up the monolithic system into a whole bunch of services, well, they got to communicate. The data has to all communicate.

Nathan Toups (58:06)

Mm-hmm.

Mark Richards (58:09)

And so we kind of saw it as two parts. Well, we kind of wrote all of these things and Neil, we were off by how many pages?

Neal (58:19)

Well, so w we created this split early on and then sort of ignored it. and then when the book was done, out of four hundred and fifty pages, part two ended on page two hundred and twenty two. So it's almost exactly halfway. We did not plan that, but I mean it was so close it was it was eerie.

Mark Richards (58:22)

So funny.

Carter Morgan (58:29)

No.

Mark Richards (58:30)

It almost, it was amazing. We

didn't plan that. I we just said these are the topics for part one, these are the topics for part two. But yeah, was literally about six, seven pages from being exactly in the middle of the book.

Nathan Toups (58:49)

That's so cool.

Carter Morgan (58:50)

Well, I wanna

we we could talk to you guys for hours and selfishly we would. but we want to ask you about the books you'd recommend to our audience. But before we do, I just gotta say, you guys did help me look really cool at work because we were talking about we're on a monolith right now. We were talking about, hey, if we were to grow, what would the boundaries be? And I was saying, you know what? I read this this interesting book lately that recommended if you're trying to decompose a monolith, just copy the whole thing and start ripping things out until you get what you need, right? And he was and my coworker was, he was like, I have

Neal (59:14)

Yep. Tactical forking.

Mark Richards (59:17)

Tactical forking.

Carter Morgan (59:18)

He's like, he's like, I had never thought of that before. He's like, what an interesting pattern. And I was like, you know what? If you read Software Architecture the Hard Parts, you too could know this.

Mark Richards (59:23)

Neal (59:26)

Ha ha ha.

Mark Richards (59:31)

Well, you know,

another book which I would recommend is Jackie Reed's Communication Patterns for Developers and Architects. It's a fantastic book because it's so different than all the other books out there. And it really deals with different patterns and techniques and ways of communicating verbally, visually, and...

Carter Morgan (59:36)

Yes.

Mark Richards (1:00:01)

It's things that we often overlook. And as all of you know, on the podcast and the audience, I'm a big proponent of soft skills, especially within software architecture. It is, not might be, it is the differentiating factor and it is how you keep a job. And this is just a great follow on to it. So that's one book that I would recommend without hesitation. It's just...

Carter Morgan (1:00:13)

Yes, yes.

Mark Richards (1:00:30)

full of some really great techniques that might seem obvious, but we never do them.

Carter Morgan (1:00:35)

Okay.

Well, fantastic. what about you, Neil? Do you have anything you recommend to our audience?

Neal (1:00:42)

Yeah, so I'll recommend a couple. We mentioned it before Team Topologies. we for years we hand waved around all of these really important things that we we knew in our gut were really important, just didn't couldn't articulate it. And then Skeleton Pias came along and articulated it beautifully in Team Topology, so that was really nice. We actually, in the second edition of our fundamentals book, added a section for each of the architecture styles as to how team topologies might impact the use of that style.

Carter Morgan (1:00:45)

There we go.

Neal (1:01:09)

So it's a it was an influence on us for the second edition of our book as well. The other book I'm going to put out there is I'd be shocked if anybody actually picked this up and read it, but it it is a fantastically written book. It is a book that is about opera and philosophy. It's called The Tristan Chord by Brian Maggie. and it is absolutely so well written. He's a great writer. He's a philosophy writer at heart, and he's such a good writer. And that book in particular is just extraordinarily well written.

Nathan Toups (1:01:10)

Smart.

Carter Morgan (1:01:25)

Okay.

Neal (1:01:39)

It's has some fascinating stuff in it too, even if you don't care anything about opera but care about music and the and the history of music. There's a lot of interesting stuff in there about that. So that's an easy recommendation.

Carter Morgan (1:01:51)

Okay, there we go. You can hear Nathan typing, so we we do take these recommendations seriously.

Mark Richards (1:01:55)

hahahaha

Nathan Toups (1:01:56)

Yeah, I was like I was throwing I was like I

like these are going on we have a public backlog now, that that we of all the books that are in our queue. So yeah, this is helpful.

Neal (1:02:00)

Mm-hmm.

Carter Morgan (1:02:06)

Well, Neil, Mark, thank you so much for coming on. thank you for coming on again. I I would I yeah, like we said, we we have had you on several times 'cause we we love your work and we're gonna keep reading it. And so what's the name of this next book you're working on?

Mark Richards (1:02:08)

Yeah.

Neal (1:02:19)

So the one that's almost done now, if it doesn't kill us in the process, is software architecture patterns, anti patterns and pitfalls. that's on early release on O'Reilly right now. And then the one that we are frantically working on, which is our perspective on L L and how to teach agents architecture is architecture as code. Also on O'Reilly, also in early release. So

Carter Morgan (1:02:23)

Yeah.

Okay, great.

Okay. very cool.

Okay, great. Well, we will absolutely read those when they come out. And we'd love to have you guys back on to talk about them. And thanks for come both coming both on at the same time. It was such a pleasure to have you both and and hear both from you. well, you can always find our work at bookoverflow.io. You can contact us at contact at bookoverflow.io. I'm on Twitter at Carter Morgan and we're on Twitter at BookOverflow Pod. And Nathan has work with his consulting agency is at rojo roboto.com.

Mark Richards (1:02:47)

yeah. Yeah.

Carter Morgan (1:03:04)

Neil, Mark, we can't thank you enough for coming back on. And again, thanks so much for all the quality content you contribute to the world. And we are rooting for you to get that money from Anthropic. It's well deserved.

Neal (1:03:15)

Well, thank you for

having us and

Mark Richards (1:03:16)

Yeah, thank you.

Neal (1:03:16)

and a double thank you for being people who still read books. So you're a rare and disciplined breed and we we that's our favorite people. So

Mark Richards (1:03:19)

Yes.

Carter Morgan (1:03:22)

We we really believe it's

so important. And I think I think that's on our websites. Like in a world of it we're dominated by short form content and engaging with long form ideas is so important. and even, you know, we've read books where we're like, you know, I don't really like this. I don't I don't agree with this. And but even the fact that it it's so important to engage with an idea you don't like over sustained period of time rather than just reading a hot take on Twitter or whatever. so thank you for.

Neal (1:03:48)

Absolutely.

Carter Morgan (1:03:50)

doing the hard parts, right? And and writing all these books. it really is so valuable. And we're we're trying to evangelize it all the time. We're like just read books. You want to level up your career, read books. Like it's crazy how quickly you can kind of vaunt from 50 percentile talent to 90th percentile talent just by reading some books and and applying it. So, to all of our listeners out there, if you want to get a jump start on that, anything by Mark Richards and Neil Ford, that is the place to start. And

As evidenced by the winner of the the book overflow March Madness bracket. But who knows, maybe we'll do another one this year and y you gotta watch out. You can never be too secure in your victory because we're reading great books all the time. There we go.

Neal (1:04:22)

That's wild. All right. I'm n never secure. I'm I'm all I'm always just happy if it happens. So I

Mark Richards (1:04:28)

You

Neal (1:04:31)

as hey, as if we're included in the list of those books, I'm pretty happy about that. So it was just the icing on the cake that we ended up

Carter Morgan (1:04:35)

Yeah.

Well, fantastic. Well we'll see you folks later.

Mark Richards (1:04:41)

Okay, thanks. Bye-bye.

Neal (1:04:41)

Right, thanks.