Chris Keathley
The Elixir Outlaws now have a Patreon. If you’re enjoying the show then please consider throwing a few bucks our way to help us pay for the costs for the show.
Episode Transcript
Amos: Welcome to Elixir Outlaws the hallway track of the Elixir community.
Amos: Hello.
Chris: How's it going?
Amos: Good. Yeah. I told somebody yesterday that I feel like I'm on a really dark version of Gilligan's island where I'm around the same people constantly. And nothing's funny and it's mostly just hateful.
Chris: So it was Gilligan's Island. I don't remember Gilligan's Island being that funny,
Amos: *inaudible.
Chris: What now?
Amos: I'm ready to get out of the house. I'm starting to get a little-
Chris: A little crazy?
Amos: I went over to Kansas and went for a walk and, uh, uh, there's a tall grass prairie national park. It's just like trails in a giant open prairie, so you can walk anywhere and never have we pretty much. I think we saw a couple people all day and the parking lot was packed, but you don't see anybody.
Chris: That's cool. All right, I'm ready. I'm ready to engage.
Amos: I'm ready. You're ready.
Chris: I'm super not.
Amos: I'm engaged in a hot conversation, uh, on the Kansas City Elixir Slack with one of your coworkers.
Chris: Uh, which one?
Amos: Jeff Utter
Chris: What about?
Amos: Uh, friend of the show Jeff-
Chris: Oh, is this a show?
Amos: Yeah, I think this can be the show.
Chris: Yeah, I have to, I have to eat right now. Unfortunately, cause this is lunchtime, but I also, you know, like I got, I got an interview thing I'm doing with a different podcast thing, right? Like immediately after this, I got I'm meetings all day.
Amos: I'm doing another podcast today too. And a meeting right after this.
Chris: Are you? That's amazing. Yeah. I have a meeting immediately after that, so I don't know when else I'm going to eat lunch. So I'm eating lunch through this, but I figure at this point, no one expects any sort of quality from the show, so it'll be fine.
Amos: Right. We're just going to eat up.
Chris: We, really we've set the bar so low that it just doesn't matter anymore. Like we are now any sort of quality we bring to the table is, uh, that's just a bonus. That's a cherry on top of this Elixir filled sundae.
Amos: We're probably going to talk about random BS outside of Elixir and maybe complain about something, but offer no real solutions.
Chris: Classic.
Amos: That's the goal.
Chris: Yeah. We should just rename this podcast, "Two White Guys Talking"
Amos: And Sometimes Anna. Um, so J so there were some people talking about telemetry and things like that. And, and Jeff brought up, um, why would you call telemetry from your application? Like telemetry should be called in libraries, like Ecto, Phoenix, HTTP clients, things like that. Why would you ever call it in your library? And I was like, well, I call it in my code.
Chris: Do you use telemetry in your app for application-level stuff?
Amos: Yeah. Well, yeah. I mean, so, so camera system, I know it's been brought up a lot, but we, we track like frames per second, going through the system and error rates from cameras, like when they send us bad frames and stuff. So we can kind of get health of hardware and overall system as part of what's going on. Now, we're also using, you know, some there's Phoenix in there cause there's a front end. So we get some of those too. But like the main part of our system is, is dealing with the camera stuff. So we have lots of things that we collect around that.
Chris: And you export that all you, you export all of that through telemetry and then you have handlers to dump it somewhere to do something with it.
Amos: Promethease.
Chris: Okay.
Amos: But we are calling telemetry in, in our application or sometimes telementry metrics, but you know, that wraps, some of that stuff, but for the most part, yeah, we call in telemetry, right in our application. And I, I, so I was very confused at, at like why somebody thinks. Yeah. And so, what's your take on it?
Chris: Well, so I mean, part of its probably context dependent, which is Jeff, as you know, a coworker of mine and friend of the show.
Amos: And an awesome guy.
Chris: And it's just a generally all-around great guy. He's a great musician. .
Amos: Yeah, I was going to say he plays bass.
Chris: You want, here's the thing, is, you want a Jeff on your team. You can't have this one he's mine, but you want to Jeff on, on your team. You need one of those. Just-
Amos: If you don't have a good bass player, like how are you ever going to get the band together?
Chris: The bass player is, you know, the fulcrum upon which you place the lever.
Amos: Right.
Chris: Right. So you need a Jeff. So, but I think a lot of its context dependent, right. And it also has a lot to do with the fact that for our services, we use statsD metrics everywhere, which means that, you know, broadcasting stuff ad hoc is kind of just like what you do with statsD but you can just sort of do that. You don't have to like pre set up anything, which is slightly different than Prometheus, right? Like you, you kind of have to say like what your buckets are and like what you're going to, you have to kind of set up the metrics right. In the same way that you have to set them up for telemetry. You know, you, you have to add the handlers, the handlers have to do specific things. They have to take those actions or whatever. For sheer convenience and, you know, ease, we tend to just use statsD directly. We have a fork of statics that we use and we ha we also have like a custom, we have our, we essentially have a giant custom metrics, tools, tool set that we built that, you know, adds generic telemetry handlers for all of the, all of the like libraries that we use out there in the world. And it does. And it sends like unified metrics for all of our services. So like all the services are , look the same. It's essentially like a unified metrics collection. So you can just configure it and say, you know, listen to listen for Ecto. Here's my repos, here's plug here's Phoenix. Here's, Redis like, whatever, uh, whatever, the different things that we care about are the export telemetry. And it hooks all those up. You just say like, which ones you want. And then that's all. And it's just, you just say that, and then it'll immediately start sending out like APM. So we get tracing, we get distributed tracing for it all, and we get a time series for all of it. But we also, like I was saying, we use StatsD under the hood. Cause we're using Datadog. We vendor all of our stuff there Datadog, and that's not likely to change anytime soon. Like we'll probably never change that at this point.
Amos: Come back to that. I would like to know why Datadog, but go ahead and keep going.
Chris: Oh, sure. I mean, yeah. There's not a ton of great reasons. It's just because, but because metrics are hard. Cause because like, I don't want to vendor that stuff and I don't want to run Prometheus, like -
Amos: Right.
Chris: You know what I mean? There's not that not having anything wrong against running Prometheus, it's more like operationally, you know, metrics gathering and tracing, trace gathering is actually hard. Like you need a team to do all that. Right, right. Um, so we just vendor it. So it's like one less thing we have to worry about. In any case, so that's part of the context is that because we use statsD it's really trivial just to export whatever time series, like whatever metrics that you want whenever you want them in place. And there's not a lot that telemetry buys you, if, like there's not a lot, that's elementary gets you, that you don't get by just doing like statsD in place, basically . Like, you know, you can either call a telemetry handler or you just call a statsD metric. Which one, which one do you want to do?
Amos: So it's just giving you a common, telemetry is just giving you some kind of common interface with all of your libraries and everything.
Chris: Right. And we definitely use it. I mean, I definitely use it in all my libraries. We definitely like use it for, we even use it for internal libraries, right. But we don't use it at, like, the application layer just because it's one more step to get metrics out and we just don't really need to. And so, and if all you're to do is like, just call it telemetry handler, which turns around and calls it a histogram, like why not just call the histogram.
Amos: Yeah.
Chris: The benefit to doing it, to, to using telemetry in your application, as you can conceivably, in my opinion is you could conceivably like do two things, right. You could send out multiple time series metrics. You could, you know, do tracing and time series. You could do logs. Like you, you know, obviously it's like, it's maximally flexible at that point. But, you know, I don't know. I don't find that to be that imperative. Like I don't find it to be that necessary . Often because like we just do that stuff and we just like control all that stuff when we want it, right. But there's, non-trivial costs to adding like observability to things, right. And being able to selectively do that in place where it matters is kind of nice. Um, and like all you’re really all telemetry would really do for you at that point is add in direction.
Amos: Right.
Chris: With no real benefit, unless you just really want, I mean, it does clean up everything. Cause like, you know, just adding tons of observability everywhere is like a very quick way to clutter up your system. That's a very, very, and but you do need that stuff. Like you, you super need observability having it like everywhere, which is kind of where you want it or at least around like key things, right. Like it can make stuff really messy.
Amos: How, how do you clutter it less with a call to statics versus telemetry? How's it, how's it less cluttered?
Chris: Oh, I don't know. I don't, I'm actually not arguing that it is.
Amos: Oh, ok.
Chris: Um, I think it's a hundred percent ease. It's just its ease of use. But I also think part of it's that a lot of the conventions for how we're using telemetry now in libraries are not well understood by a lot of people, like there's maybe 10 of us who do like fully understand, like, not even fully, but like, you know, who understand the conventions that we're using, uh, that are being used in libraries now, you know, talking about spans and like that sort of stuff.
Amos: Like key names too.
Chris: Yeah. And like how you name things, you know, a lot of people still do it wrong. I mean, it just is. Cause it was like, everybody's trying to figure it out, right. And by wrong, I don't mean wrong, like, you know, you're bad for using it. It's just like, there are patterns that now that, that emerged early in the life cycle, which are now considered really annoying. Like one of them that we've talked about a lot is, um, the idea that, you know, you can configure the pre telemetry prefix for stuff. Uh, what that means. So in telemetry, you're right, you have the, you have a list of stuff that you like, the key is like a list of stuff. A list of atoms, right. And it's like, this is, this is the event essentially that happened.
Amos: So you have like Phoenix requests start. I don't even remember if that's one of the Phoenix ones or not, but-
Chris: Yeah. It doesn't matter. But, but, but exactly right. You know, Phoenix request start, Phoenix request stop, right. And they have measurements attached to them and all this kind of stuff. Well, there's, there's thing where, you know, you need to be able to differentiate sometimes between let's- so this happened in Ecto, uh, it also happened in Plug, but you need to be able to differentiate between different repos potentially in Ecto, right. And they both need to be able to send metrics out. They both need to, or rather they both need to go send telemetry events out. Uh, but you need to build a know like which repo it came from. And so the way that that was solved at the time is the prefix is customizable. And my prefix, I just mean like the first part of the event that gets admitted is customizable. So instead of being Ecto, you know, query start Ecto query stop, it's your app repo name query start your app repo name, query stop, which was well-intentioned and, and is fine, you know, for, it's not like a huge deal, right. But it turns out that if you, what you want to do is build a generic way to capture all metrics, like all telemetry events, like moving in the system, that gets really annoying. Cause it's another thing that like the, that the user has to configure. Like they can't just say do it, like they have to explicitly tell you like what the prefixes are going to be or what your repo names are, like, you need a way to get back to the events. And there's other discussions in other libraries where people are sort of like, does this, like, we're going to, we should do this, right? Like, cause that's what Ecto does. A lot of us who've had to build these libraries, I know, I know other folks who work at, um, APM companies and stuff like that are sort of saying, yeah, it's like really annoying if you do it this way, please don't. It's like a lot easier if we can just selectively choose what we want to listen to on our end. Um, so stuff like that, that's all still emerging, right. But that's not super well understood. And I, I'm not surprised that people don't quite understand it cause you have to kind of be paying a lot of attention to it. And you know, I think because of that, not a lot of people are using it in apps generally because it is kind of like an extra step that doesn't always, it isn't always clear, like why you, why you care about it or what you want to do with it or how to use it or how to use it even the most like how to use it most effectively.
Amos: Yeah.
Chris: Which is still a thing that I think everybody's learning, you know, that's a, that's a shared thing that lots of people are, are learning about. And that's not unique to anyone really.
Amos: I guess for me, a lot of it was at the beginning. We didn't know if we wanted to use Prometheus or statsD or any of that and telemetry kind of gives you a little bit of a plug and play for that with the handler, like you just throw, throw whatever handler you want, or you can have both of them, like you said, and unlike, unlike databases, uh, I think it actually is fairly switchable between the two, depending on, I mean, you're, you're, you're endpoint like ultimately where you're looking at things is going to be very different, but, and you have to reconfigure all of that. But I think ultimately like whether your handler's dropping in the Prometheus or statsD is more, more switchable than I'm going to move from SQL to PostgreSQL or-
Chris: Yeah, I think I probably agree with that overall, you know, like different databases are not interchangeable and really any capacity, unless you use the most, you just like super don't care and all you've ever used is the very basic features of your form of choice.
Amos: Right
Chris: Right, uh, at which point you're leaving, I mean, in my opinion, you're leaving so much on the table. Like if you just use Postgres like MySQL, like you're leaving so much money on the table.
Amos: If you just use MySQL, you're leaving so much on the table.
Chris: Well, I mean, it's just, you know, yeah. I don't know. I think the whole, I think the whole, the, whatever that, you know, what is it D it's DDD that, uh, the whole, the, The Big Blue Book is the one that tells you that databases don't matter, right? Yeah. That book is dumb. That book is wrong. That book is a hundred percent wrong . Databases super matter. And anyone who tells you, anyone who tells you they don't matter, isn't using a database that much, or isn't using it for anything. Interesting. I don't know. Like that's just a, that's a ridiculous statement on it's, on its face, whatever. Like you have to just completely not paying attention to think that, but-
Amos: So, I do have to say that, uh, Jeff, Jeff brought this up, but Jeff was speaking to Samuel Mullin about telemetry is where this came from. I don't want to leave Samuel out of it because he's a good friend of the show too.
Chris: Oh, cool.
Amos: And you should go read all his blog posts. Um, anyway.
Chris: They're good.
Amos: They are good.
Chris: But yeah, no, I mean, I think I do, I will argue that there's a pretty big, big difference between Prometheus and statsD and it's, I mean, beyond how they work, right, operationally, which is the most obvious one, right? Push vs pull. There's also some semantic differences. Well, not semantic, but there's also some differences that you need to be aware of in terms of like granularity, you know, certain message just going to get dropped more often, et cetera. Like there, there are big differences between them, but I do take your point, which is like being able to swap that out is probably less a big deal than moving from MySQL to Postgres.
Amos: Right. So I see it as a benefit there.
Chris: Yeah. I think if you're doing it, I don't think you're wrong. I'll say that. If you're using telemetry in your application to get observability and metrics and time series and all this stuff out of your application, I don't think telemetry is wrong. I think it's a reasonable way to do that. Especially if you're familiar with certain patterns like using stop and start and exception events. Those are, that's killer. Like once you have that, you can really do anything you want. So that's a pretty major improvement. I think that's reasonable. I don't think you're wrong for doing it that way. I also don't think you're wrong, if you just, you know, call statsD metrics.
Amos: I didn't think he was wrong. It was just the,
Chris: Well that's what you said. You said that to me off air, you were texting me and you're like "Jeff is full of crap. I don't even know what he's talking about"
Amos: Jeff is so dumb, man. He's not even that good at bass. No, I'm kidding. I'm joking. Very big joking. Uh, Jeff teaches too. So he's, he's amazing. I'm sorry, Jeff. Now I now I'm backpedaling. I, I apologize. Is this the show? Did we start yet?
Chris: Yeah, the show is started. This is the show. This has been the show.
Amos: Um, yeah. I didn't mean to trash his bass playing skills. I've never actually heard him play. I would love to though. I, I wish he would just show up at one of the MeetUps with his bass. But I think, uh, carrying a standup bass around is a little bit rough.
Chris: I suspect so. I've never done it. They seem big.
Amos: They do seem big.
Chris: It seems like a big boy.
Amos: I don't even think I can pick up. I'm kind of weak. My thing was just when he's like, "Why would you do that?" I was like, "Why would you not?" Like, I didn't even know, like it was so out of left field for me that I was like, what are you talking about? And I could not understand what his question really was.
Chris: Right.
Amos: So I think hearing from you, I did it, this happened like right before we started the show. So I didn't even get very far in with him. Otherwise we pr, we might have a different discussion here.
Chris: Right.
Amos: But I think hearing from you how you're using it makes sense to me.
Chris: I think that's the big, that's the big thing is it's just, you know, it's, it's ease at some level.
Amos: Yeah.
Chris: And that, that is kinda the best thing about statsD I think that's the thing that made statsD so popular, right? It was maximally easy to get metrics out of your system in a world where that actually was non, pretty, non-trivial. Like at the time that was hard or, or just definitely more arduous than, you know, send a UDP packet to this, to this spot. And it'll, it'll like work.
Amos: Just let it go. Yeah.
Chris: You know, that was like, you know, you, you did Java, do you remember trying to get like JMX metrics out of, out of a JVM? Like,-
Amos: Uh, yeah, that was painful.
Chris: That was awful, right. And like, and a lot of systems had that same problem of this was just hard. It was just hard to get metrics out of the system. And I think statsD that it proved to a wide range of people that it didn't have to be hard and you could make good trade-offs about it and start, you know, observing your system. Uh that's always going to be useful.
Amos: And, and having custom stuff. The JMX was like, if you needed anything, that wasn't built in was-
Chris: Oh, it was awful. It was awful. I still, I still have, I don't really remember how to do it. I just know that at some point it works.
Amos: We worked on a secure chat server for the, um, I won't talk about who it's for, because I don't know if I can, if I want that recorded out loud. Um, but that's actually how I got into Erlang. but the server was written in Java originally, and I wanted to collect like message rates coming in and going out to different people. And I finally, it was just like, you know what, it's easier to actually scan log messages than to do this. So I figured out a log format and put it in a log. I only needed to do it for a short period of time. Um, it was more like a quick study of what would happen when we were, um, out using it. Uh, and I just needed a little bit of metrics and I was like, whatever we're done, I'm just doing, doing the right log format and searching for this stuff.
Chris: Yup.
Amos: And then I made a, made it another application that went through all that and gathered them up by people and kind of did tracing type stuff. Um, I didn't have a word for it then. And tracing was not my word. It was, uh, where did this message go and how did it get there?
Chris: Right, right. Yeah, exactly. Yeah. Those were dark days. The dark, those are, those were very, very dark days. All kinds of custom reg ex to like parse through logs and stuff, and it's all slow and you get overloaded, it's all bad.
Amos: Now, they're better.
Chris: Things are better. Things are better these days.
Amos: I also feel like another reason why I like to use the telemetry in my application is because then I have this common, um, I'm going to say language, common pattern, that when I'm looking at libraries that are using it, I'm looking at the same things. Um, and that makes, that makes it powerful for me. Uh, even if something else might be slightly simpler.
Chris: Yeah.
Amos: I guess I don't really find the telemetry stuff that I do that difficult. And I, I probably, for the most part, um, have all of my telemetry calls kind of wrapped in another module to a point like any of them that are complicated are all wrapped in some kind of other module. And I'm, I'm not really calling it directly in my application code.
Chris: Yeah. Yeah. I had did this for Finch. Um, cause we added telemetry pretty early to Finch and um, we have a whole module that we call that actually has all the like span semantics built into it. But you can just say telemetry dot start elementary desktop and that sort of stuff. And it knows what to do at that point. And yeah, I w I just built a whole module for it. Cause I , calling the handlers and whatnot is kind of annoying. Like, like calling execute on everything is kind of annoying. Honestly, the more annoying thing though is threading all the metadata and measurements through that you need. Cause sometimes you need metadata that lives at a higher level than the actual function that you want to trace or observe in some way. And so you have to pass down, uh, you have to have all these extra things that you pass through all the functions, just so you can send it out. Just so you can like observe it and get it out into, into your measurement somewhere, which is kind of annoying, but it is what it is. It's not too bad. It definitely adds clutter though. And even so like, even if you're using telemetry, you know, you, you're going to add noise to your system.
Amos: Right? Like that's one of the things, whenever, whenever I'm looking at a function of any kind, you know, I want to be able to step through and say, what are the important things in this function? What is it doing? And the telemetry stuff does add like, ope, there's a line that's not really doing anything for the logic of this function. I don't need it to understand this function. And I, sometimes I wish I could hide that away a little better, but I haven't, I haven't figured out anything great. Other than just say, yeah, that's telemetry moving on.
Chris: Yeah. I think I, you know, just modules and functions, help clean stuff up. It's always the same stuff. It's modules and functions. Just use some good models and functions. You'll be fine.
Amos: And functions with functions.
Chris: And then you can document it all too.
Amos: Yeah. So do you, do you do anything special to document your, um, like the way that you're doing your statsD stuff like what you're sending out? So all of our, well, not everything yet cause it's a new library that we wrote. We wrote a library called Hertz that that's what provides all of our common metrics stuff. It's not open source. It maybe, should be one day. Maybe I'll do that. But it's pretty like tailored to our use cases. So it's probably not useful to a whole lot of people who aren't us, but it has a ton of common monitors, uh, all of which, not all, but most of which are telemetry based and it has them for Fuse and Hackney, but, and those don't use telemetry, but, uh, we just fake it like they do. And, um, anyway, you, you configure all that stuff. And then it's, there's a giant list in the documentation of like, if you're using Ecto, you're going get time series. It's going to have, you know, Ecto dot, I don't know, whatever it is, like Ecto dot query dot, you know, duration, microseconds, uh, Ecto, Q duration, microseconds or whatever, and they're all labeled. And they also tell you all the tags that we're providing. So it's like, here's the repo name. And, uh, here's the amount of things that were returned and all that kind of stuff. So we have like extra metadata that goes with it, which is a big document. So we just have this giant document with all the like metrics that we export. And then they're all consistent. So for the services that use it, that use it currently. So you can have like, it's really a lot easier to have unified dashboards for stuff. Where not everything needs like this bespoke dashboard, um, for all of its things, you can actually kind of start to see how just generally how this service is doing. I'm not a huge believer in, I'm a big believer in having less dashboards. I'm not a huge believer in the idea that every service needs a completely cookie cutter, the same dashboard for everything, because I just don't think that that's realistic for services like services and have different use cases and different needs. And, um, and the people who are operating them need whatever they need, but it does help because you don't have to guess anymore at what the metrics are called, because prior to this, it was sort of like, is it, you know, cause none of it is, you know, cause it is all statsy and none of it's unified. It's just kinda like whatever anybody used at the time, because there's always questions about like, is this my service name dot Ecto dot query dot duration? Or does it have microseconds? Is it in microseconds? Like, is it a duration? Like, is this the hit rate? Like who knows? I mean, there's always those kinds of questions. And so just removing those kinds of questions, like reduces everybody's cognitive overhead. So it makes it much easier to get in there and find the stuff that you need when you need to find it. Uh, which is a pretty big benefit. We don't, at this point we we've removed, you know, we, we used to have like prefixes on the metrics based on the service name and we removed all that junk. Cause I don't believe in that.
Amos: I mean, when, if you want to see process times across all of your services, like average process and you're like-
Chris: Yeah. Yeah, exactly. So we have other ways to track the service. So to be able to select the service, using different tags and stuff. In Prometheus, they'd be labels, you know, those sorts of things or, you know, hosts labels or whatever, but we have different ways to talk about that. And I think that that's a lot more useful. We do still use application level metrics. And when we use an application level metric, like a business, like metric for that application, right, when that, when we do that kind of stuff, we do prefix it with the app name because it's, you know, it is your app's metric of some sort. And it's typically like a business related thing. Like this many users were deleted, this many users were added or whatever, that kind of stuff. So yeah. I don't know. So we have, so now it's all generic, which is nice. And then you can just kind of look at the docs and see what it does. The next step I think is figuring out how to codify all the different dashboards and stuff as actual code. Um, cause right now they just like live in Datadog and that sucks like, cause you know, you can't, they change a lot and people add stuff and remove stuff and they get gnarly. And if you happen to change a metric for your application and then forget to update the monitor and all of these kinds of things, it's very annoying. So I'm trying to think through how we would codify monitors and monitor like based on metric names and making sure that we didn't like change them and break that. That'd be pretty interesting. I think we can do that. I think we could, we can just build a library to do it. Um, we haven't done it yet.
Amos: What's the hardest part of solving that?
Chris: Uh, you need to, you need some way to like check it, right? Like you need some way to verify it. The names that you're using are actually what you still care about, which means you need some way to introspect the code and uh, see what monitors, like what metrics using and then double check them against the existing monitors or like have monitors in code basically. Which is just not a thing we have right now. They just live on the, on this service. So that's the next step. But it's down the list. There's a stack. There's a project stack and it's not at the top. I can tell you that right now.
Amos: It's not that painful.
Chris: No, no.
Amos: Or that fun, maybe.
Chris: It's, it's mostly that. I follow my muse, you know, I follow, I follow things that are fun.
Amos: I have the, the painful fun, like what solves the most pain but is the most fun to solve? And then I never get to it.
Chris: Yeah. Yeah. That's the hard part is, um, making the time.
Amos: Yup. And I thought I would, uh, I thought being an at home like this, like all the time and not really having to like take kids to different events and things would give me a lot more time, but it really doesn't because now I just spend more time entertaining children where before I might drive them to entertainment. Now I am the entertainment.
Chris: You are the entertainment.
Amos: Right.
Chris: Look at me. I'm the entertainment now.
Amos: Whether I like it or not. And they don't like it when I come and sit down on the couch and pull out a distributed systems book and just start reading. I am not entertaining enough when I do that.
Chris: Yeah. I mean, you know, it takes a while to really understand the elegance that is Paxos.
Amos: Right?!
Chris: You know, it's, it's, it's like looking at, it's like looking at the Mona Lisa, right? You get that it's beautiful, but you don't know why. And it takes a long time to really stare at it and really understand it .
Amos: Well and you can't get that close to it. And it's a lot smaller than you think it is.
Chris: Oh yeah.
Amos: I tried to, I tried to read Dune now out loud to them too.
Chris: How did that go?
Amos: Uh, my son's like, you, just gave it to me. Cause he can read way faster than anyone can talk out loud and it'd be understandable. And so I think it frustrates him.
Chris: He has prescient sight. From the spice.
Amos: Maybe I just told him whenever I finished reading it, he can have it. And that kind of lets him, he'll leave me alone for a little while, while I read it.
Chris: It's time to ride, to capture and ride one of the large makers, the Shai-Hulud.
Amos: See, I'm missing half of this, cause-
Chris: Have you never read Dune?
Amos: I read like half of the book when I was a kid and I never finished it.
Chris: What? WHAT? How are we friends?
Amos: I know, right. So-
Chris: I make at least three Dune references a day.
Amos: Well, I checked it out and now I'm halfway through it again. And this time I have more resolve than, than seventh grade me.
Chris: Yeah. Man. Tell you what.
Amos: I was into like really old sci-fi, like when, when I was a kid, I was reading like Jules Verne,
Chris: Oh, ok.
Amos: And like the War of the Worlds and stuff like that. Yeah.
Chris: An alien race so advanced that they attacked a planet that is literally made up of the one thing that kills them.
Amos: Yeah. Super smart. Yeah, so.
Chris: Oh no, that was Signs. That was Signs.
Amos: Signs is, Signs is-
Chris: War of the Worlds, in War of the Worlds, they just get it, like they get-
Amos: They get sick.
Chris: They get like a bacteria and die.
Amos: Yeah, yeah. But Signs-
Chris: Signs, it's water.
Amos: Is water? Okay. I couldn't remember. I was like, it was, is sound or water?
Chris: Again, an alien race so advanced that the came to the one planet that's literally made up of the one thing that kills them. Genius. I think it was Signs when I realized that M. Night Shyamalan was fallible. That was when my faith was really shook.
Amos: Oh my gosh. And then every movie after that just shook it more.
Chris: Oh yeah.
Amos: The thing is every movie that he came out with, I still felt like I needed to go watch and then I'd go watch it and be like, crap, he did it to me again. It was, it was just, just like, uh, the first movie, like you get, you get along through Sixth Sense and then you're like, whoa, I did not even see that coming.
Chris: You didn't see that coming?
Amos: Well. I made a joke about it being that way, but I didn't really believe it was that way. So like part of me, I guess knew, um, and my friend got mad at me at the end of it because I spoiled it by saying what if he's dead? But I was just kidding. Anyway. Uh,
Chris: I just remember getting to the end of Signs and being mad because I was like, wait, so God made asthma to save this one kid's life this one time. What about all the people who die from asthma? That sucks for them! Like I was so mad. I was so mad at the injustice of it all.
Amos: Well you go from that. You have Lady in the Water.
Chris: Dude. I literally didn't see any after I watched Signs. I was like-
Amos: Oh yeah, you didn't watch Avatar?
Chris: Uh, with the blue people?
Amos: No, no. Avatar: The last Airbender.
Chris: Ta- Tonka.
Amos: I see you. I see you. I see you. I feel like that goes right along with, "I have spoken."
Chris: What's that from?
Amos: Oh, it's the, uh,
Chris: I don't understand this reference.
Amos: Star Wars.
Chris: My pop culture references ended, like, I don't know, maybe I'm going to say close to 10 years ago.
Amos: It's uh, the Star Wars series. Um, Mandalorian.
Chris: Oh yeah, I hear that's good. I should watch TV at some point.
Amos: It's pretty good. It's pretty good. The only reason I have it's because Verizon gave me Disney Plus for one year for free.
Chris: Nice. My in-laws gave us Disney Plus for free, however long they're going to pay for it.
Amos: Oh gosh. Don't start watching Mandalorian.
Chris: Don't?
Amos: Well, actually it's only got one season. You'll just blow through it, like-
Chris: Well, I'm aware of that Baby Yoda is a thing that exists.
Amos: Yes. Yeah.
Chris: That's not a spoiler at this point.
Amos: No, no. It's just-
Chris: Really taking the internet by storm.
Amos: Interesting to watch. He has. He's like the meme among names now.
Chris: Came in like a wrecking ball.
Amos: I, yeah, I can type in anything in Giphy and I get a picture of Baby Yoda. I'm like, I'm like what? I typed turtles all the way down. And I got Baby Yoda.
Chris: Well, they know what you want.
Amos: And then I just say, yep, Giphy's right.
Chris: They have melange future sight from the spice, spice and completely blue eyes.
Amos: Yeah. So where I am in the book right now, uh, Paul just
Chris: Giamatti?
Amos: Uh, uh, yeah. Should I not give it away? Is it a spoiler-
Chris: No.
Amos: When a book is that old?
Chris: No, I'm sorry. I just, I didn't know where you were going.
Amos: They, they just went into the caves. Um, and somebody finally like, I mean, uh, you know, already, but the planetologist is Leit. Yeah. Yeah. And so he just shoved them back through the, uh, secret passageway. and that's where I ended.
Chris: Nice. Oh that book is so good.
Amos: You should read it again.
Chris: What do you- I, yeah, I did like, like literally like at the end of last year.
Amos: All right.
Chris: I've read that book a lot. You don't need to tell me to read Dune, okay. In our like local Papers I Love group, like it more or less devolves into Dune references almost, almost universally every time.
Amos: Here's what happened to me with Dune. I was reading the book and somebody was like, oh yeah, there's a movie about that. It's got Captain Picard in it. So I just went and got the movie from the library and watched the movie. And then yeah. I was like, okay, I got it. But now that I'm older
Chris: Yeah. Not, it's not a great interpretation of the book.
Amos: I'm like, oh, I really shouldn't finish that book.
Chris: It's interesting. The Sci-Fi miniseries was great.
Amos: I also never finished The Lord of the Rings series.
Chris: Yeah, but that's because those books aren't that good.
Amos: Oh! Got'em. Ha-ha!
Chris: The show would not be complete without, uh, first of all, a complete, without not talking about Elixir, but also without some sort of hot take, I actually don't even think that's a hot take.
Amos: I, well, you haven't sent me anything-
Chris: Those books are amazing. I mean, let's be clear. Lord of the Rings is an amazing book series, but God are they hard to read. ,
Amos: I did not read the last one, actually. I read the Hobbit and then the two beginning ones and never read the last one.
Chris: The Hobbit's the best one in my opinion.
Amos: I think that's what hooked me is The Hobbit. Uh, again, I was younger. It was easier to read. I flew through The Hobbit and was like, yes, this is awesome. And I picked up the other ones and was like, oh God, this is not so good.
Chris: They're all amazing. It's just that they're hard to read. They are actually hard to read.
Amos: They are very different reading levels.
Chris: Anybody who says they not as hard to read them as a kid and imprinted on them. And now in nostalgia wins out. Like those books are hard to read. Also the also there's a God-like character that just wanders through the woods and is basically forgetful. It's a very, a very, very lonely wandering God. And then is never seen again.
Amos: And apparently in the movie he has bird poop all over him.
Chris: And they tried to give him the ring. They literally don't give him the ring because they're like, eh, he'd probably just lose it. He's so forgetful. But he's also like a prescient, all powerful being .
Amos: Covered in bird poop.
Chris: I will say that there's a lot of twists and turns in that book. There's a lot of, there's a whole lot of highs and a lot of lows. It’s like Tolkien got through Tom Bombadil was just like, uh, Tom Bombadil is when you accidentally give, you know, a level four cleric in D&D a Mace of Pelor. And then all of a sudden you're like, oh no, I see what I've done here. This character is now going to run away with the campaign because I've made them, I've allowed them to become too powerful, right. That's Tom Bombadil. Tom Bombadil is the Mace of Pelor. And you're like, oh God, no, he's I got to get rid of this guy.
Amos: I'm going to have it back this up.
Chris: I need to rect con this.
Amos: Make him forgetful.
Chris: Yeah. So he's just going to forget things like a lot. And now he needs to go, okay, Tom Bombadil's disappearance is the equivalent of the dungeon master, "Alright, you enter the room, the room was a mouth. You are devoured. You don't make a saving throw."
Amos: *laughing* Yeah. Done.
Chris: Give me your character sheets, please.
Amos: I don't like any of you. Or this story. I'm out.
Chris: The room is a mouth, you are devoured, is a time-honored dungeon master tradition.
Amos: I've never used that one. I might have to do that.
Chris: You also got to pull out the rust monster. Occasionally just whenever your players don't seem like they no longer fear, you, you need to pull out the rust monster.
Amos: What's the rust monster?
Chris: See? Exactly. Don't tell them. Okay, so here here's, here's a DM trick. If you give somebody an item, that's too good, you give them the rust monster. And the rust monster is, I think it's, I think it goes all the way back to like just Dungeons and Dragons. Like I don't even think it's an a D&D thing. I think it's been in the game basically forever. Uh, but basically it's a monster designed to kill weapons and armor .
Amos: Perfect.
Chris: Because anytime anyone hits it, their weapon disintegrates. So you just don't tell them what it is. And then you're like, there's this giant beetle looking thing. What do you do? They're like, I attack it. You're like, oh, that's interesting. You don't have a weapon anymore. Your weapon crumbles in your hands.
Amos: I hadn't heard of this, but.
Chris: It's a, it's a terrible DM trick. There's a reason that DMS are not well liked.
Amos: I accidentally gave you too strong of a weapon. Here's how we're gonna deal with that.
Chris: I think that was the intention was, you know, this is a course correction.
Amos: Yep.
Chris: Uh, it became used by a hostile and uh, angsty teen boy DMS the world over.
Amos: In the DM versus player game, right?
Chris: Yeah. Whenever your group disintegrates into, uh, you know, the, the DM versus the other players when it's a game against them, but you could win, that's when your game us going off the rails.
Amos: Yeah. Yeah. I remember the first time I ever played, we made a bunch of characters that were all like level one, because we were going to start out as level one. Cause that's what you do. And the DMS, like you meet a black dragon on the road.
Chris: He devours you.
Amos: It was over. And he was all like boastful and gloating about it. And you're like, this is the dumbest game I've ever played.
Chris: Yeah. You need a good, a good DM is hard to find, you know, you, you gotta, you gotta get a good DM. The DM is like a bass player. It's like a Jeff, like, you need a good, uh, you need a Jeff as your DM.
Amos: You need a good Jeff.
Chris: You need someone who's going to just be the, the, the, the anchor.
Amos: To be fair, I tried to get a really good Jeff. And then you hired him.
Chris: Well I didn't hire him, but he was hired at Bleacher Report, yes.
Amos: Uh, so Jeff is awesome and Samuel is awesome. So thanks guys.
Chris: There's a lot of awesome people in the world.
Amos: There are a lot of awesome people in the world. Most of them live here in Kansas city or have connections in the area.
Chris: Okay. Yeah, sure. That's a weird flex, but okay. Whatever you say, fam.
Amos: I mean, as long as you don't count Chris's family and Topeka.
Chris: They're nice.
Amos: No, they're nice. Wait, that's your connection up here.
Chris: Oh, I see.
Amos: I was saying all the people in the world are cool that are connected here.
Chris: I got it. Yeah, all roads lead to Kansas City.
Amos: Let me, let me explain, let me explain this joke because they're way better when they're explained.
Chris: Yeah. Well, that's, that's, that's been, well-established in the lore.
Amos: All right. I've been working shopping.
Chris: Alright, I need a hard out.
Amos: I've been, I've been workshopping something. You gotta have one last joke.
Chris: Okay. I'm ready to hit me. You got a tight five.
Amos: Everybody knows this joke though, so I can't, can't, uh,
Chris: You got a real it's a real tight five, on whatever on this Kansas city material.
Amos: So, you know, um, you know, we homeschool and a lot of times I'm talking to people and they find that out and they're like, oh, so you, you have socially awkward kids that are stupid. And, and you know, like, I mean, no matter how true that is, it hurts. But now like with this, uh, you know, everybody is kind of in this homeschool mindset now. And now whenever I'm I say that we homeschool our children. Nobody says anything. Cause I'm here alone in my room. That was it.
Chris: No, that was good. I think you were selling, you sold a little bit after the close, which is, you know, the funny, the funny, but it was like, no matter how true that is, it still hurts to hear it. That, that one got me.
Amos: Well, that one I just added. So, so that it got workshopped in just now and it was better than the original.
Chris: Go, go with your gut.
Chris: Go with it. All right. I gotta go.
Amos: All right, sir. Thanks for coming