The Elixir Outlaws now have a Patreon. If you’re enjoying the show then please consider throwing a few bucks our way to help us pay for the costs for the show.
Episode Transcript
Amos: Welcome to Elixir Outlaws, the hallway track of the Elixir community.
Chris: How's it going?
Amos: It's been a busy week, you know, my, uh, my life's been a little crazy with the medical stuff with my daughter, but, um, we're doing quite well.
Chris: Good. I'm glad to hear
Amos: She's healing. Well, um, has both of her, her nerves are healing and now we just play the waiting game.
Chris: Yeah. Well, that's good. I mean, it's good. It's not good that it happened obviously. Right? It's good that, uh, she is recovering. Yeah.
Amos: And she's, she's the hard worker when it comes to like, she's doing physical therapy and she is, she's like, “Whatever, I'm gonna, I'm going to own this.” So we just got exercises yesterday and her left hand, she couldn't close. And when I got, we got those, when I got home, she was closing it all the way. Right. She's like look at this. This is no problem. No problem. I got it.
Chris: Well, that's great. That's good to hear.
Amos: How about you? How's your life?
Chris: Oh, you know, I'm just working, working.
Amos: (singing) Work, work, work.
Chris: Working and playing D and D uh, you know, it's good. It is what it is.
Amos: You keep saying, you're going to invite me to that.
Chris: When did I say, I said that,
Amos: Tell me you were going to ask everybody and I know the people you play with. So I assume they just said no.
Chris: Well, I mean-
Amos: Do not bring Amos. He'll never shut up.
Chris: I actually have, uh, completely forgot to ask them, do you want to play? You also seemed busy, so I didn't want to, you know,
Amos: Well, the last few weeks, if at all.
Chris: Mnmm, I didn't want to make. I don't want to make your life even harder. So we can play D&D I like playing D&D I'm always down to play more D and D.
Amos: So I just got an email from a friend the other day. And have you ever heard of DND online?
Chris: I'm familiar with ddo.com. Yeah. I mean, yeah, not specifically, but I am familiar with the idea.
Amos: Yeah. So, um, I guess a lot, because of like COVID and stuff, normally you pay for a bunch of adventures and they're givin' like 90 of them away for free.
Chris: Well, that's awesome.
Amos: And then they have a couple of them that are on like this super sale that are usually would cost you about 50 bucks. And now they're $3.
Chris: Nice. That's awesome. It's a good time to play, you know, RPGs role, playing games, D& D whatever, whatever, your, whatever, you know, your, your game of choices. It's a good time to do that. Online. The tools to do online are really good and they're getting better all the time. They exist. Like some of them aren't, there's not like a standout, like this is the best way to do it, but some of the tools out there are sort of, as I described them, lovably jenky.
Amos: Oh, you're talking about roll 20. Yeah.
Chris: So like, it's, it's funny with one of my, so I am in two games, I'm a player and a DM. Um, and my, the game where I'm playing I've become, because I have, I learned a lot about rule 20 to do. Okay. So back up a little bit, roll 20 is a web app that you can use to run RPGs. Um, online, it's basically like a tabletop board simulator, like a tactical map simulator kind of thing. It has tons of features, very clearly features that were developed, not as a grand vision, but as like, sort of ad hoc, like, I need this new feature, you know, it's got a real Emacs feel to it.
Amos: Oh, it does.
Chris: It's like, it's like, you can cobble this thing together to make it into a lightsaber, but it's really good. I mean, it's, it's good for what it is. And it's all real time and all that kind of stuff in it. And I really like it. It is loveably jenky. I am a huge fan for whatever reason, something about it, like appeals to me, I suspect it's how people feel about type systems. It's like, you can't it's, you know, you like it. I.
Amos: (laughing) Loveably jenky.
Chris: It's, lovably jenk. It's like, and it's also like a problem to solve that didn't probably need to be solved. Like there could just be a better thing, but, you know, it makes it more fun to solve the problem. And it has like a macro system that can call other macros. And like, you know, I've gone real deep on all this stuff and automated screen players. You can do music through it, so you can play music through it for all your players at different key events stuff, which I definitely have gotten into. Yeah.
Amos: You want to have some fun with that?
Chris: No, I'm having fun with it already.
Amos: Put on music that never-
Chris: I don't know what other fun I can have.
Amos: If you put on music that never repeats. If somebody tries to close their browser, it won't close.
Chris: Oh, fun. Yeah. Good to know.
Amos: At least on Chrome.
Chris: Good to know.
Amos: I had to force quit Chrome because that was running.
Chris: Any-hoo, roll 20 is a thing that exists and it's fun to use if you enjoy kind of solving problems and stuff like that, it does tons and tons and tons of stuff. Like you could, you could play it. You can run your entire game via roll 20, but it's pretty complicated. You got to really like learn it, but it's become a joke in my game where I'm a player that like, whenever we get stuck on a roll 20 thing, I've become like the roll 20, like lore master. Okay. So the deal, Chris, how do I do this?
Amos: You can ask me anything about a gin server or roll 20. I'm there for you. Yeah,
Chris: Well, you know, it's like, it's a common DM practice to, you know, you always get a rules, lawyer in all games-
Amos: -Uh, geez.
Chris: And in our, in our game, we have, we already had a rules layer, but now I'm the rules rule, roll 20 lawyer
Amos: Is Greg your rules lawyer?
Chris: No, in that game, I'm in that game. I'm the rules lawyer.
Amos: Yeah. Nice. Cool. Just as a side note, there is a control board in this podcast room. That'll allow us to have buttons for like laugh tracks.
Chris: Oh, like sound effects?! The thing to do here.
Amos: Yeah, so I have to figure out how to use this and we're totally going have that.
Chris: We should set it up because I think I can set one up via, you know, there's like apps and stuff you can get to. Okay. So here's the, we're going to figure out how to do sound effects. This is our homework for next, for next time. We're both going to figure out how to do sound effects. And then we're going to bring that to the we're going to bring that to the next recording. That's the challenge for next time.
Amos: Alright. Well, this one looks like it's just got, uh, well, the buttons are all labeled, like drums-
Chris: -No, none of that,
Amos: but I'm pretty sure there's a way to do that differently to relabel them. Cause it also has input numbers and, and all kinds of things on here.
Chris: I'm going to have so many good things. It's going to be great.
Amos: Alright, alright. Laugh track. Well, actually-
Chris: Yeah, I was gonna say, it's gonna be a lot of just, it's just going to be a lot of accents.
Amos: (In a tv commercial announcer tone): Are you tired of dealing with X library for web development? (laughing, then awkward silence). Okay, I'm done.
Chris: No, continue to the joke, I see where you're going with it.
Amos: Nope. We're good.
Chris: You quit before the punchline.
Amos: I think that whole joke was the punchline.
Chris: You're the punchline.
Amos: Nice. So we got a question, right, from a listener, and look at me just trying to avoid the rest of that conversation.
Chris: I was just going to- no, listen, you can be the funny man. This, this, epsiode.
Amos: (laughing) Nope. I'm good.
Chris: You bring me in here. You bring me into your podcast. You demand I tell jokes. You then ruin the jokes that you demand that I tell you, and you sic you and your, your hoard of people. You, you sic them on me. Okay? You, you, you bad mouth me at conferences. Okay. To other people,
Amos: What?
Chris: You, you throw me right under the bus. I'm under the bus and I'm reaching up and like, Amos helped me get out from under this bus and you're like, hang on. And then you throw more buses on me. (Amos laughing) So now you get to be the funny man. Make me laugh.
Amos: Wait a second. I gotta start out with, I don't know if you're joking or not.
Chris: Entertain me.
Amos: (laughing) But if I've ever bad mouth at a conference, I'm really sorry.
Chris: (laughing, in a Colonel Sanders accent) Son, you are in a 12-piece bucket of trouble.
Amos: At least I avoided saying bad things about you when I was on Elixir Wizards, right?
Chris: No you didn't. You did not avoid that.
Amos: No, I did, I did!
Chris: You mean the lamest faint at trying to be like, I don't really want to say anything negative, but also Chris is a scumbag.
Amos: You see,
Chris: (in a grumpy accent) I don't want to be negative or anything, but- If you ever start anything with, I don't want to be blank, but-
Amos: -It's negative.
Chris: Yeah. Well it's whatever X is. Yes.
Amos: (in an high voiced, reality TV star-esque accent) I don't want Chris to kill me in my sleep. So (sigh) he's a really nice guy. (back to normal voice) We actually, so I actually do this podcast completely out of fear that, um, if I don't continue, Chris will show up on my doorstep while, like, in pouring rain and lightning behind me when I open the door.
Chris: Just continuing to talk to you. At least this way, it's at an arms distance. And it only lasts for an hour.
Amos: I don't know if anybody else knows this, but the way this podcast ends is I just shut down Zoom and move on. Like stop talking to Chris.
Chris: Yeah, yeah, yeah.
Amos: That's that's that's how it never no, no. I lied.
Chris: We have a topic.
Amos: Oh, we do.
Chris: This is, this is, this is all you you-
Amos: -Well, you pulled up the-
Chris: -You're driving the bus. Listen-
Amos: -I don't have the email pulled up.-
Chris: -We have already established that you and buses are synonymous.
Amos: I'm in charge.
Chris: You're in charge of the buses.
Amos: (laughing) Crud. Now I've got to pull up the email. Well, we got an email from a listener. Um, I believe his name was Kyle. I'll tell you in a minute, I need to, [inaudible mumbling at computer]
Chris: You'll probably need to start on the person who sent us this email. Do you want me to read the email? I literally have it right here.
Amos: Oh, I, now I have it up. You waited long enough. I was able to search for the email.
Chris: I just wanted to make you do things.
Amos: Um, I would really like to have a show where Chris talks less than 85% of the time.
Chris: (melodramatically) Sometimes I think I should just run away, But who would have me? No one. that's who.
Amos: I would, I would still have you, Chris.
Chris: No one.
Amos: This is, this is Chris and I's radio personalities. When we're off the air, we don't, we don't banter as hard at each other as we do on the air.
Chris: No. Yeah, this is, this is, this is all a bit.
Amos: Alright. So you read it then.
Chris: So yeah. So we got Some letters. Here's the thing. If you send us letters, we'll answer them. At least probably. Most likely.
Amos: At some point. Maybe.
Chris: We'll answer the question. Yeah. I'm reading, Designing Elixir Systems by, uh, Bruce Tate and James Edward Grey, friends of the show. They didn't say that, I'm adding that, parentheses. And I have been following along with the podcast for quite some time. Oh, thank you. My mind was totally blown by the processes are not data segment of the book. I was wondering if you all could talk about how to reframe your thinking to work with functional data. Amos, your thoughts? Tell me.
Amos: Yeah. So that, (mutters) processes are not data. I, so I pulled up the book today and tried to search for processes or not data in the book to remind myself. That phrase does not appear in my copy of the book at all. But I think that the hard distinction there is, um, processes versus functions, right? So we talk about in functional programming that that functions are data also. So it seems to fall like everything is data, but, but processes are, are external. They are the impure pieces of, of your functionality. So you can pass the pit around purely, and that's just data. But once you try to send anything, uh, and receive from an, from a process, you are entering a world where, um, you may not get something back or the state of that process changes your function to be impure. So that's, that's how I took this. I dunno. What are your thoughts, Chris?
Chris: Yeah, I think, you know, one of the main thrusts of the, of that book is to model the sort of central pieces of your application as, as data structures and as data transformation and push it's, it's sort of, uh, leaning really hard into Gary Bernhardt's functional core imperative shell stuff.
Amos: Um-hmm.
Chris: Its leaning really hard into that. And the idea being we can do, you know, we're, Elixir is a language that allows you to talk about data and you can sort of rely on manipulating that data as data. And building out most of the core abstractions, uh, as, as data. And I think that's kind of the idea. I don't, I also don't know specifically what that idea is referencing, but my assumption is it's, it's, it's this notion that you can model most of your application as a data transformation, uh, at least as presented by the book. Um, that's, that's the notion.
Amos: So then how do you, how do you start thinking that way? Uh, actually, like, what are your opinions on that? Can you, do you think you can model most of your system and data transformations?
Chris: Um, yeah. Uh, yes, yes. With like huge caveats. So, so one of the ideas is that you can build out all these data structures and you build functions around all these data structures and then you can compose them together. And I think that that's a really good idea, like just as a general rule, like thinking about data structures that transform into other things is a really good idea. And we've talked about that for the past three episodes, you know, about the idea that what is a HTTP request, if not a data structure, a map filled with, with keys and values that you then transform into things and then eventually transform into a response. That's all the same idea. As applied by the book, what they're sort of doing with that is taking all of these data structures and then finding a place to put them. And that's the real trick. And I think this is where, you know, my, my philosophy begins to diverge from the book's philosophy, which is that in the book, most of what they're sort of presenting is like, you have all this functional, this great functional data transformation stuff. Now you need to put it in a place where you can use it. Uh, and it just so happens to be that you don't really get to use it in a stateless HTTP web app thing that often, uh, inter in, in typical web app fashion, simply because there's no reducer, there's no ability to like, hold onto that state. The reducer is whatever's in the database and they they're sort of design philosophy at that point is to take all those data structures and put them in a process to a gen server or something similar, supervise them and like care about their life cycle, put them in a process somewhere and then run them and execute them that way. And that way they can hold that state. And so then you can build up sort of your application as processes, uh, and then have those processes, you know, have your web app, you know, client thing, your interface, your web app, be your interface into the running application. That's my interpretation of the pattern as presented by the book. Does that jive with what you got from it?
Amos: Yeah, yeah, it does. Um, I don't think that I really have anything to add to that because that is exactly what I got from it. And, um, I do, I think it's, I think it's difficult to, to come up with that data transformation stuff. It takes a lot of thought. There's a lot of work that goes into that. And then in my experience when that works, it's pretty awesome, but I don't know that that works every time for me.
Chris: It's been my experience that that design can lead you to, if you're not careful that design can lead you to places that sort of trap you in terms of design of a system,
Amos: Is this the design where you have, where you're, where you're holding on to state inside of processes
Chris: Like, yeah.
Amos: Effectively as caching.
Chris: Yeah. And the problem is, and I, and this is, you know, I wrote a whole blog post about this called like the single, the single global process or whatever. I think unfortunately, the, it only really works if you kind of don't care about the data that you're manipulating. And the reason for that is, you know, what, if you need to run on two notes, if you've got a process and you're holding on to like important pieces of state and you're deriving views from that piece of state, you have to put it somewhere eventually, which means putting it into a database or putting it somewhere that is global. And databases. I mean, let's be clear, like a database is a giant piece of global, i's a giant global mutable variable. That's what like a traditional relational database as used as, uh, in prescribed relational ways. It's just a giant mutable variable. It's actually like a really crappy design pattern at the end of the day, but it's a design pattern that, that makes trade-offs and provides you a set of guarantees depending on database technology that you're using.
Amos: Right.
Chris: And you, and that's, again, gets back to this idea, like you choose databases for reasons. Like you choose them because they have these guarantees. And so, yeah, it's a giant mutable variable, but its a giant mutable variable with rules and locks and transactions and all these other things that you kind of need to be able to talk about a giant mutable variable in a semi-coherent way.
Amos: So, so the big problem, whenever, so the database allows you to have one thing to go to for that data. But if you have these processes sitting on separate nodes, they each have their own cache, then they can get out of sync or you have to go through some, there are, there are things in Elixir Erlang that allow you to like have just one of those in your entire-
Chris: -Ish.
Amos: -set of nodes. Ish. Yeah. Or-
Chris: -Yeah but the ish is the important part.
Amos: Or you have to go through a lot of distributed systems work to have those at least be, I mean, you have, you start to deal with cap theorem at that point.
Chris: Right. And that's why I say like the ish is the important part. The ish is where all of the ish is why this is hard.
Amos: Um-hmm.
Chris: Ish is, you know, can you have exactly one of those things? And using the tools, using the off the shelf tools that exist? The answer is no, you just, you actually can't just have exactly one of them using the, using the, off the shelf tools. Now, maybe that doesn't matter, maybe you're maybe you literally just don't care about your data and in that case, fine, you know, I'll on Z. I think it turns out that a lot of people do care about that data and overriding it, or somehow destroying it or, or mutating it incorrectly in the database layer that single big global variable thing, that's really bad that has, that could have really, really bad implications, especially when you consider that if you don't have a paper trail for how that stuff ended up that way, you might never notice that it's wrong or corrupted until it's like way too late.
Amos: Um-hmm.
Chris: That's really, really bad. Um, and it's one of those things that like, I think you have to be really careful about when you talk about building systems and thinking about this, it's one of the reasons why I still encourage people to, you know, I, it's one of the reasons I keep, I always kind of push newcomers to be like, just, you know, especially if they're for Ruby, it's like just build your Phoenix application kind of the way you would've built your Rails application-
Amos: -and leave it. Yeah.
Chris: Yeah. Don't, don't get too all all about all this OTP stuff or, you know, too early, because that stuff is great. That stuff is really important. Don't model your domain as processes. And I don't, you know, at the end of the day, putting entities, even if they're just big aggregates of other of lots of entities, like I think there's a trap where people feel like I, you know, it's not that I have a users and it's not that I have a process per user, I have one process to model this one interaction or whatever. And it's like, I have a process that models, lots of, you know, aggregated entities all is one thing. But if that thing is also in charge of writing to the database and those sorts of those sorts of side effects, you have, you still have an entity. It's just a, it just so happens that your entities, this big aggregate of a bunch of different keys and you still have all the same problems. It's still, OO, at the end of the day, like you're still just doing object-oriented programming.
Amos: So I guess for me, then that leads straight into exactly what Kyle brought up is that processes are, are not data because they're they're objects at that point.
Chris: Right. I think that's exactly right. I think that's exactly right. And I, and I think at the end of the day, that's not really how you want to model your application. You don't want to model your domain as processes at all. That's not what they're there for.
Amos: I think that the book tries to push that too, a little in saying model your domain as much as you can in data.
Chris:
Yeah. Yeah. I don't really know. I, to be honest, I read only the, I think I read the same review copy that you did. And then I didn't a follow-up review copy. So I haven't read the full book. So I'm not really here to say the book is wrong and I'm right. Or whatever. I don't actually really know exactly what the book says. I'm sort of, this is more just me saying this, uh, on my own, this is just my opinions on it, regardless of what the book says, right? Like the book might totally agree with me. I don't really know. My point is just that, uh, you know, I think processes are really good at modeling. They're really good at providing abstractions around concurrency fault tolerance, communication, all of these sorts of things. And at the end of the day, like the process is the, the, the, you know, you have to have a process somewhere, right? Cause the process is the base unit of computation and the beam. I think you don't want to get, fall into this trap of doing DDD, which is dumb anyway, and you shouldn't do (both laughing) But, I don't think you want to do DDD with with Gen servers. It's a bad idea.
Amos: Or you shouldn't have a user process and, uh, an invoice process and a line-item process, which I have seen where it tr trying to reimplement objects and each individual object is its own process,
Chris: Right.
Amos: Is, is not good. Um,
Chris: And don't trick yourself into thinking that you didn't do that simply because you're joining line items and users together into one data structure and then putting it in a process, right. That's just an aggregate. And that's that, it's just as bad putting that in a process, if you don't need to. Like putting it in a process, just because it makes manipulating, like essentially building a reducer like that can manipulate that code and manipulate that data structure. And as a reducer, it's not worth it. It's like super not worth it for most use cases. You need to know why you're doing it. You need to have a really good reason to do it. And convenience and easiness is not a good reason,
Amos: Right. Because there are drawbacks. So you got to think through those. So, so we've talked about the one drawback of it when you need two of these things. So what, what are other drawbacks that you've run into?
Chris: I think that's the, that's the big one is that at some point you need two of the thing or you need to run on two nodes for stability and a little bit of scalability. Like you don't want to lock yourself into a decision wherein you can run on one node. That's not good. Like, like I, you know, I'm not the, I'm not sitting here saying if people have listened to before they know I'm not the biggest fan of things like auto scaling and that sort of stuff. Like, I don't think that's the way that you win this either, but you need to be able to run on more than one node.
Amos: Right. (laughing).
Chris: But I'm saying like, I think a lot of, I think a lot of people are just sort of like, but why, like, I'll just do this for, and then eventually I'll fix it. And the trick is like, you're going to do this design, then you're never going to be able to fit it.
Amos: So yeah, it gets, it gets hard to fix. It's like a rewrite at some point you've built too much structure on top of the way that works. And then, then you get to rewrite it.
Chris: So, yeah. So I don't know that. I, I mean, I totally agree with the fundamental statement of processes are not data. They're not there. They're these, in as much as they're not a data structure that you can manipulate, they're just their boundaries, they're little services, they're little things that you have to send messages into to communicate with and then tell them to do things. And you want to do more modeling as just data. And we've talked about other ways to do that on the show recently, but I, yeah, I don't know. I don't want people to presses are good. You should use them, but use them, you know, but think through like why you want them think through what it is that you're achieving by using them when you use them,
Amos: Right. Processes are about fault tolerance and, and restarting. I mean, really to me, processes are all about the supervision and, and taking part in that, let it fail, restart strategy, but processes can store to keep data around. Store is, a, uh, iffy term there for me, since it is just a loop passing back into itself, they appear to store data. How about that? Uh, so what kinds of data should you put into a process? Because cache, I mean, caching is just hard in general and cache invalidation is hard in general. So if you're using it as a cache, know that you're going to have a really hard problem ahead of you.
Chris: Right? Well, and you know, the other, the other trick people use the people use the trick, a lot of like write through caches. They're like all the reads come out of the process, but the writes go to the database. And that's also tricky because again, if you have more than one of these things ever, then you're almost certainly overriding data unless you're doing Cass operations on the database layer, which almost no one does. So, in practice anyway, it's just like, you know, write the stuff into the database. There's no like monotonic and increment, incrementing counter thing that determines, I mean, some people do that, but by default, most people aren't. And so you get in these positions where you're overriding previous data, stuff like that, or you can be, um, which is not good. Places where it's really useful are like, you want to control some amount of, you know, you want to control things like concurrency, or you want to, I mean, there are places there's times to do caching, but at the end of the day, you don't want a process to do caching with, anyway, you want to probably put it in ETS table. So it scales like, so it works. That's probably where you want to be putting this stuff anyway. So, I mean, you want to use it for mechanical stuff. And unfortunately, I think most people, the mechanical stuff has already sort of solved for them. Like most people just need to talk to a database and they just need to make web requests
Amos: If you're just doing Phoenix. You're good.
Chris: And it's, and that's kind of, it's kind of like boring to say, because I think the processes I know when I was getting started processes are like, what I got really excited about.
Amos: Uh-huh.
Chris: And it feels like really deflating to be like, yeah, but when do I get to do all this fun OTP stuff?
Amos: I feel like, rare, not very often.
Chris: They're like, “Oh, somebody else already did it for you or whatever.” And then that's boring. I don't want to do that. So then people, you know, add it to their specific domain. And I just, yeah, I just don't think you want to do that. There are times where you want to build little caches and stuff like that, and that's good, but you know, you don't want to do that for anything. That's not really a femoral or you can't be wrong about it. That's like literally the point of a cache is to have a thing that you know, that you can be wrong about.
Amos: I, I have, I have, uh, like three levels of data that I kind of look at and it's like very ephemeral lives very shortly. Probably doesn't need to be stored in anything at all. Like it's just request throw away. Um, whether that's a web request or whatever, any kind of interaction that I have midterm mid-level data that doesn't need to be stored in a database can probably be a little off sometimes, eh, but might need to stick around like, um, I'm gonna, I'm going to go back to Phoenix, like Phoenix presence type stuff, like people's connections, right. It can be a little off and it's not a big deal. And then, and then long-term like, hardcore, this is the core of our business. And that midterm stuff is what I store in a process is the stuff that needs to stick around for a few requests, but isn't that long-term core business data. If I'm going to store anything in a process, that's where I do it. And then in thinking about processes versus data and where I'm going to put things, processes are really all about, for me, failure and, and performance stuff, parallelism. And if I need, I need to handle a certain type of failure or a parallel thing, so I try to separate my system in the, like, if I do have a process, maybe it's, uh, over it, it doesn't, uh, it might do a, quite a few things. It's not like a single thing. Um, I think I get this a little bit from, from Sasha Urlicht's blog post from a long time ago, about when did you process it? And it's like, when I have the subsystem that I know how to restart this whole subsystem.
Chris: Right.
Amos: And it might have failure and I don't want it to take anything else out. So like, if I'm talking to an external service.
Chris: Right
Amos: That's going to be inside of a process.
Chris: Right.
Amos: But I, yeah, I don't, I don't store a lot there. I might store, I wanted to say credentials, but that's not really it to that external service, but I might put something in there that is like a rotating key of some kind that if it goes away, I just grab another one.
Chris: Right. Yeah. And we use it for that kind of stuff as well. Like that's where I use processes for exactly like all this mechanical stuff. And yeah, there are times where, you know, we, I built, I think we've talked about this in a show before, but at one point I had to build like a fuzzy searching kind of algorithm thing where the clients could send requests really rapidly and it would join up all of these pieces that all, you know, that it would do this search across multiple different services. And so I use a process to govern that for that user session, that basically what we call it, a session, a search session. And we shoved a bunch of the data in there. We got, we essentially like we're pulling more data in the background and throwing it in there. And then as it, and then as they're typing and sending us events, we're doing like this fuzzy searching on it to try it out,
Amos: The data that you've, that you've already been database and cached.
Chris: Okay. Or, or pulled from other services and those sorts of things. Uh, and, and we're inner- mixing all that with live data. And so if the live data is like failing to get back fast enough, then we're just returning stale data and all that. And then, you know, at some point that process, if it doesn't receive their message within a certain amount of time at times out, and it goes away.
Amos: But as a trade-off you consciously made.
Chris: Yes. And it was like in an app that was already clustered, we just registered it. We registered that process globally and then said, okay, find it if you're on the box already, you know, if, that way, any requests that get round robin-ed to some other service can go find that one. And we super don't care. If all of a sudden, we can't talk to that other node. It does not matter, you know, worst case scenario, they're starting their session over or whatever. And it's like, it's just, it doesn't matter. But those are really those sorts of scenarios. I find just to be few and far between when and when, when, you know, when those happened. Yeah. It's really cool that you have a process. You can use processes for that stuff and make them like cluster aware and send messages to them pretty easily. That's really rad. I just don't think that happens all that often.
Amos: And spend time in that functional core. Like I think that that book really does have a ton of great information and learning about how to design processes and test your code well, and they do talk about some of the drawbacks in there. So it is a fantastic book.
Chris: Yeah. It's totally agree. I mean, I've had conversations with James and Bruce about this stuff, you know, offline, like I've talked to them about these things, you know, sort of, sort of outside of all of this sort of stuff as well. And, and yeah, I think they, I think what's presented the book is really good. I just think really be clear about what it is you're trying to achieve when you start to use these processes.
Amos: And I th- I think that that's true of almost any technology thing, whether you're have a blog post or a book about it, um, you need to know that everything about design is, is trade-offs. And sometimes people aren't going to get all the information in there because it's impossible to actually give you everything. Just know that that often there be dragons.
Chris: Yeah, for sure. Well, unfortunately I have to run, this is going to be a real short, short episode. I want to talk about this more, because I think this is a really, this is a rich topic to discuss.
Amos: Yeah.
Chris: And hopefully we can do that.
Amos: And it's hard to design pure data stuff if you haven't been doing it.
Chris: Yeah. It is. It is.
Amos: Even if I take a break from it and work on a project that's like in Ruby or, or something, that's object oriented and I come back, it's like, Oh, crap. I have to get back into the thought process of this.
Chris: Right. All right. Well, cool. This is going to be short today.
Amos: All right. Well, thanks, Chris.
Chris: Alright, later man.
Amos: I'll talk to you later. Bye.