Episode 99: Big Tubes

 

The Elixir Outlaws now have a Patreon. If you’re enjoying the show then please consider throwing a few bucks our way to help us pay for the costs for the show.

Episode Transcript

 

Amos: Welcome to Elixir Outlaws, the hallway track of the Elixir community.

Chris: Nailed it.

Amos: Always, always.

Chris: I mean, on my end, on my end, we nailed it. This is always the case.

Amos: Oh yeah? Not on my end.

Chris: Look, the internet is a thing and pipes are a thing. It's not just a big truck that you can put things on. Okay. It's a series of tubes.

Amos: Some of these tubes are bigger than other tubes.

Chris: Yeah. Yeah. I've got big tubes over here.

Amos: I always believe you have to be behind. I got a gig up gig down.

Chris: I have a gig up and a gig down.

Amos: Yeah. Problegation. Not fast enough.

Chris: Weird flex. I have a gig up and a gig down in my network isn't run by an evil monopolizing corporation.

Amos: Oh. Well mine is. Who, who claim to do no evil.

Chris: Hm, no, I don't claim that anymore, I don't think,

Amos: Ooooh, they don't, they stopped that. Whenever part of your claim is do no evil and then you remove that. It's like a, it should just be a big beacon of light.

Chris: Oh no, yeah, for sure.

Amos: Here's a problem.

Chris: I don't think "Don't be evil" is, is their slogan anymore. I'm pretty sure it's like harvest the world's knowledge,

Amos: Money, money, money, money, money, money.

Chris: Yeah. Right. Yeah.

Amos: At all costs.

Chris: I'm pretty sure it's, it's a do whatever it takes to not be branded a monopoly. So we don't have to figure out how to, you know, succeed in an antitrust suit.

Amos: And you know, when you have a behemoth that size, you have to swallow a lot of money to get things going and keep them going. A lot of people, depending on them for their livelihood, which is sad too.

Chris: But they probably pay taxes. I mean, I, I assume they pay taxes, right. They have to pay taxes. So they don't pay taxes? Oh, uh, oh, whomp-whomp.

Amos: So there is a, um, a large company in the Kansas City area who got a building paid for through taxes. And the day after like the contract, uh, that they had to maintain the building in the area was up. They own the building. So the day after they sold it.

Chris: Interesting, interesting how that works.

Amos: Taxpayers pay us for our building. We use it for 10 years and then we sell it and make even more money and then get taxpayers to pay for the next building.

Chris: But think of the shareholders, Amos. Okay. Who's looking out for them. All right. You know who you're supposed to be looking out for our people, Bob.

Amos: That's what it’s all about. Shareholders.

Chris: That's what, think of our shareholders. Okay. Who's helping them out?

Amos: Oh man. I don't know. I don't know. That stuff drives me nuts. I also like helping people grow a businesses that makes a lot of money.

Chris: Yeah, listen, that's the thing is, there's a, there's a, there's a, you can hold these two things. These are two ideas in your head of building businesses is interesting and fun and a worthwhile pursuit, uh, doing new and innovative work is a noble worthwhile pursuit. And you can also not like that, you know, giant tech corporations become anti-competitive and squeeze out any ability for anybody else to, uh, make any entrance into that market. It's, you can be upset about both of those. You can, you can hold both those things in your head. They're not mutually exclusive.

Amos: I also have a hard time, like, cause I understand as a company, wanting to grow. And so it is sometimes at odds, but I, I, I don't know how you can convince a corporation to be like, Hey, we need to move out of the way for somebody else, you know.

Chris: Uh, the way you convince them is, um, via legal means and creating a fair marketplace.

Amos: So they have to get better. They have to get big enough to where there's a certain size where the legal system helps you, right. Like they, oh, hey, it's great for your company to come in here. So taxpayers are going to build your building for you and pay for everything. But then there's a point where they get big enough that you're like, okay, now you're bad. And we've got to take care of that. So there's like this medium- actually, I think if you get to a point where they're like, hey, taxpayers are building the building for you, you, I don't know. You better be bringing a lot of money to that region. Especially with like multi-million dollar buildings,-

Chris: Right, you have obligations, you know, to pay those people back, right.

Amos: Right. And then some. I dunno. It's wild.

Chris: This is good. This is good content. This is what the people are tuning in for. Um,

Amos: Come in, talk about our anti-trust monopoly lawsuit.

Chris: This s this, this is the hard hitting, uh, this is the hard hitting economic discussions that people tune into the Elixir Outlaws for.

Amos: So, well, you know, you gotta-

Chris: It's this , children's sandwich, uh, discussions- no, yeah. We've really nailed our demographic. I think .

Amos: Favorite mayonnaise. Oh man.

Chris: Dukes. Obviously.

Amos: All right. So we, we talked about, um, oh, this is going to be the greatest segue. We talked about company behaviors. So now let's talk about behaviors and protocols.

Chris: Ummm. Juicy.

Amos: Terrible, terrible segue.

Chris: Okay. Sure. So we, we, we discussed this off air, uh, the notion of talking about protocols and behaviors. And we'll say when to use either one of these things and, uh, what they're good for, what their, what their purposes are and uh, my own spin on it, cause it's me still, is ways in which I think we might be misusing some of these, some of these things, and ways that we could improve.

Amos: You didn't say that neither, you, you said that they both exist, though. This is not a normal podcast with you. You gotta say something doesn't exist. It’s not a thing.

Chris: So why don't you, uh, elucidate-

Amos: Elucidate.

Chris: Our listeners on, uh, just in case anyone's not familiar with behaviors and protocols. And you tell me, what do you feel like those are used for?

Amos: Alright, uh-

Chris: What do you, what do you feel like their best use cases are? And then how and how are they different from each other? Because they seem similar when you first learn about, uh, behaviors and protocols and do similar-ish things.

Amos: Ooh, ish. Yeah. Okay, so, um, uh, a behavior is being able to define a specific set of functions. Function signatures, is what I should say, that, uh, a given set of things to implement those behaviors. So you're, you're telling the system, hey, I need a function that takes these arguments and returns this type of result, if you want to work with me . And then you can go and implement those behaviors, um, on your own, in your own modules, uh, it doesn't even have to be like a structure anything. Protocols on the other hand, two string is, is a well not, is a pretty common one that you might want to implement is you have a struct, uh, that you implement a protocol for that allows a single function call to get dispatched across different struct types, kind of implementing a polymorphic type behavior within a functional world. Uh, one, one common one that I've seen if you're doing Phoenix is IO data. Uh, Phoenix, HTML safe is the protocol and the functions to IO data, uh, so that you can create your own internal structs that you drop them into your EEX and they just print out in whatever way you've implemented these functions. Um, that's, that's one way that I've used them for protocols. Protocols to me are, are nice when you, um, want to take any kind of input, um, any in quotes, uh, a given set of types of inputs and do something with them. And it really is great for a library author, I think, to allow users to have user data structures that work within their library. Behaviors. Behaviors, I usually use for internal things to a module. So, callbacks GenServer, you know, handle info. Those are, those are handle call, handle cast. Those are all behaviors. So I use behaviors for internal things. To me, behaviors are different in that I actually need to know the module, the, that is implementing that behavior in order to call that function. So that's why I wouldn't use it for an outside world thing. I would use it for a protocol if I wanted to be more dynamic. How about that? I feel like I've talked a lot.

Chris: No, that's good. I'm trying this thing where I let you talk more.

Amos: Whoa.

Chris: No, I think, I think that nails it. So, uh, some flavor I would add to that, is behaviors, mechanically, are attached to modules. Modules implement behaviors, uh, via a set of functions, right? So, uh, the canonical example of this is a GenServer, where the machinery of a GenServer is hidden from you. You know, there's a process in there somewhere. It has an implementation that does things like handles casts, handles calls, handles different -signals other messages, knows how to delegate those and call into the behavior, the module that you, that, that implements a behavior. And that's where you get your handle call handle cast handle info. The, the machinery of a GenServer knows how to call all those various functions. And so when you want to, uh, implement a GenServer, what you're really doing is you're writing, you're writing a bunch of callback functions, handle call, handle cast, a knit, uh, and handle info, and others, are callbacks that are going to be then called by the actual GenServer process. And that allows you to use, it allows you to take those GenServer processes and use them for a whole range of possible, possible things. Like, they're like it's one of the most generic primitives in OTP, the notion that you can take a, one of those GenServers and you swap out the implementation details simply by swapping out the callback module that you're passing into it. So behaviors are, are attached in that way to modules. And that's a very specific thing.

Amos: It feels like, a lot of times to me , is that it's almost like having a function that you you're injecting a function into. Like the callback is actually like you're injecting into a piece of another function with a callback , without, without having to pass like an anonymous function or something in, and it's, it's hidden. That, I I've experienced with, with, uh, our interns and maybe people who haven't done a lot of callbacks that that's like the magic part of Erlang and Elixir. Uh, and it's, it's sometimes the hardest part to be like, wait a second. When's this called? Where's it called from? Um, it's a little harder to grok, I guess.

Chris: Right. But I think, I think a super important thing about behaviors- And I think this is something that people kind of get wrong - is behaviors aren't interfaces, in the Java sense of the word interface. I think people use them like that a lot and assume that they are like Java interfaces. And they're really not. Behaviors in my mind are really about separating out truly generic ideas, like a GenServer. A GenServer is a super generic idea, hence the name, right? Like it is, it is a general purpose idea that can be used in a wide range of things. Your implementation of the behavior is a specific concrete thing. And so behaviors allow you to separate those two pieces out. They allow you to break apart general and specific. That's a really powerful design idea. Um, and that, that is something that often we should strive for in a lot of design, I think, I think. I think it's a really useful way to think about design is like, what can be separated out? What is truly general and what is more specific? That also means that the, you know, the behavior you have to abide by the contracts, right? You have to, you have to implement a certain set of functions. Those functions have to take a certain type of data and then they have to return a certain type of data. Like there is, that is all contract that you have to abide by. Uh, if you're going to, for instance, opt into the GenServer thing, um, you have to, you know, if you have, so you're going to get called, you're going to have a GenServer call you, rather, if you're going to call a GenServer, and that's going to turn into a handle call callback somewhere, you have to be able to accept the message. You have to be able to accept who, where it came from. And you have to be able to accept your current state. You have to have a handle call three function, and it has to return a specific thing. And if it doesn't return one of those things that blows up. So in that way, you have to be, you have to be aware of all of, uh, of, of what those contracts are on, on all sides of it, right. And they're attached to modules. And I think that's really important that they're attached to modules. I see this general trend in a lot of Elixir code that I'm looking at right now, where people are attempting to hide away the passing around of modules, and they're doing it by, you know, use like, like using macros or whatever. It's interesting to note that GenServers, to me, feel very, never feel burdensome in terms of like how you pass stuff around. And I think part of the reason is because when you start a GenServer, you literally pass as the argument, the callback module. You just say, here's the callback module I actually want you to use when you start this GenServer. That's like fascinating to me that, that I think we just internalized that as part of the machinery of using GenServers. But I don't see people doing that a lot when they actually, when it comes to their own libraries, they're taking more of like the Ecto approach where it's kind of all hidden away from you and done kind of, um, via macros and macro expansion. And I think that's really interesting because if you accept the idea of just passing modules around as like a, a generally acceptable idea, it opens up a lot of design ideas.

Amos: W why do you think that they want to hide the modules?

Chris: Uh, because it looks more like, OO, I mean, that's my, that's my really curt answer. I think it looks more familiar and it's like, well, I have to type one less thing, you know. There, there are for a lot of people, a rubric where typing less is better. Typing less indicates that you have better code. I don't really personally subscribe to that idea, but I think that that idea does is prevalent in the, in the ecosystem. I think that's a lot, a lot largely because, um, most people coming to any functional language probably were first introduced to an object oriented language, just statistically speaking. You know, if you're on the bell curve, like most people-

Amos: Most people anywhere.

Chris: Most people anywhere are, you know, I mean maybe, uh, oh yeah, I get it. You learned Scheme and Haskell in college. That's cool. But like, most people are, you know, picking up an OO language, or a procedural language, like Python as their first language, you know, and, and learning OO as the, as the sort of default. So I think that's where a lot of these ideas come from. It's like, it looks more like OO if you hide all that stuff. And it feels a little bit weird to pass around modules, module names, as first-class things . We're not used to that idea, especially if you are coming from, from Java or something like that, it'd be like passing a type around, like, you can't do that. Types aren't reified things, you know? So it feels weird to pass a module name around, even though that's totally a thing that we do all the time under the hood.

Amos: So-

Chris: That's my hunch.

Amos: Well, and I, and I, I think that I understand like where you're talking about passing types around and it can feel that way, especially with like structs, right? You have, you have a struct it's named after its module in a way. So it does feel like you're passing a type around. Even if that module isn't a struct based module, like a GenServer is not a struct in and of itself. You can have struct in it, but in itself it is not a struct. So it feels like a type, but it's not a type, it's just a bag of functions,

Chris: Right? Yeah. It's a name that has functions attached to it. That's yeah, you're totally right. Yeah.

Amos: You're just, you're just giving it a name so that you can remember where they are better instead of having one global namespace.

Chris: Exactly. No, I think that's totally accurate. It's interesting to me, I think that there is, that's definitely a thing I see a lot is people kind of trying to hide the module names. Or you put them in configuration, right. And then you're even worse off, you know, cause now it's like that's global, you got all the problems with, with application config at that point. And it's useful for its , it's, I would say not useful. It's convenient in a lot of ways to use application inf to stash module names, so you don't have to, you know, pipe them through all your function calls everywhere or whatever. But I dunno, there's something very freeing about choosing to go more the route of being explicit about how, what, what module names you're passing in, into your various, uh, either function calls or into processes that you're starting. Another thing about behaviors, and this is something that, um, this is an idea that I did not come up with at all. This is something, uh, Quinn has talked about a lot and I've, I've been really intrigued by this idea ever since she sort of put it out there. And I've been thinking about it a lot because it rings so true to me. Uh, the idea that some of the best behaviors are things like GenServers, but in a lot of ways, what we would Elixir people end up using them for is much more like public interfaces of things. And we swap out the module with like to, to use a different public interface. And I actually wonder if that's really wrong. Um, if that's really not what we want to do. And this is like I said, not, not my idea. And maybe I'm, maybe I'm leaping more, uh, into this then, then I might be putting words in Quinn's mouth. Uh, but, and then maybe, you know, sort of interpreting it slightly differently. But I think it's interesting that we, we and I, myself included, tend, in Elixir, tend to use behaviors to define like sort of public interfaces. And then we swap out the, the public interface with a different thing that , we swap out the implementation by swapping out the other public interface, you know? So it's like if I want to talk to Reddis or if I want to talk to talk to, uh, uh, Postgres or talk to ETS, I would have one behavior that defines a public interface. And then I swap out the modules and that's like still just implementing the public interface. And I think that that maybe is wrong. Like maybe that's not actually the parts of it that we want. Maybe that's not how we want to be approaching that.

Amos: How do you think we want to be approaching it?

Chris: I think if we want to be high- I mean, I think it goes back to the idea that like, you're not really removing the parts of the machinery that are, that are generic, the interface to me, isn't- yeah. See, this is interesting. I don't know that I have words to describe this yet. This is me kind of figuring this out still, putting words to the feeling that I have after doing this for a long time. It's just like you develop an innate sense of this stuff. GenServers are great. I never worry about, I never think about the machinery of GenServers. I just know that I have these callback functions and I can utilize them. Whereas whenever I've attempted to use behaviors to define an interface, to be able to swap out, you know, I want an in-memory database versus Reddis for testing purposes or something like that. I've never been satisfied with that because I don't think it really is getting to the, to the heart of it where you're separating out the truly generic pieces from the specific pieces. Because in that case, you're actually sort of attaching all that stuff together. You're, you're coupling together the generic and the, and the, and the specific, if that makes any sense. So, uh, I think that's, that's my general feeling on it. So that's, that's my overall feeling on the behaviors thing. It's interesting to me, I feel like protocols are criminally underused.

Amos: I, I do too. Um, ed notice all the time, especially in things like displaying user information, I find protocols to be really useful, deeply just fathering. Um, perfect. The thing that I I've noticed with choosing behaviors, I almost feel like behaviors, when I have implemented my own, they almost all belong in like a loop, like a process type loop. I don't define a module that has some behaviors outside of that very often. I think I've done it like once and then was like, wait a minute. I think this is better served as a protocol. I've always found that if it's, if it's not something like a process, something that's looping, that's working over and over and has, has usually multiple callbacks on just one, then it's better served as protocol. Your mileage may vary.

Chris: Kids are yelling in the background.

Amos: It's perfectly fine.

Chris: Sorry, what was the last thing?

Amos: I was just saying that, uh, if it's not looping in and hanging on for a while behaviors, I, uh, that's where I see behaviors is like processes, things that are long lived, not in like some kind of module. If it's, if it's data bound, then to me, it's a protocol.

Chris: Yeah. I agree with that.

Amos: If it's action bound, like doing something, then I look more towards behaviors, but those also typically are long lived things.

Chris: I think I overall agree with that. And at the same time, the only reason I feel that way is just because protocols are attached to structs. Like that's kind of the only reason to think of it that way. I'm not sure if that's a good enough reason. Like I'm wondering about this, right? Like I, I, I actually think that there's no reason you can't express actions as data as strucks, even if they don't hold anything inside of them, you just need a name for the thing. And then you get full dynamic dispatch, which is really the thing that you want in a lot in a lot of cases for runtime systems is to be able to dispatch on an, on a named thing and, and have different outcomes depending on what you pass in. I'll say this, like I, a lot of the libraries I've ended up working on that I think have really stood up to my own development cycles , were built predominantly out of protocols. And it was a great way to create a set of extensible dynamic things. So, yeah, but I think, I think protocols are great because what protocols allow you to do is extend your system, your library, your whatever, by just adding a new struct and implementing a thing. And so there's no coordination effort that needs to take place to do that extension- so

Amos: So the wild thing that I'm getting here, idea as you're talking, is like, you could, you could implement a GenServer type thing instead of using behaviors, using protocols. And the initial state that you pass in would be the data that, that the protocol is implemented around. And then you could actually change that out in the middle of something running.

Chris: Yeah.

Amos: So you could build like a state machine that way. Like I have this state and then they switch.

Chris: Yeah, if you built it all based on protocols like that. Yeah, absolutely.

Amos: Yeah. That would be interesting.

Chris: Yeah. You would have different structures that define different states that you could be in and they can transition back and forth between each other and just by simply returning the next, the next structure.

Amos: So the, the one thing that the behavior buys you over the protocol is a guarantee that you have implemented. It doesn't mean that you implement it necessarily right. But you've got a little bit of a compiler hint in a protocol of, or in not in a protocol, in a behavior saying, hey, this has to implement this. Where with the, with the, uh, protocol, there's nothing telling you that until runtime.

Chris: I believe that's the case. I, I don't know. I don't know any more because I don't know enough. I mean, Wojtek would be able to answer this question better than me, but like, I don't know.

Amos: In a second.

Chris: Yeah. Right. Exactly. I don't know. Protocols are also special, I think, because, and I think this also has to do with one of the reasons why people still don't really use protocols that much. I think partially because like, people don't quite get it. They're a little bit confusing where behaviors like map a little bit closer to things that like people, you know, the, the ways that people think about interfaces. Like interfaces in Java, you have a mod, you know, you have a namespace, do you have a class, and it defines these functions, it stubs them out and says, you have to define these things. That feels like a behavior to people. And like, it looks the same, like in your text editor, it looks the same.

Amos: Because you're telling it what functions you have to implement. And the compiler tells you, you have to implement them

Chris: Yeah. You know what I mean. It feels right. It feels more like it. I think it's, I think it's mostly aesthetics, right. But I think that's what people are drawn to. That's why people are drawn to behaviors. I also think, back in the day, protocols had some pretty severe performance problems, for like a hot minute there. Um, and that's why you do the whole- do you remember this? You had to do the whole protocol consolidation thing. There's like a, like a, a compiler pass that consolidates them also. It's there, you don't take a performance hit.

Amos: Yes. And it's still there. It's just, I have to think you don't have-

Chris: You don't have to care about it anymore.

Amos: I think the default changed or something. I don't remember. It's been awhile.

Chris: No- er- I don't remember. I believe that that requires a special, a special compiler pass to be able to do that consolidation. And in that way, they're kind of special, but I don't remember what, what were we talking about? Why don't we talk about that?

Amos: Uh, the reason that people don't, didn't jump on strings

Chris: Well and I think, like, I think that's part of it is that you, there was sort of like, uh, like some, you know, fear about like performance or whatever. But I think it's more just that it's, it's not as intuitively obvious of why you would use these things. Um, and then they are attached to structs, which means you have to have a little bit more ceremony when it comes to creating a struct, attaching it, all that sort of stuff.

Amos: Well, it, you know, you mentioned Java interfaces and really the, the interface is more of protocol. It's just that you don't get the compiler help. Like, that's the only thing that you get from the behavior, that feels like interface. But, and, and even when you were describing, when you were describing a behavior, you described it as an interface and it is, it is a definition . You're defining like how, in order to you use me, you have to define these things and.

Chris: Yeah. Good point. Good point.

Amos: And protocol doesn't, doesn't give you those strong guarantees at compile time. It gives, at runtime, it'll tell you, hey, wait, you can't do that. So I guess that's, that's where I think that it gets equated. A behavior gets equated to an interface, a Java type interface.

Chris: Yes. Yeah. I think that's right. And there's kind of a rule- Oh, that was what I was gonna say before is- it may be possible to do more of those checks at compile time. Cause it does do that consolidation effort. I don't really know. And maybe it's just not. And uh, and that that's fine too. I do think that there, it could be really interesting if the mechanism of dispatch, eh, that's probably not possible because of the consolidation stuff, I was gonna say, it would be interesting to have a more, even more generic dispatch mechanism that was runtime based than, than protocols, because protocols are attached to structs, but like, and sometimes like you have to do a lot of work to get a struct. Sometimes you just have a map with keys inside of it, and then you got to like convert it and figure out how to convert it correctly into the struct you want.

Amos: And sometimes you don't even care about changing it. You just put it in a struct to give it a name.

Chris: Right. Exactly. And so it would, I mean, we dealt with this when we were working on the Kafka stuff at Bleacher Report, Jason and I were dealing with this and it was, it was annoying. It was actually really annoying because we wanted, we thought, we felt like protocols were really good use case for the different events that we were seeing coming over Kafka. But it means that you can't just JSON decode into a map and then dispatch based on key and have it be open. That's actually what you want, right? Like you wanna, you want to dispatch based on the value inside of a key somewhere. Cause that's what tells you the type.

Amos: To be fair, cause I, I know somebody out there is screaming, well, it, your protocol doesn't have to be a struct. It can be there, like, Tuple, you can, you can, you can do it.

Chris: You can. Kind of.

Amos: But when you start trying to get really generic like that now to imp- for yourself to even implement that protocol, you have to be super specific. And I think that you lose a little bit of telling the user about how to use -that name gives them, gives the, the user of it a little bit more, too.

Chris: Well, and the other big thing though, is, is that if you've got a bunch of generic events and you're going to extend those events over time, you're going to allow new events to come into the system in order to coerce the plain map into, uh, a struct in order to dispatch on the, you know, dispatch that event based on a protocol, you've kind of like already done all the work. Because by the time you figure out what the kind of event is, so let's say you get a, you get a JSON blob and it has a key called event, event type, or just type, or whatever. And then you need to figure out how to coerce it based on that type, that, unless you have a, a general way to do dispatch on, on the, on the value inside of that key, you're basically down to like, well, pattern match and have a case statement or a bunch of function heads that identify the different types that we know about, and then coerce them into the correct structs and then do the dispatch operation. But at that point you've already done the dispatch operation. Like you've already pattern matched, and it's not, and that pattern match is not opt in. I can't, I have to do a code change. I have to get back in and, and, and, and touch this central point, this one pattern match, in order to extend it. Now, people who like types are going to argue that that's what you ought to do. I'm on, I'm on the fence about it. Like, I actually don't know how I feel about it, but you can't opt in.

Amos: Um, I like being able to do it. I like to be able to do it at runtime. The only time, um, this was not my idea, I did not do it, the only time I've actually seen it successful is doing something exactly like what you're saying. And there was a struct type that was called event or something like that. Very generic. And there was a type in there. And then the implementation of the protocol that they had created looked at that like, they just pattern matched. And, but you still couldn't really do it at runtime, right. Cause you're still pattern matching. You're still hitting some function head. Um, I guess you could have a generic thing and try to do some crazy stuff, but, but it, I don't even know what anything more dynamic would look like, to tell you that, in my head, like, I'm trying to think of what, like what does that interface look like? To be more dynamic.

Chris: Sure, well, so Clojure, Clojure has a concept of this called multi methods, um, which is an idea that stems from common lists, if I remember correctly. And they've sort of refined it, uh, as, as, as, as the way of Clojure, but the idea with Clojure is you define a dispatch function and that dispatch function is generic. And it just tells you what, what key, what thing, what value you're going to dispatch on. So maybe it's the type, you know, you just call type on the, on the piece of data that you have. And it tells you, this is a map, or this is a vector. This is a, a class object from Java or whatever. And then you, you, it goes and looks up in a registry, the handler sides, the handler functions for that, and finds the one that matches or finds a generic one, because you can define, much like with protocols in Elixir, you can define like, kind of in any case, like a fallback case. So you could do that still. And that's how they that's how Clojure does it. And then, and then you can write whatever dispatch function you want to go sit in there. So for instance, in your case, in my case, it could be like the dispatch function could be, well I have a map, look at the type key, cause we know we have to have a type key, and then use that to dispatch on, and then I can attach any events I want and I can have multiple teams attached to those events. I could change them at runtime. I could allow them to, like, upload a new, a new Elixir script file and do, and do that. Now I don't have to deploy code. That's dangerous. That has its own scary, scariness to it, but it's totally opt in and now I don't have to define a struct. I don't have to have a struct at all, but the trade-off there is, it's almost certainly going to be slower because with protocols, I believe, what it's amounting to is a giant set of case statements. And those are very fast because the compiler can optimize those and knows how to handle that very well. So that's, that's what it, that's what it comes down to.

Amos: Uh, Julia also has multi-dispatch, but to me, whatever, I, I have very little Julia experience, so if, if somebody knows for sure, like, yeah, come tell me how I'm wrong. Um, but to me it just, it looked like pattern matching and I could not figure out how to make it dynamic.

Chris: Gotcha.

Amos: Like truly dynamic, like something in defined at runtime.

Chris: The, the way the thing I've been playing with is I wrote a library called, uh, synch_ dispatch. And it's basically, it's, it's basically just telemetry, the internals of telemetry, the telemetry library. I actually think the internals of the telemetry library are amazing. They're really good. And I want that as a generic thing. And it will definitely be slower than just calling functions and letting the compiler optimize them away. But the notion is that you can go inside of that thing and say, dispatch this kind of event, and you could have a generic function that allows you to define what that looks like. And then you're looking up in an ETS table, the handler. And so that gives you that runtime ability, where you can attach and dis, disconnect, and add new multi-method at, at runtime. That's pretty cool. And I think that that's cool. That's an interesting idea. I don't know if that's a good idea. don't know if it's, I don't know if that's actually what you want. Cause that kind of code gets really confusing to track and follow like this conceptual overhead, um, to being able to find, like , where's the struct? Does it implement? Which protocols? Like it's nice because it's open, it's open for extension always. And that's a, that's an amazing feature of having things be open for extension is, is, is a really, really powerful design goal because it allows you to build stuff with less code. You can just maybe if you can extend the system without touch, like touching everything and no breaking changes, like it's awesome.

Amos: If the user can extend the system too, it becomes quite powerful for them. But, like, in my experience, everything, the more dynamic it is, the harder it is to figure outputs.

Chris: That's true. I think that is, that is true. And so you really have to pick what parts of it you're going to have, you know, what parts of it you want to be concrete and what things you want to be more dynamic about and picking those in good spots, like picking those, those, those sort of seams is really important. I don't know that I have a good rubric for it outside of this feels right, this feels wrong.

Amos: I, I, in listening to you, I like the idea of it, if there, you know, if, if you have like a distributed system and you can bring up a new node that has new functionality, it can put its functionality into that dispatch table. And you can dispatch that node without like, without taking down the rest of the system or redoing anything.

Chris: Yeah. But you really have to, you really have to opt, you really have to design around that and you have to build a core around that. That is really solid. So you don't have to go digging into it a lot. Like that needs to be pretty rock solid. Um, and then, and then, you know, uh, you, you allow people to extend it on their own and, and, and govern that on their own. Yeah, I don't know, but that's, I do think protocols are probably underused a little bit. I say that looking at a code base that is filled with protocols and it’s really hard to understand what is happening and to track down anything. So they're really useful, you know, as with anything you can go way overboard with this stuff. This wouldn't have been, been better with behaviors to be clear, it would have been just as bad, but it's, you know, uh, it would have been just as confusing, I should say. But, uh, but yeah, I think protocols, I think people should probably try, like, protocols mapbetter, mapcloser to Java interfaces than you think. And in that way, I think it's, they're very useful for extending and changing the system and allowing other people to extend and change the system without coordination. There's a lot of value in that.

Amos: I like to use them for transforms from one type to another. I think they're extremely powerful there. Um.

Chris: Norm is built completely out of, out of, uh, out of protocols.

Amos: That's pretty cool.

Chris: All of Norm is protocols. And so whenever I added a new function or a new thing, it was basically adding a new struct and implementing like three protocols and then I was done and it just worked. It all just worked together. That was all it took to extend that system.

Amos: That's awesome.

Chris: Which is pretty powerful. And it composes. So, um, same with vapor. Vapor is all protocols. Uh,

Amos: I'm going to have to dig in. I also think I'm going to, uh, implement a GenServer type thing using protocols just to, I mean, just to play around,

Chris: That’s good. I mean probably be confusing as hell, but definitely interesting, right?

Amos: Oh yeah. I just, I, I want to do it as a throwaway thing, but I think that would be cool. I have a hard stop coming up.

Chris: Yeah. Yeah. That's where we're going. Oh, so do I actually, now that I look at the time. Yeah. Perfect.

Amos: Well, I guess, uh, I'll catch you later then.

Chris: Later.