Chris Keathley
The Elixir Outlaws now have a Patreon. If you’re enjoying the show then please consider throwing a few bucks our way to help us pay for the costs for the show.
Episode Transcript
Amos: Welcome to Elixir Outlaws the hallway track of the Elixir community.
Amos: All right.
Chris: Hi Amos!
Amos: If we, if we don't clap, I don't feel like we're in the show yet.
Chris: That's the, it is the now traditional mark of the show.
Amos: That's right.
Chris: So it starts.
Amos: That none of the listeners get to hear.
Chris: No. We cut that out. It'd be very, it'd be very loud and would hurt their, would probably destroy some headphones.
Amos: That's the, that's probably, well, wait a minute. Maybe we should leave it in once.
Chris: Highly malicious
Amos: Surprise cut it right into the middle of a show. Well, we'll talk really quiet.
Chris: NPR voice NPR voice, Having a sandwich today. It's good to see you again Amos. You're listening to Thoughts For Your Thoughts. My name is John Blankenship. I'm joined here today by new author, Morgan LeFonhaser, all the way over here from France and has come in just to explain to us and that her new book about new wave philosophies and how the intersection with counter-cultural art has been taking the internet by storm. This has been John Blankenship with Thoughts For Your Thoughts. We'll see you next time.
Amos: Do you write these scripts up before we start?
Chris: Yeah. Yeah, no, this is just, this is just what it's like to be trapped inside of this brain of mine.
Amos: In some ways I'm envious, in some ways I'm glad-
Chris: No, no, no, you don't want to be in here with me. I don't want to be in here with me.
Amos: I mean, the things that I do in the car when I'm by myself and nobody else is around, you just do it in public.
Chris: Yeah. I just go for it.
Amos: Oh, I talked to Juliet about that, like in the car, I'm like, oh yeah, I'll be driving. And I'll just make weird noises. And she's like, me too, that's where I got it. And her friends are all looking at us like, what is wrong with you people? And I'm like, no, no, no, no, you do it too. We're just willing to admit it.
Chris: We just have less shame than you.
Amos: That's right. So I've been sick the last couple of weeks. I haven't been able to keep up with much of anything. Uh, I just started this morning before we got on here and uh, watched, um, Fred A Bear, or Hiebert for our American friends doing, uh, Advent of Code. So that was, it was pretty cool. It's like, um, it's like doing it yourself only, only you don't have to type
Chris: Exactly. That's the best kind of coding is the coding where you make someone else do it.
Amos: That's right.
Chris: It's been good. I've been, I've been, I've been doing Advent of Code. I've kept up, up to this point, but I feel like my energy for Advent of Code is waning.
Amos: It always does that for me. I never have made it all the way through because I just get to a point where it's like, okay, my family is either going to kill me or I'm going to have to stop. And I get exhausted with it too. It's like, all right, I'm tired of solving this.
Chris: I set an arbitrary time limit. I give it, I give it an hour. If I'm not done in an hour, I'm not done. That's just what it is. And so far I've managed to get everything done under, uh, under, uh, under an hour.
Amos: Okay. I was going to say they build on each other. So once you miss one, you're kind of out, right?
Chris: Yeah. A lot of them, yeah. They build on the previous one and that kind of stuff. I mean, it is fun. I, I like, you know, it's a fun thing to work on and I like those kinds of challenges and stuff like that. But at some point, you know, it's like, they want you to, you know, they want you to do enough stuff that you have to kinda like dig in and really understand that it eats up some time.
Amos: Right. Right. I think they're interesting problems. I just feel like I have other interesting problems to solve at work that I should probably be working on or, or interesting problems at home like taxes and paying bills and-
Chris: Right. Yup. Yup. Those are also, those are also important.
Amos: And then being sick, I was like, I'm not even starting. And now I'm so far behind that. I feel like it's a waste of time.
Chris: Yeah. Well, and you know, the prompts are all also like really goofy and stuff a lot of times. So that makes it more challenging too, to kind of understand what it is that's even being asked of you.
Amos: I think that's why I like it though. Like that's the only reason I keep coming back is because the whole story gets me coming back. But then I get bored with the story too. And maybe that's why I actually stopped. It's not the problems necessarily It's like, alright, I have solved one more problem. How many of these issues can we have in a row?
Chris: Yeah.
Amos: And I get bored with the story. And then I was also, um, uh, just finished reading the intro to Steve Buesey's new book. It's
Chris: Oh yeah. The, uh, I I'm forgetting the title. Real Time, Real Time Elixir?
Amos: Real Time Phoenix.
Chris: Real Time Phoenix. That's what it is.
Amos: Yeah. It's a working title. It's still a beta book, right? It's working. Um, so yeah, I, I only read the introduction though, so I haven't made it that far, but I am excited to read it. Like the introduction itself drew me in and said, hey, this is going to be fun. I hope there's a fun project along with it in there that I can just follow along with.
Chris: There is this, it's a good project.
Amos: You read it. So, okay.
Chris: Yeah, I was a reviewer.
Amos: That's what I'm doing right now. I'm doing review on it. So-
Chris: Nice! I enjoy doing reviews.
Amos: I do too.
Chris: Mostly because I like reading books
Amos: And then you don't have to buy it.
Chris: And then you don't have to buy it.
Amos: You get to read the book.
Chris: Yeah. And I get to, I get to critique someone and read a book.
Amos: So I like it because-
Chris: It's really a twofer.
Amos: -if I don't have someone else to read a book for and a deadline, then when I want to sit down and read a book in the evening, everybody wants to bother me. But when I have that deadline, I'm like, hey, I have to get this done. This is for somebody else. This is not for me. And then everybody actually leaves me alone to read it. So-
Chris: Nailed it.
Amos: It's like, it's like a win-win. I get to hopefully help somebody out. I get a free book that I actually get to finish.
Chris: Perfect.
Amos: Then. So, so everybody else out there, if you haven't been a book reviewer, it can be a lot of fun if you like to read. Uh, so I've been kind of out of it. I've been doing a lot of Ruby lately, a little bit of Elixir, um, just, and, and being sick has made it so that whenever I'm not actually working, I'm just sleeping.
Chris: Yeah, yeah I understand that for sure.
Amos: So it makes it hard to explore the ecosystem and see what's going on. So maybe you could fill me in, on what's been happening lately?
Chris: Ruby is basically just like Elixir now. Cause it has pipes now, right. So I mean, they really it's, you know, that's the thing is languages are like other languages when they adopt each other's syntax features, um, semantics, aren't important. It's only about the superficial syntax layer. That is what keeps people in a language.
Amos: So, so, okay. We, we can go there. We can go down this road. So what has killed me in going back to Ruby is, is hidden state. It's so weird whenever you go to something that's more functional and then you go back to those objects and hidden state and even not objects, but like, I guess, I guess objects. So I, I find functions that pass in a list and then inside that function, add to the list, but that function doesn't return a list. It's like 15 lines of code two functions over later that suddenly that list is used and you're like, whoa, why is it bigger? Where did all this stuff come from? Uh that's it's and that's become very frustrating to me. There are some things about objects though, that I, I like, and I, I don't really miss going back into the OO world from, from FP is just, I don't know, there's some, some, I guess for me, there's, there's an easier, I have an easier time organizing the code and knowing where functions belong. Cause I just put them with the data that they work on and then sometimes in FP, I'm, like, hmm. It kind of goes with this concept, but it's working on that data and I don't know where to put it. So I don't know.
Chris: I thin, yeah. I think, I think that's, I think that's true. I think what you're describing is true. I think though, in the sense that I think the notion of having behavior right next to the thing makes it easier to sort of know where it's supposed to go, kind of, but I actually don't think that that's like what you want to do most of the time. Like most of the time you do want it to be an abstract thing. Like abstractions are good. Abstractions have gotten a bad rap. Like people don't, people talk a lot these days about like, oh, you know, everybody reaches for abstractions too early or whatever. Like there's these like, memes about like about abstracting things, but like part of good design and being a good designer is tearing apart what the core thing is that you're talking about, right? Like what it is like the sort of err principle that's inside of the thing that you're building. And then you actually do want to make that a singular function and make it open to be used by any type of data structure or any type of thing, right. You actually want to make that like a general thing because that's how you, that's how you grow a system. And so in some ways I think like objects are like this huge trap because all the, all the ways that are like the most convenient or the most obvious way to do it, or actually is sort of like the worst ways to do it. And they're limiting in that way, because I think you're right. I think you're right in that it's like, oh, it's obvious, like this goes with this because this state is over here inside of this object thing. So these methods would enact on this, on this object, right. Or whatever. But I actually think that's, that's like not what you want to do most of the time and what you want to do is actually build a general thing. And so you have, you know, OO, I'm, you know, I'm not a moron. I realize that like, OO languages support this via some sort of, you know, abstract class thing or some sort of, you know, um, um, for getting the, what was the name of it in Java, uh?
Amos: Interface?
Chris: An interface, right, right right. Yeah. Or, you know, Ruby's got it’s like mix-ins and whatever, uh, to be able to support all this. But the problem with all that stuff is, is it then, um, it, it moves around all of the different sort of like bits in such a way that it's much harder to discover that stuff. I think that's true in functional languages as well. Um, that's like the trade-off that you're making, but especially in like the, in like Ruby right. Where you just like mix in a bunch of stuff and it's like, where did these methods come from? Like, I don't even know anymore. And then like mix-ins can override other mix-ins because, and they get stacked in some arbitrary way. It's like, yeah, deeply confusing to me. And you got to look in like 10 different files to figure out where anything came from, why anything's doing anything.
Amos: Right.
Chris: But you have pipes. So it's all, so it works.
Amos: It's all good. It's all good. Now that we have pipes. The other, the other thing that I run into is so, so I, I not sure what term I enjoyed FP before I started doing solely FP.
Chris: You like FP, you know, before it was cool. You're an FP hipster. A fipster.
Amos: I would say, I don't know that I would say that at all. I don't know that I was that early, but, uh, but I, I enjoyed the principles of FP before I was doing FP full-time right. And now that I've done it full-time and also going back, that's the other thing, like, I totally get all the people who had been doing FP for a living who talked about functions being the primary abstraction and, and how that's, where the power of abstraction and building things together comes from is actually at the function level. Not at like some object level. And I was always like, yeah, okay. Kind of, I get it, you know, innumerable, uh, no, now I see it all over. I'm like, whoa, well, I just to build that function out of this function with this function added to it and like, and, um, it's almost like function wrapping in composition all the way through. And I think that things like Ruby with the blocks gives you good ways to do that, but-
Chris: Yeah. More so than other languages, sure.
Amos: Yeah. But it's still not utilized in my experience in that as deeply as once you end up in a functional language, just because there's other ways to handle it, they get ingrained into OO, but I find that doing those and building up functions, even with objects in there that abstraction of functions can be really powerful, mixed into that.
Chris: Yeah, I think that's, I think that's true. I'm thinking about this a little, this a lot. Like, um, just what are the, what are the value adds of, of functional programming at this point, right? Like where w you know, where's the benefits of it or whatever, and like, how do you talk to people about it? Um, and it, it, it seems to me that there's a lot of people who, for them functional programming, I think probably the majority, if I'm going to be honest, like the majority of quote, unquote functional programmers, they have a fairly like modern view on what functional programming means. Um, and so do I, but, you know, everyone takes for granted that functional programming means immutable data structures at this point, um, which is like a super new, relatively new concept, especially when you talk about having good immutable data structures, like cons lists are garbage. Um, they're not a good data. They're not a good data structure, but
Amos: It's the primary data structure.
Chris: They're crap. They're total crap. Like they're, they're, they're garbage data structures. Like if you built a language today, so like, oh, you know, Elixir is fine in that it uses, it uses lists basically because it needs to inter-op with Erlang. And so it's like worth a worthwhile trade-off for Elixir to make. Elixir's right to make that trade-off. But if you're designing a language from scratch today where you don't need to inter-op with Erlang, you should a hundred percent not use cons list. Cause they're terrible. They're awful data structures.
Amos: What, what, what would you use?
Chris: Vectors! Vectors! Vectors have so many better properties. Like immutable vectors have better properties across the board, every, for practically every use case .
Amos: Let's hear them.
Chris: So the benefits are that you can access at any element in a vector in, oh, log 64. Oh, log 32 in time. Right. So that's pretty, pretty darn close to, just to consistent to O(n) or to, O (1) you know what I mean? Like you're getting pretty close to, to that. It is slower to add to the front of a list, uh, than it is to end to take from the front of it, of a list than it is a cons list. It's long time to get, uh, to get the front of the list, but, or, sorry if I'm a vector, but it's equivalently fast to put something at the beginning or put something at the end or put something in, in the middle or split.
Amos: It doesn’t matter where you put
Chris: Or to catenate, it doesn't matter. It's just always like the same. You always get the same, the computational expense basically. And so it's like, to me, I will happily trade off the fact that it's a little bit slower to access the front of a list, the very first element in a list for all these other benefits and like the ability to concatenate things and the ability to like split things arbitrarily and all this kind of stuff. I'll happily make that trade off any day of the week.
Amos: Okay. So I'm trying to, I'm trying to picture how this is implemented in, internally and efficiently in, in an immutable language, right? So like, if you want to insert into the middle of it. So with, uh, with a list you have to go to that element, you have to copy everything to that point and then point to the new element. And then the old element can just point to the rest of the list and you don't have to go through anymore.
Chris: Yes, yes. The O(n) That's the worst.
Amos: Okay. The beginning is super easy, right? Cause you just, you're just, you have nothing to copy. It's the copy that's the problem. And stepping through, but, so, so how, what is, what does this structure look like? Can you visually explain it?
Chris: Uh, it's hard to explain it's, it's a, these days sort of the, I think the, probably not the best, best one at this point, but the, the one that most people implement for vectors is as an RRB tree.
Amos: Okay. I was going to guess it was some kind of tree.
Chris: Relaxed radix balanced tree, but it's a variation or it's a, I would say it's like an improvement on a hashrate map try. It's an extension to hashrate map tries and, um, yeah. And that's kind of what gets implemented in a lot of these different scenarios. And so they're really good because they give you all these nice benefits. Um, and they're just a really, really wide they're a really, really wide tree.
Amos: Okay. So let me from my head, cause I haven't really read any of the papers and like hamps on any of this. So my understanding from talking to you and other people, and maybe I should finally go and read these is that it's really just like a tree that has a set level. And then at the end of that level, at each point is kind of a list. It is a list at the very bottom, right?
Chris: Right. So it's a uh, I'll pronounce it. Try. Um, just because that's, that makes it distinct from saying the word tree. Some people say it differently or whatever.
Amos: Its t-ri-
Chris: It's the demon damon problem all over again. But in any case, it's jif-jeff, jeff-jif, Jif- GIF.
Amos: Perfect.
Chris: So it's, they're tries, which basically just means that the important data is stored in the bottom leaves. Like that's all a tri really is, right, is, is important data is stored in the leaves, like in the bond of the leaves
Amos: The rest is more like how to get to the data at the top.
Chris: You can put metadata in the, you can put metadata in the intermediate nodes and often that's what happens. But the terminating nodes are what are, where all the important information are stored, right.
Amos: Right. So the tri doesn't have to rebalance and stuff all the time to stay fairly flat.
Chris: Yeah. Well that's one of the benefits, right? And then they have constant branching factors, right? That's when you're talking about depth, what they have is like, they have constant branching factors for the, for the bottom, for the bottom leaves. And they're very, very wide. The ones in Clojure are, they have a branching factor of 32, others use branching factors of 64. It's typically common to pick a branching factor at the power of two. And it just so happens to be that because of the branching factors in Clojure, they're able to fit in like a cache line in the JVM and all this kind of stuff or something they're, they're, they're optimized on the JVM because they fit into a contiguous bit of memory correctly.
Amos: Makes sense.
Chris: So in any case, they have very, very wide branching factors and the branching factor allows you, it gives you is what allows them to be fast, right. Because they're so wide because if you think about the branching factor, right. If it's, O, log 32, what does that mean? Like, it means that you need to increase 32 times the amount of elements before you incur, before you go up another, another step, so to speak. Yeah. Yeah. Before you add another like thing that you need to do, like another jump you need to make another operation, you have to take. Right. So, you know, it takes 32 times the amount of elements to go from, O 1 to O 2 and then 32 more times to go to O 3.
Amos: And this grows with your branching factor. So if you have a branching factor of 64, then it's 64 times. Right.
Chris: Right. And there is an inflection point between those. And then, and that's when you start getting into sort of the, like the kinds of optimizations that honestly I'm pretty bad at, at, which is like knowing how L2 CPU caches work and this kind of stuff, to be able to like optimize how these things fit into memory and fit into different like registers and those sorts of things, right. Like these are the, those are the, those are words that I don't quite know how to really talk about intelligently. So I won't pretend that I do, but I mean, all that stuff is, is held in tension And if you read like through the, uh, the, the, the RRB tree paper, it talks about this. It talks about finding that the appropriate branching factor and that sort of stuff. But in any case, they're very, very wide. And when you make copies, there's a ton of benefits to that, uh, for having them be really wide, because it also means they're very shallow. And because they're very shallow and very wide, you're not copying much stuff. You're really just copying like three nodes on your way down and you're retaining all the other stuff. So the copies are very, very small as well. So they don't take up a ton of room in memory. So that's another major benefit of them. So you get all these other access patterns, like all these things that you just can't do in lists right now, or you will never do in lists because it's inefficient to do them in lists. You not just do them. Now, you can do it. Now you can access a vector like an array, right. And not be scared of that. Like, you know, most people, like if you even, this has come up on Advent of Coe, like there's tons of problems where you're, where you're really working, where you want to work on something, that's list shaped. But you're, but what everyone in Elixir is doing is using a map because, and with like indexes, essentially with it, with like an integer index into it, because that's a more efficient way to go and access it. It's like, well, what if you actually had a list shaped thing that worked, that did what you wanted it to do and had the correct, like time complexities and all this, like other stuff to be able to talk about it and use it ergonomically, right, you know. And unfortunately like, like again, like I said, at the top of this Elixir makes the right trade off. It's, it needs to inter-op with, with Erlang, Erlang stuck with this garbage data structure, because Erlang's a product of its time and we'll never change it because that would break everything. And so that's fine, whatever, like I'm okay with that. I'm cool with understanding that, but it's a bummer because it's like, there are, there's just like so many better data structures that you could use for a lot of this stuff that we just don't because we need to inter-op with this old VM.
Amos: So, uh, is, is there an implementation of this in Erlang or Elixir that you know of?
Chris: I don't know, there are persistent vectors, which are like the Clojure version, which aren't our RRB trees cause Clojure predates that stuff, kind of, because Clojure like took the Bagwell structures and then made them immutable and invented this whole thing. And so it predates all like the work that's been done on immutable vectors since then-
Amos: Do you know what, like-
Chris: I mean, but I'm building one at the moment off and on.
Amos: Always.
Chris: Because I want, I want it to exist because I want it to be there. So I'm working on an implementation right now, but it's, it's like, you know, projects are a stack, right. So I keep putting more stuff onto the stack and then popping it back off to get back to the vectors. And it's, you know, it's just slow going. So
Amos: That's life, that's life. So in implementing it, what are your thoughts on like, what's the conversion time from this data structure to a list. So if you wanted to use this for the main portion of your application, and maybe you have to inter-op with Erlang at some point, uh, what, what is the conversion cost of going from a list to this or from this to a list? Is it-
Chris: Uh, to build it? To build it?
Amos: Yeah. Like in, in your program, like now I need to get down to Erlang, so I need it to be a regular list
Chris: Log (n), I mean, it's O (n), right. You have to, you have to go through the entire data structure and build it. Like you can't just like kind of convert in place. Like you have to go through each thing and then build it.
Amos: Okay. Yeah. I didn't know if there were, it didn't sound to me that there would be a whole lot of optimization there, but I didn't know if there was something
Chris: And it really only, you know, it really makes a difference when you want access. Like when you want to be able to go to a place and then update in place or, you know, get something at a specific index and you want that to be ergonomic. Like you want it to be, you want to talk about it as a list. You don't want to talk about it as a map. Like you want to be able to map over it and not have a tuple that you want to interact with now that has an index on it. You know what I mean? It's like, like you want to be able to just sort of like access it the way you would access any, any list.
Amos: So I'm going to bring up the structure that should not be named in Erlang is, is what I'm always told. But what is the advantage of this over something like the Erlang array module? Was that implemented as a list underneath and it's just hiding it from you or-
Chris: I don't actually know. I don't know how arrays are implemented in Erlang.
Amos: Okay. I didn't know if it, yeah. I'm my guess is that it's going to drop down into C and it's going to be pretty native.
Chris: Yeah, I don't actually know. I actually have no idea. I'm not sure, it’s something I've meant to look up, but yeah, I've never, never delved into it.
Amos: I've used the array module once. Ever. Um, and it, for my reasons that I never ran into any performance stuff, I didn't have very much data in there.
Chris: Yeah. Well, and at the end of the day, most people don't. Like most people aren't passing around lists with thousands of elements in them, or even hundreds of elements in them. Right. Let's just like, not a thing that happens that often. And so it's also kind of one of those things where it's like, they are garbage data structures, but you kind of don't notice it that often because like, you know, if you have less than a thousand things in it, it's like, what's, what's the, you know, what's the big problem.
Amos: Right. All right. So I'm going to go back to, to the, what I miss from Elixir when I hit back in Ruby. All right. If you're in tmux, and you have your, a Rails application, let's say, it doesn't matter what it is.
Chris: This is a highly specific example
Amos: It's got, well, I don't think it's highly specific, really. Anything that would lock that, that terminal, to stop the IO from being able to go out actually locks all of Ruby. Like you can no longer serve requests or anything until you allow that terminal to start printing again.
Chris: Hmm. That makes sense.
Amos: Yeah. Yes. Yeah. As soon as I saw it happen, I was like, oh, I know why this is happening.
Chris: Oh, right. I remember
Amos: I forgot how this, this works. Uh, and, and in, in that same vein of doing that is like there are problems that I'm trying to solve right now with people. And I'm like, well, this is just MapReduce. Spread it out. Oh wait, that's not easy.
Chris: Yeah. That's actually really hard as it turns out.
Amos: In Ruby. Yeah. And I got so used to being able to in Elixir, just be like, oh, well, we can do it yourself with GenStage or use flow or, or whatever. But you can spread that out amongst all your cores pretty quickly, pretty easily. Even with one, one, one application. And then in Ruby, it's like, well, we've got to have background jobs-
Chris: Now I need a Reddis. All I wanted to do is two things at once. And now I've got a Reddis involved.
Amos: Well, and then you end up at that point, so like that whole reduce thing, right? Like I, the one problem I'm trying to solve right now, as I pull a bunch of stuff out of the database and I have to generate a bunch of PDF files, put them into a folder and then zip them up. Well, right now that folder is a temp directory, but temp directories are owned by the process that creates them. And then they go away. So if I have background jobs, they're all different processes. So one of them creates that directory and nobody else can access it. So now I have to have like some kind of name directory that I'm passing around everywhere.
Chris: Yeah, it sucks.
Amos: And then I need to know when all of these jobs are done that are completely disparate. So I need to know like a count of how many jobs there are at the final job to be able to know, hey, everybody's done. So now we can go ahead and zip this up. And so there's just all these other problems that I have to think of. And I'm like, well, that sucks. I, I just want to rewrite this.
Chris: Right. I have, no, I feel ya. I'm not disagreeing. I think it's one of those things that you don't realize- this kind of goes into what I was talking about last, last week, or whenever we recorded last about Elixir being sort of a boring choice. And this is kind of like, and I think that's like such a good thing and this like, and this goes, this goes into it because it's like, oh yeah, you just like enum dot map, tasks dot start or task dot async enum dot map, tasks dot, await, collect the results. And like, just be done. And like, it's like little things like that, that all of a sudden start realizing like what power you have in, and at the runtime level to be able to do certain operations and like make them really efficient. And you just like, you're so unburdened. Uh, but not in like a really dangerous, like, you know, now I've just got the power of threads in the palm of my hand and we'll, and we'll, you know, use them, uh, to, to wreak havoc upon the world. Like, you're just like, nah, it's just like spin up these processes and they just do stuff. And then, um, and then it works. Oh, I'm done.
Amos: Well, and, so I, I wondered when I, when I was jumping back in to doing this-
Chris: But I mean, the problem is Elixir is not optimized for happiness, like Ruby is, so
Amos: I dunno, Elixir makes me pretty darn happy.
Chris: I mean, you know, what makes me happy? You know, what makes me happy? Software that works.
Amos: So I want to get at though is-
Chris: Concurrency. Doing two things at the same time
Amos: That, that does make me happy.
Chris: You know what's never made me happy,? Debugging performance problems, single-threadedness and nil, like exception, no method nil on whatever, whatever exception. That's never made me happy Ruby!
Amos: So I'm going to jump back though. Debugging in Ruby can often be easier.
Chris: No. I disagree with that. I disagree with that. 100%
Amos: Because, because, well, okay, so in Elixir, you have, let's say you have just two processes, right? And you want to debug something and you, so you put a break point in one of them using pry. And
Chris: See this is your problem. You've already failed. I already know. I already see, I already know why you have a bad-
Amos: Habit. Habit. Because then that other process is still running
Chris: Because you're trying to debug, like, it's Ruby.
Amos: I know!
Chris: You gotta, you gotta, you gotta, you gotta, you gotta get galaxy brain on this.
Amos: Uh, I start jumping in with like recon trace and stuff like that.
Chris: See, there you go. You gotta get all on the trace, I mean, the thing is, is like, I think probably not enough people know about tracing just generally, and recon trace specifically.
Amos: Well and pry being right in the core makes that the, like, the thing that people go to.
Chris: I literally never used that. I've literally never used it.
Amos: I've used it twice. In five years?
Chris: It’s probably one of those same things that I don't use that everybody else uses is what I assume. We use a ton of recon trace both locally and in production.
Amos: I'm more likely to use a put statement than use pry whenever I'm in Elixir and in-
Chris: I have this really cool debugging tool that I use actually. Um, it's called my brain and I sit there and I think about the problem and then I solve it
Amos: And go on walks.
Chris: Yeah. Yeah.
Amos: Perfect. Perfect. I I'm, I'm on board with that. The brain is the best debugger that you have.
Chris: It's a, there's a repo for my brain on my GitHub.
Amos: I'd check it out. But I think it would hurt me
Chris: It’s filled with all sorts of, you know, random, random NPR voices and references to old movies and weird TV shows.
Amos: So I'm curious though, in, in going back to Ruby and not to, not to completely ignore your comment about random TV shows in your brain on GitHub, but, like, now that I'm going back and I'm doing these problems, I often wonder, cause I remember solving those problems before in Ruby, but, or in other, even in Java, like I avoided making threads and stuff like that, right. Because it's just painful. Uh, even though
Chris: Its cause there's tons overhead. Even in the language where they actually work, which is to say like Java, right. And not like Ruby, like where you have real threads and can do real things and yeah. All you Ruby boys coming at me with like, well actually the Gil unlocks for IO. I know it doesn't matter. It's still -
Amos: Kind of.
Chris: Kind of. Okay. Miss me with that. Anyway. So yeah. Even in a language that works like that has real threads, like Java, like threads have tons of overhead, like, like a ton of like memory space.
Amos: Usually I don’t need a ton of parallel IO. I mean parallel processing of data, not IO of data.
Chris: In any case. Yeah. In any case, threads in Java have, have their problems as well, just because of overhead.
Amos: I wonder sometimes is, is going back to the, um, I just-
Chris: Are you the problem?
Amos: I mean, really, am I seeing all of the problems in this light because I've been doing Elixir lately and there are other ways to solve it? And are those other ways better? Maybe, maybe not. I don't know.
Chris: For certain problems. I mean, for certain problems. Yeah. The other ways are better specifically, like, you know, it's li-it's mitigated now with persistent term. Persistent term is a pretty great addition to the, to the runtime. But like, you know, ETS isn't good enough for certain scenarios. Like you really actually do want global access to memory occasionally. Specifically you want global access to memory that doesn't get copied into all your processes, right? So if you've got, let's say for instance, like a big graph data structure, and you want to be able to traverse the graph and you'd want people to be able to traverse the graph. You don't want to copy the whole graph into all of the different places that you need access to the graph. Like for instance, if you're going to do a graph traversal and a web request, and the graph is let's call it, I don't know. Let's, let's make it small and say that the graph is like 400 megabytes, right? Well, three of those just pulled a gig of, a gig, a memory. Like three web requests now just pulled a gig of memory if you have to share it. And in Elixir and Erlang you typically either ha you have to share it, or you have some hack where it like goes in ETS.
Amos: Well, and even in, in Ruby, what I see people doing is they're not sharing it anyway. They rebuild it a lot of times at each request. And then you have to take the time hit on that.
Chris: Right. But so, but my point just being that, like, there are certain scenarios like that where you have this piece of data that you know is never going to change. You're going to change highly infrequently, like the graph examples. Like it's probably one of those things where you, you start the box, you build the graph, you put it into memory and then you start accessing it globally from all these different places, right. And you're not changing it, you're just walking over it. So there's no, it's, you know, perfectly safe to share it across everybody who needs it or share a pointer to it or whatever. And Erlang makes that really hard. And cause like, if you want to do it efficiently, historically, you had to do it in ETS. Or, you know, I guess you would use diagraph, but that's going in ETS right. And then all of the operations are happening against the ETS table. So they're all super non ergonomic. Like you can't just like map over the data or like, you can just recurse over the data structure anymore, because now you're, you're making all these ETS calls because you have to, um, and that's really a pain and that's a, that's a, that sucks. Uh, that's a, that's a, that's a bad part of its, it's, it's a type of operation that really sucks to do. Um, when you can't have shared memory, um, which is, you know, again, like persistent term now allows us to do, like, you could have this like giant data structure that never gets copied, but just gets pull, you know, but you're able to access from all these different processes now.
Amos: Do you find that you're using that, um, by regularly, I don't mean like every time I go to do something I'm reaching for persistent store, but do you, do you find that you're doing that pretty regularly in your job, that persistent store stuff.
Chris: Yeah. We use it in a bunch of places now. Um, we use, uh, we use it for a whole host of things, but a big part of it is because yeah, we don't want to, like, we would have used ETS before, or we did use ETS before and now we use persistent term because it saves us like that amount of memory, right. And it saves, and it's faster than doing an ETS call. Uh, you don't have contention on the table or anything like that, which is a thing that can happen if too many people are trying to read or write to it at the same time or just read. Then you got to do read optimized tables and all this kind of stuff. Where you just don't have those problems anymore. It's a lot easier to get access to it. And you don't copy all the memory everywhere. So that's a, that's a big benefit. We, we have, we, we have some like really rudimentary, like we do a look up, we, we scan some texts do looks at lookups against like a map of words to do search stuff and all this kind of these kinds of things. And we build this big data structure to be able to do that efficiently. And that goes in persistent term and then every web requests can access it out of persistent term and those kinds of things. So yeah, we use it for all that kind of stuff a lot.
Amos: Cool.
Chris: It's a really useful tool, but yeah, but this is an example though, of like where you have to make this allowance in the VM now, you know, in the runtime to allow you to do this, because it is a use case that you actually do want to support and a language that has mutexes or whatever is actually sort of like, that's like a, you're better served by that, for this specific kind of use case. Um, so there are times where it makes more sense to access things in that way, which is good. You know, that's, that's just is how it is. That's just, you know, like there are use cases where you want this big one singular shared group of memory. It's not often, but it does happen,
Amos: But managing it sometimes the overhead of that, it's pretty crappy. The mental hoops.
Chris: Yeah. In your back end mutex semaphore you know, land. And that sucks.
Amos: Yeah. And those are not fun to debug either.
Chris: Yeah. Well, well, and the other thing is you have to, if your problem isn't such that you can basically go when this thing boots up, build the data structure, put it in this one place in memory, and then everybody can access it. And we're never going to touch it again throughout the lifespan of its existence. Like if you're changed to the, to the, if changing the, the data structure is equivalent to kick the service over and let it reboot, like then you're fine. That's a really good place to be.
Amos: Right. Read only.
Chris: Pretty much. Yeah. Basically. Yeah. If you, all of a sudden are also having to change that data structure in the middle there, then it's a nightmare and you don't actually want to do it. And then actually you're actually better off doing it still with processes and with persistent term, like that's a much more tenable understandable way to build that problem or solve that problem. And for your benefit, for your benefit or for the problem that you're specifically working on right now. Sure. You've probably, you've seen the light. Like you've, you've, you've gone to the promised land where you're like, this is pretty sweet. Like I can just do things now, unencumbered. And they're like by and large safe. And like, they can't like mutate other people's state and they run pretty well. And I only have to run one binary. I don't need 20 unix processes just to serve 30 web requests. And you know, all these other benefits, all these other things and you go back and you just feel hamstrung. Like, you feel like you, you feel so debilitated because of the runtime. That's how I feel whenever I work in Ruby now, occasionally I write scripts for myself and they get written in Ruby, but it's like, we have some Ruby in production here and it's like, every time I'm working in that stuff, I'm just, it's just like, it's like working with, you know, three limbs tied behind my back where I've got like, and I'm like having to type with like my face. Like, you know, it doesn't make any sense and you just making these decisions and you're doing these things that you're like, this is gross. I would never do this in a real language.
Amos: Uh, so I have the same, like I have that output. I have the same thing with ingesting large amounts of data. Like at the same time I need, I want to spread it out. It all, everything seems to come down to, if we could do this in parallel simply.
Chris: Right. If you could. Imagine a world, imagine a world wherein you could do that.
Amos: And some things are pretty, I will say some things lend themselves well to background jobs, but not everything is simple to put into a background jobs.
Chris: But what if you just didn't need to do that?
Amos: Right.
Chris: You know what I mean? But that's, that's the takeaway. I dunno, this has become a, this is not interesting to discussion anymore because it's now it's just become like, uh, uh, uh, it's just become thrashing on, uh, other programming language that, uh, ostensibly is wildly more popular. And I know, and it has way more people using it and all that kind of stuff. And people love it and all that. And, you know, good for those people.
Amos: I still do like it. I don't love it, but I like it. I used to love it. And I think a lot of my love-
Chris: Or some definition of love that involves satisfaction.
Amos: I spent so much time digging through like the object model and how Ruby actually builds things up. And I started Ruby before Rails, so it, it gave me a lot of-
Chris: That's a weird flex, but okay.
Amos: Well, I was doing COBOL and Java when I discovered Ruby, so.
Chris: Weirder flex, but ok.
Amos: It's way cooler to do Ruby. It made my, it made my day job feel a lot easier. And I was actually, there, I don't want to talk about Java.
Chris: No, no.
Amos: No, not really old school web stuff. Like before Spring Java.
Chris: Yeah. Struts.
Amos: That's what I used. Struts.
Chris: I've worked on a Struts app
Amos: And iBATIS.
Chris: That's a word I've never heard before.
Amos: It's an it's SQL.
Chris: You just made that up.
Amos: No, I wish.
Chris: You're just making up words.
Amos: I wish I did not live through that.
Chris: Whatever grandpa.
Amos: It was an ORM ish thing. Uh, maybe not, maybe more, just like a SQL manager. That was all XML. It was horrible.
Chris: Oh, well, you know, clearly it was cutting edge.
Amos: That's, that's how you do things back then. Right. XML.
Chris: So much XML.
Amos: And then, and now all we've done is drop some of the tags and called it Yammel.
Chris: Well, I was going to say, we've dropped, we've dropped some, uh, we've dropped the, the, the, you know, the gator mouth brackets, and we replaced them with squigglies and now we call it ProtoBuff.
Amos: Perfect. Oh, well, my nose is sounding like it's time for me to take some more medicine, so we might have to get out of here.
Chris: All right.
Amos: All right. Thanks for filling me in on a vectors today.
Chris: Yeah, don't- friends don't let friends use con lists cons lists.
Amos: And letting me gripe about MapReduce and (inaudible)
Chris: And the good news is you have pipes now. So it fixes most of the problems with Ruby. And it's with- and definitely you shouldn't use Elixir now because you have pipes here in Ruby, and that really is all you need.
Amos: Right. Right. Now I can use dots.
Chris: I'm making that joke, but did they actually add pipes to Ruby?
Amos: Yes. Yes. Uh there's there it's, it's just a different precedence than the dot.
Chris: But it's dot.
Amos: It's dot with a different precedence. Yeah.
Chris: Got it.
Amos: And they also, um, I haven't seen it yet. I had a friend coming back and telling me about it. They've also added some, some pattern matching that you could use in certain contexts, but I
Chris: Yeah. So you nailed it. See exactly. That's all you need. And now there's no reason to use Elixir everybody. We did it.
Amos: Yeah. I don't think the pattern matching is, is at the function level from my understanding it's more like a case statement, but that's not bad. Yeah. Not terrible.
Chris: Oh, yay. It's good. You know, Perl has like Perl six has like a real concurrency model now. You can just go use that.
Amos: Oh yeah.
Chris: You want to go back in time.
Amos: Then you get all your regular expression power too.
Chris: I don't know. Perl, uh, I, if you told me today, you have to write Ruby or you have to write Perl 6, I know what I'm going with and it's not Ruby.
Amos: Whoo! Those are fighting words for some people.
Chris: Perl 6 is cool.
Amos: I haven't looked at it. Maybe I should take a look.
Chris: Rakudo Star or whatever. However it's pronounced.
Amos: I'll have to ask, uh, Alan Voss. I just wanted to mention his name, make him feel good. Thanks, Alan. Alan. Alan's a big Perl guy. Well, he does a lot of Elixir, but, and Ruby, but he likes Perl.
Chris: Okay. Yeah. Cool.
Amos: He can wrangle a regular expression with the best of them. Which I'm not sure is a superpower or it looks like you've been in a concentration camp.
Chris: It's a superpower of sorts. It's a superpower of sorts.
Amos: Maybe. All right. Well, it's time for me to go make a turkey sandwich.
Chris: I just got done eating one of those.
Amos: Nice, nice.
Chris: Well, I hope you start feeling better.
Amos: Thanks.
Chris: Later.
Amos: Bye.