They are joined by Aimee Knight and Joe Eames. In this episode, Chuck and John-Daniel discuss how (and why) Raygun migrated to .NET from Node.js, plus some interesting and little-known facts about frontend frameworks.
If you like lots of real-world examples of how different frameworks work in a software company, look no further. This is a technical and high-level view of how Raygun improved our throughput by 2000 percent.
Listen to the full podcast below:
This podcast outlines:
- Why we moved Node.js to .NET Core for our API.
- The results of the move.
- Managing the migration.
- What does the rest of the Raygun stack look like?
- Thoughts on TypeScript.
- A learning curve for the migration.
- Why application performance matters.
Joe: Hey, everybody.
Charles: Aimee Knight.
Aimee: Hello from Nashville.
Charles: I’m Charles Max Wood from DevChat.tv, and this week, we have a special guest, and that’s John-Daniel Trask.
John-Daniel: Hi, everyone.
Charles: Do you wanna give us a brief introduction? I don’t think you’ve been on the show for quite a while.
John-Daniel: It has been quite a while. So, you can probably tell from my accent, I’m not from America. So, I’m from New Zealand, although, I hail now from Seattle. I split my time between New Zealand and here. And I’m the co-founder and CEO of Raygun at raygun.com. Most Americans think I’m saying Reagan, as in the old President, but yeah. So, I’m up here, lots of experience as a software developer, although sadly, doing a little bit less and less as the company continues to grow.
Charles: Gotcha. If I’d known you were in Seattle, I’d ask you to go grab lunch. AJ and I were up in Seattle for Build Conference last week.
John-Daniel: Oh, right. Yeah. Well, I had a good number of meetings with folks up here. It was a good event from what everybody was saying.
Why We Moved From Node.js to .NET Core for Our API
Charles: Yup, it was fun. Anyway, so we were discussing before the show what we wanted to talk about, and one thing that you mentioned that was kind of interesting is that you moved away from Node.js to .NET Core for your API. And, in a lot of cases, what we’re talking about is why a particular tool, or a framework, or in this case, Node, solves a problem, and we don’t talk a lot about, “Okay, when is it not a good fit?” It might be really interesting to talk about this. I know some people are gonna get their feathers up a little bit, you know, as we talk about their favorite way to code. But it sounds like there were some rather specific reasons for that and not necessarily because .NET is better than Node.js. So, do you wanna dig into, yeah, kind of what led you to consider making that decision, and then what the result was?
Aimee: Can I kind of ask a little quick question too, with that? I have heard of a lot of people going from like Node to Go so I’m curious if you explored that?
John-Daniel: Yeah, absolutely. So, yeah, to try and head off any of the religious wars, the Raygun platform itself, we do use a bit of Go in places, we use a bit of Node, and we use a bit of Full Stack .NET, and we use a little bit of .NET Core, and, obviously, we’re using Node.js as well for a part there.
It’s probably best done by explaining the history of how we even had Node in the mix which was that our core competency, at the time, was sort of writing code with C#. And so we looked at building our API layer using Mono, the open-source version of the .NET Framework. This was many years ago now. This was in like 2013, I think.
And the reason we wanted to look at Mono rather than the full stack of .NET was that our API layer is relatively dumb, right? It’s an auto-scaling group that receives billions of requests a day and, largely, receives it, validates it, does a couple of other checks, and then throws it onto a queue for processing. So it’s a pretty dumb layer and, at the time, you know, .NET didn’t run on Linux, but Mono did. We didn’t have any desire to run Windows in that sort of mass manner. And so we looked at Mono, and my goodness, welcome to “memory leak city.” It was just a nightmare trying to maintain that cluster.
So, at the time, we were sort of looking at the pros and cons of different languages and runtimes. And it seemed that Node.js offered a really nice model for short-lived sort of transactions that were running there. We shifted off Mono onto Node.js and we just had a much more stable environment. We got similar-ish throughput so it wasn’t like a major performance win, but we had that in there.
Now, fast forward some time, and it was really just through our CTO, Jeremy Boyd, sort of doing…he does a bunch of our R&D work. Looking at .NET Core when it was announced that, you know, he was looking for a small project to have a look at and so he started building a clone of our API and seeing how that would look in comparison, again, just out of personal, “Hey, I’m comfortable with the .NET stack and I want to learn .NET Core, and see what’s going on.” And the major win was when we put it in and we were like, “Wow, this is using a lot less resources per request and handling things pretty well.”
Now, what I think some people may or may not know, and I’m personally excited by this because I get off on high-performance code, is the lengths that Microsoft are going to, in terms of the performance story around .NET Core. They’re going through and looking at every allocation that, you know, the base class libraries and all that I’m making and trying to make it as performing as possible. I mean, I don’t… But now in those…you know, those… I think they’re empowered benchmarks for web requests and things. They, obviously, want to try and land in a good spot on there. And so we’ve benefited from that.
Results of the Move
So, we put it in, and largely, we ended up going from approximately 1000 requests per second per Node, and when I say Node, I meant server, to around 20,000 requests per second being handled on there [.NET Core]. Now, the large win that seems to have occurred in there, aside from their work on performance, and I know that it’s now in Node and supported, was the asynchronous handoff to the queuing service. So, keep in mind the timeframe there. We were looking at Node years and years ago. Not having a great threading model for how we pass that data off to the queue that could keep the thread… You know, effectively call-waiting for the response back from the queue to acknowledge that it accepted the data. So, that was one piece, but there was all of the performance work in there, and so that speed things up.
Having said that, at about the time we were putting this in, you know, I started seeing everybody going, “Oh, cool, the Async stuff is starting to land in Node.” So, you know, it’s potentially questionable with that around…if we just waited and waited until later it turned up, whether we would have managed to achieve a similar outcome. So that’s kind of the Core, that’s the condensed version. Any questions on that?
Aimee: Not for me.
John-Daniel: So, I know it’s quite the long story. Had to take you on the journey.
Managing the Migration
Charles: So, one thing that I’m wondering about with this is how much parity is there between Node and .NET? I mean you talked about the Async kind of being the killer feature for you but I mean, was the rest of it pretty straightforward to migrate or, you know, somebody’s looking at making a migration like this, are there certain gotchas that they have to be aware of as they move off of Node and onto maybe a different concurrency model or things like that?
John-Daniel: Well, the concurrency model stuff was, certainly, something to consider at the time. And further to that, at the time with .NET Core, you know, what Microsoft was doing is they delivered this extreme subset of what you would have enfolded on it. And so, at the time, we were looking at it, oftentimes, what you would run into is more of that, “Hey, this API that I would kind of expect anything, you know, any modern development environment to have doesn’t exist.” So I’ve either got to code it up myself or, you know, figure out a way to work around it.
So, from that point of view, at the time, Node.js had a much richer offering. Obviously, there’s a bajillion modules out there to do whatever you’d like and that wasn’t the case with .NET Core when we were starting. It was pretty, pretty low-level by comparison. It just was a case that they’ve, obviously, designed it with the concurrency stuff from Day-dot. So, from that standpoint, it was actually a bit of a gotcha going to a more constrained API. (For more on writing high performance .NET code read this article.)
Having said that, you know, our CTO was building this prior to .NET Core 1.0 and .NET Core 2.0 is out now, and not doing this big push at expanding out that API, and make really great gains on that. So, at the time, that was the hard part.
You know, this is going a little a bit away from our API but was considering this the other day. That I, largely, feel like Node.js is, and this is not a put-down to people doing, you know, actual real work with it, but I feel like Node.js is the go-to place for people to learn programming now. And that you pull something down, you can run it up, and do something very easily. And I feel like a lot of those bigger frameworks and tools don’t really exhibit that, you know? Even things like Ruby these days, with Rails being so large, it’s not really the easiest thing to approach for a new person.
What Does the Rest of the Raygun Stack Look Like?
Charles: So, the other question that I have here, because you mentioned that this is just for your API portion, is: what is the rest of your stack? I mean, you know, so the web interface, is that all back-ended on Node, or Rails, or something, and…?
John-Daniel: So, we use the Full Stack .NET on Windows for the main web app. We use Go for orchestrating a bunch of symbolification processes, so making stack traces human-readable for things like iOS and ProGuarded Android apps for that. In a way, we have Java in the mix just because we’re using things like ElasticSearch, and Druid, and things like that in various places for our data storage so there is a mix. And, you know, the Raygun platform itself supports about 25 different languages and platforms so we’ve got a pretty polyglot team, in terms of their skillset. So we will frequently look at something, in terms of, what is the best tool for the job. But when we were starting out, .NET was definitely our core competency for the app. Yeah.
Charles: So, what’s the thing you miss most now that you don’t have Node.js backing your API?
John-Daniel: That I miss the most? I don’t think we’re necessarily missing anything from it. I think the availability of packages to use, for anything you were thinking of building, you know, that was a bit of a challenge at the get-go. You’re pretty used to, you know, the fact that anything you ran a search for, you’d find somebody had built it before and you could give it a go. I mean, one of the compatibility things at the time, and this has largely been solved a bit with the .NET standard, sort of, contract stuff, is that even if you had something that was written for normal .NET, you couldn’t use it with .NET Core. So even if a package, theoretically, existed for the .NET ecosystem, there’s a bit of a mishmash trying to find a solution - it wouldn’t work out of the box for you.
Charles: I got you. So, you go in and you install a package out of Nuget, and just it may not be what you need?
John-Daniel: Yeah. It would say, “Hey, this is not built for what’s available here.” So, that was a challenge. It’s that usual theme, you know, waiting for an ecosystem to kind of get to its critical mass. And I think, you know, we can all agree, Node is well past the critical mass stage.
Charles: Yeah, and I was at Build Conference last week. I’ve been to a few other Microsoft events over the last year, and I know that they’re encouraging that kind of growth so I think is just a matter of time, if you wanna be in that ecosystem.
John-Daniel: Yeah, absolutely. Again, I mentioned, I don’t write nearly as much code as I used to. I still absolutely love writing code and find every excuse. But even some of the tools, these days like Visual Studio Code, you know, I’ve switched from using Sublime for things when I was, say, writing code for Node to now doing it with Visual Studio Code. I think that they’ve done a really good job with the ecosystem there, even outside of their, sort of, core audience.
Joe: I agree with that. Yup.
Charles: So, one other aspect of this migration that you talked about a bit before the show was getting into migrating from Backbone, and Marionette to React.
John-Daniel: Yeah, to which you all rightfully mocked. That’s for being so…
Joe: It was friendly joking.
Charles: That’s right. A little bit of ribbing before the show. I mean, you know, go with, “Oh, it’s hot, it’s fine.”
John-Daniel: Yeah. When we first started building out the front-end of Raygun, we did it with Marionette and Backbone. And, you know, I won’t lie, I was probably part of the piece that was a little bit jaded to, you know. It kind of got to the point where I was like, “I wanna wait till something has just absolutely cemented itself as the next step to go to.”
So, we’re building out a new part of our platform now. We’re launching it probably late Q3. That seemed like a good opportunity for the team to sort of deep dive into something new. And don’t get me wrong, I’ve been begging to go to something new for quite a while, and so they chose to make the migration to React. And to be honest, I was really impressed…
John-Daniel: Yeah? Jump in.
Aimee: Sorry. Can I get some clarification here? Because you said you’re building something new, but you also said you’re migrating so I’m a little bit…
John-Daniel: Well, yes. So, when you log into our app, right, you’ve kinda got the normal Chrome around all the different capabilities? And so, there’s an element where we have to both migrate the existing work while building out the new bit.
John-Daniel: So, the team started off with saying, “Okay, well, let’s learn it in a green-fields type approach just for the sort of components that are gonna go inside of the normal Chrome of the app, and then sort of bleed that out to the Chrome website. Okay, now, what’s it gonna look like to start rebuilding the Chrome of this?” So, I don’t think we’ve actually pushed those changes, at all, yet to production. It’s more predicated on the new pieces coming out.
Aimee: I’m kind of curious about, you’re using TypeScript with React then? That doesn’t seem like a very popular combination.
John-Daniel: Well, some would people say that Backbone and Marionette aren’t popular.
Aimee: Well, I know. I’m just saying it’s not a combination I hear of typically, given, like Facebook, usually supports Flow, so…
John-Daniel: Yeah. We’ve been using it a little bit just to get up to speed with it and see how we went. There are benefits that the team quite like about it that they haven’t run into too many issues in using it. The big debate, at the moment, is that question around using, you know, Redux for the state management and what they’re gonna do there.
Interestingly enough, some of the concepts that have come back from the team, we have one of the team members who’s quite the Windows Presentation Foundation Wizard, WPF, from desktop Windows apps. And he sort of came to me the other day saying that working with React actually feels quite similar to some of the things he was doing with WPF back in the day around some of the bindings and ways that you can pose up the application, which I thought was also interesting.
Joe: Yeah, that is interesting.
Thoughts on TypeScript
Charles: Yup. Are any of you using TypeScript for anything?
Charles: How have you found it?
Aimee: So, why TypeScript versus Flow? I know, you know, if you guys are using React, that just doesn’t… I haven’t heard a lot of people doing that.
John-Daniel: To be honest, I’m not entirely sure around that. The team, when they were starting to work with it, you know, I was engaged in that project but they just wanted to look… They looked at various things and, ultimately, chose TypeScript with React.
Joe: Fair enough. Anything about that transition that you found to be surprising? I mean, you’re already non-characteristic just by using TypeScript with React, right?
Joe: It’s totally different. Is there anything else that you guys do that is a little bit against the grains of say…
Aimee: And moving away from Node, or .NET?
John-Daniel: Yeah, it’s when I get to the point where I say, “We’re migrating to assembly language directly.”
Charles: There you go.
John-Daniel: I will just touch a little bit on the TypeScript thing. You know, I know that around the time the team was looking at it, they were reading some of the things like how TypeScript is being used at Slack. The team did like the idea of getting some of those tools like IntelliSense type completion that we’re used to and stuff like that, that they really, really liked. That probably weighed fairly heavily on that side of things. The downside, obviously, being that we need those tiers definitions to make use of some of those conclusions, which not everybody kind of has but, yeah, so there was that.
In terms of things against the grain, I don’t think, that… That’s probably the main one. I mean, we’ve always been really, in some ways, painfully pragmatic with how we approach software development. So like I said, not changing the framework just because it’s new and shiny, you know? It’s just because, well, sometimes, I see these companies come out there and they’re like, “Hey, we rewrote all of our front-end.” And I’m like, “Well, cool. So how many of your customers even give a shit?” Like, that seems like a whole lot of wasted effort. Having said that, you don’t also wanna end up where you’re so out of date like we were, rapidly ending up with the Marionette stuff so you’ve got to make a call on that. So, we take a pretty hard-line view of, like, what’s in it for the customer?
And there is stuff for the customer. And that, you know, there were lots of limitations around the Marionette-Backbone setup around how we were having to bring state, couldn’t be easily split. The way that we were doing combining was using a tool, or a framework called Cassette, which was slow and not optimal. You know, we’re still seeing actually Node in our build environment to run grunt tasks for how we build these things and lint them so, yeah. Yeah, I don’t think it’s necessarily that we’re doing things that are particularly weird for weirdness’ sake.
Charles: Well, hopefully not.
Joe: It’s an interesting… It would be a weird way to run your development.
John-Daniel: That’s right.
Was There a Learning Curve to the Migration?
Joe: What about the time and the learning curve on switching things around? Did you guys already have skill sets in those areas when you switched? Or was it a matter of, “Well, we’re doing something radically different, and it really took some effort on our development team’s part?”
John-Daniel: No, to be honest, we’ve always felt… and with the timeframe, you know, when we first elected to play around with Marionette and Backbone, the choices were a lot more limited than what they are today. It did seem like it was a reasonably good choice out of the bunch. But where we ended up at was that the learning curve on that was really, really steep. So, when we would onboard new engineers into the team, specifically the front-end team, it would actually take them a long time to fully sort of get how to work with those frameworks.
Compare and contrast that to even the team picking up and running with React. Like I said, I’ve been really impressed at how easy it’s been to pick up. You know, I’ve played with this stuff as well and I found it pretty straightforward and not historically… you know, if anything, the previous models that we were using drove me away from wanting to touch any of the front-end code and sort of live a little bit more on the back-end. So, I think it’s, to the existing team members, a little bit of a breath of fresh air. And with the way that the component model works, within React, it’s gonna be a lot easier for the new folks to sort of jump on and get up to speed with.
Joe: That makes sense.
Charles: So, you mentioned that you kind of went on Hacker News and saw what people were using that it kind of became the next stage of web development that was stable. Do you feel like if you had done this a while back, you would have wound up with Angular or something else? Or were there other things that you did with it that kind of, you know, helped you make that decision? I’m just thinking in terms of, you know, if people are evaluating a framework like this, how do they know that this is the right choice for them too?
From memory, and this is really straining the old gray matter, when we were first building out the Raygun front-end, you were seeing more sort of data-binding frameworks rather than full SPA-type frameworks at the time. I think Angular was just coming out. It was very, very early days so you didn’t know if that was really a go or a not and so, that’s why we ended where we were there. Like I said, I said, we keep a finger on the pulse of what’s going on out there, but we went… yeah, sort of sat down and said, “Hey, if we’re doing this new green field part, it might be a great time to actually explore a new framework.”
The writing was on the wall a while ago. There was discussion about, you know, “Is Backbone dead,” you know? And there were people that were like, “No, it’s not,” and there are other people saying, “Absolutely, it hasn’t had any updates in quite a while.” That wasn’t the only writing on the wall. We also were just finding that the scope of how we were using it was starting to impact some of the performance on the front-end. So, again, customers benefit from us not having to…well, being able to break things down into smaller parts as the application has gotten bigger from when we launched it.
Joe: Makes sense.
Charles: So, it also seems like there’s a little bit of risk here. I mean, as far as errors go, I guess you know how to track errors? But keeping track of the performance and, all of the other considerations that you have to make, what did you do there to make sure that you weren’t, you know, backsliding, or making mistakes, or that you were getting the results you intended?
Why Application Performance Matters
I’ve kind of been surprised, overall, at how few people actually track the performance for end users. You know, the space, the APM space has been really, really popular with measuring the timings for, “Hey, how long is it taking my server to serve stuff?” But, given these richer frontends, you know, it’s not actually uncommon for us to see customers who will put in the real-user monitoring capabilities and go, “Yeah, my server is spitting back the content in like, say, 30 to 50 milliseconds but I didn’t realize that it was now gonna sit in the user’s browser for 4 seconds while it initialized, made subsequent Ajax calls to fetch some state and whatnot. And, actually, it’s quite a slow application to use.”
Charles: Now, it seems like there are some tools out there that do that kind of evaluation for you. I’m trying to remember. I think there’s a free one.
Joe: There are…
Charles: Like, Website Test or something like that?
John-Daniel: Yeah. So, Website Test will do that, typically, on the load times for fetching the assets. You can also, obviously, just use Chrome DevTools or Firefox dev tools to do it. But what we’ve typically found is larger organizations will want some sort of continuous monitoring, what’s going on, feeding back into their deployment process to give feedback. So, you know, if we’re shipping code several times a day, we definitely wanna be alerted if, say, the website render times are going up.
Joe: I open it up by hand and check it.
Charles: Yeah. You open up the Chrome DevTools and see how long it gets to execute stuff.
Joe: Yeah, there you go. I love that.
Joe: You know, it just seems like it’s always changing, right? Like, today, the focus is on this kind of performance, and tomorrow the focus is on that, you know, and then… and that’s a reaction to what the frameworks are doing and how they’re doing it, right? And then the frameworks get better at something and so then the focus gets away from it.
Joe: And it goes to something else, right, like, what’s the hot topic today? So I feel like there’s not necessarily… like it, you know, today it’s a focus on rendering time. That seems to be a big focus in, you know, first content that the user sees. And that’s because we’ve really come…made the frontends’ frameworks complex and so they’re doing a lot now.
And so, back when we were, you know, doing server-side rendering, the problem was the user’s experience was garbage-y. Now, that’s not the issue. Now, it’s that we give them a whole bunch. You know, so, the performance from the user side was terrible because it would take forever to make a roundtrip back to the server. So now, we’re not doing that so much anymore but instead, we’ve got this issue of what we’re sending. We’re doing so much on the client-side now that it’s taking the user’s computer too much time and so we’ve got to optimize that process in various different ways.
Charles: Well, and then it gets complicated too with all of the stuff involving like progressive web apps, and… you know, it’s like, “Okay, so it performs okay on my laptop that has 64…or my desktop that has 64 gigs of RAM in it, and, you know, 8 Cores, but, you know, on my phone, not so much.
John-Daniel: Yeah, that’s the same as well. It’s the challenge around, “Hey, we’re all, it’s pretty rare that a software developer has a, you know, pretty crappy machine that they’re working on.” If you’re building software, particularly, for what I like to call the “normals,” you know, they might they be on a pretty old machine. And so, understanding that is also challenge.
Charles: Or just a Chromebook. I mean, my son is doing online school and, you know, we bought him a Chromebook. That’s what he spends his time on all day and so it’s pretty resource constrained. And he’s on the Wi-Fi connection downstairs, which generally is fast, but sometimes isn’t so fast. And so even then, you know, it’s not a phone, but it’s still not an ideal machine for loading some of these complicated apps.
John-Daniel: Yeah, I’m kind of curious. I mean, we’re keeping an eye on various things, some of them that we, obviously, won’t need for Raygun, like, you know, some of the WebView apps bits, but I’m kind of curious about things like web assembly. And I know it’s super early days on that for performance but I’m kind of curious when we’re gonna see some of these frameworks that are gonna do some sort of inter-operation with it to try and make, you know, that the actual, constrained parts quicker. And I know that that interrupt layer is somewhat non-existent today but seeing where that’s going… I mean have any of you sort of been looking into that space and how to play with it?
Joe: Not really.
Charles: No, I haven’t, personally.
John-Daniel: Yeah. I kind of think, since people want to use it for games but I’m, you know, getting a slight… usually, it’s so performance focused to build games that it’s nice when you sort of link it up to, you know, not normal frameworks and apps that people are building.
Charles: So, now that you’ve kind of made these changes on this framework and stuff, I’m a little curious too. Do you feel like this has enabled you to build other features or other areas of functionality now into Raygun to make it work better or do more things for your customers?
John-Daniel: Absolutely. I mean, part of the reason with this major piece coming, that we wanted to look at it, was because we were pretty aware that we were sort of running with our shoelaces tied together. You know, just partly because of the fact that it was very difficult to build components and selectively load things. We were sort of having to cart around too much state around the entire app. So it’s definitely speeding that up.
And, you know, there’s things that the team is building now where I would have kind of thought, “Oh, yeah, this will take two or three weeks to kind of get a prototype up.” But then, you know, one of our frontend engineers will be like, you know, “Here’s an example I’ve built and that’s taken me a couple of days.”
In a lot of ways, we’re sort of seeing it more as a task of repaying a whole bunch of technical debt that’s been floating around for a while. And so absolutely moving more quickly is a key benefit.
Joe: I get that.
John-Daniel: You know, it comes back to your point earlier about, “Hey, performance matters. Hey, errors matter.” You know? And at the end of the day, I think it is just the growing maturity of the software industry where we cost a fortune to build stuff, it needs to get better and, you know, for what we’re paying, and part of that comes down to the velocity of the individual engineer and what they can achieve.
Joe: I agree with that. Well, that’s all my questions.
Charles: Yeah, I don’t know if I have any more either.
Aimee: I do not.
John-Daniel: Hey, so this has been the easiest interrogation I’ve ever had. But, yeah… well, I’ll keep you posted hopefully, you know? If anything, we can do another podcast in probably about 2027 where I’ll talk about how we’re shifting off React.
Charles: Yeah. So, do you have any specific advice then if people are using Node.js and they’re finding that it’s just not meeting their needs for making a change like this then, just overall?
John-Daniel: Well, one of the things, I’d say, that’s specific to the Node.js discussion at the start is that it is a framework that’s moving. So, we actually post a blog series every sort of… I wouldn’t say major release, but release of Node.js that comes out. We benchmark the performance across a lot of the web frameworks. And it just seems like, you know, there’s tremendous momentum in the in the Node.js community. And so that is one area where we definitely did not rest on our laurels about moving up in the frameworks. There’s a lot to benefit. Like you say, Async stuff now coming in is fantastic to see. That would be my advice to folks who are working with Node.js, don’t wait too long.