Eclipse At eBay

Eclipse is known to be at the core of a lot of large enterprises, and eBay is no exception. Following his talk on Eclipse @ eBay, I caught up with Michael Galpin to discuss a little bit of what goes on behind the scenes at eBay and how Eclipse plays a key part of their development landscape.

DZone: Hi Michael. Could you introduce yourself, and your role in eBay.

Michael Galpin: My name is Michael Galpin, and I am an application architect at eBay. I have been at eBay for 2.5 years, and have been working in software for more than 10 years. I studied mathematics in college, but always dabbled in programming. I moved to the Bay Area in the dot com boom and took a job at a precocious startup. I was employee #3, and I asked them what they needed me to do. They said program, and I never looked back.

DZone: Could you describe the evolution of eBay's technology stack. How did you get to Java being the core of your architecture?

Michael Galpin: Well, eBay has obviously been around for a long time, so there has been a lot of evolution. When Pierre first coded AuctionWeb (the original name) over Labor Day weekend, he did it in Perl and didn't even use a database. The Perl stuck for a long time -- but with a real database of course. Eventually the site was re-architected and (mostly) re-written in C++ with an XSLT front end. Basically every request you would make to eBay would cause a C++ command to get executed, and that would produce an XML data model for a page. The appropriate XSLT transformation would be performed and HTML would be produced and sent to the browser. We had a single master/slave database setup back then, and obviously that quickly become the bottleneck of the system.

The second major re-architecture was to allow for our databases to be split ad infinitum and to put in a lot more caching layers. We needed a lot of long running processes to manage the caches and manage many-to-many mappings between logical databases and the real physical machines. Java was picked as the best way to do this, and what became known as the V3 architecture was born. Java really made it possible for us to achieve true horizontal scalability. However, we did not change the front end at all. Instead, Java commands now created those same XML data models. This worked for a long time, until the front end of our system became more and more complex. Having a site available in 30+ languages really fanned the flames as well. Anybody who thinks you can localize a site using property files or resource bundles has never serious localization. Anyways, all things combined together and we looked at a lot of different solutions for changing our front end. Name a popular framework -- including non-Java ones -- that was around in 2006, and I can go and find you some analysis we did on it. However, nearly all of the frameworks still relied on templating systems, and we really wanted to get away from that. We wanted to get the type safety of Java throughout our front end. Thus the V4 architecture was born, and it uses Java from end-to-end.
 
DZone: What does Eclipse give to you? Could another IDE have provided this in the past or is there an IDE that can compete well against Eclipse now?

Michael Galpin: Doing everything in Java had its cost. Java is a very verbose language, so there could be a lot more code that our developers had to write. To balance this out, we built developers tools so that our developers did not have to write lots of boilerplate code. Many of those tools were standalone tools -- they had to be, so that we could use them in or build system as well -- but it made it more convenient for our developers to make the tools into Eclipse plugins. We were already using Eclipse before V4, and its open system made it a perfect and obvious choice. Today we have dozens of Eclipse plugins that we develop internally. It would have certainly been possible to build many of the same tools as NetBeans plugins. However, even a company like eBay has only so many resources.

We really wanted to standardize our development environment. Sometimes less choices is better. By standardizing our development environment, a new developer can get up and running very quickly. They don't have to figure out what plugins to install and how to configure anything. They install what we call the EDE -- the eBay Development Environment, which is Eclipse plus a selected group of plugins (some our own, but many are also popular open source ones.) They check out the code that they need to work on, and with one click get any extra plugins that they need based on what they are working on, and that's it. We don't waste time with versions, configuration, setup, etc.

It would certainly be possible to do everything we're doing in a different IDE today. NetBeans has really come a long way since we decided to standardize on Eclipse. Like everyone else in the Java community, the future of NetBeans under Oracle concerns us. Another thing that NetBeans has really benefitted from are the improvements in Swing and the JRE. There used to be a major performance/usability advantage in using Eclipse because of SWT, but not anymore. Similarly, IDEA has benefitted from these improvements as well. With the recent open sourcing of IDEA, I could definitely imagine a similar IDEA based development environment. However, most of the web development features of IDEA are not open source, and that would pose a huge problem for a web company like eBay.
 
DZone: Could you give a background into some of the tooling that you use in Eclipse?

Michael Galpin: We use a lot of popular open source plugins, like the WTP and FindBugs, but we build a lot of in-house plugins that tie in closely to our architecture. Those are not available to the public. However, we would really like to make them available. At JavaOne this year, we announced a trial offering of our VJET plugins -- which are plugins for working with JavaScript. We were basically trying to assess public interest in using our technology. It takes a lot of effort to support a major open source project -- public source code repositories, bug tracking systems, managing patches, establishing committers, and figuring out how to roll all of this back in to our core technology that our business relies on.

DZone: How does Eclipse help in managing code complexity?

Michael Galpin: We take full advantage of all of the usual benefits of Eclipse: code completion/navigation, refactoring, etc. This is magnified by the fact that we use Java for everything. Ever had a piece of UI code that gets repeated in lots of places? For us, that is one "extract method" refactoring by Eclipse away from being simplified. We also use static analysis tools available for Eclipse, like PMD and FindBugs, to help battle code complexity. All of that being said, no IDE is a suitable substitute for good programming.

DZone: If I took Eclipse away from you would it disrupt how you develop code, or could you continue well with another IDE?

Michael Galpin: Since we have built our own ecosystem around Eclipse, it would certainly be quite disruptive. You don't build up a system like this overnight. At the very least, I think we would lose a lot of time around setting up projects/classpaths, build system, deployments, unit testing, etc. We have been doing a lot of work with Maven as well, and that can help a lot. Beyond the boilerplate stuff, other IDEs are certainly viable. We don't rely on any configuration heavy frameworks, we're just heavy on Java.

DZone: From reading your recent post it seems that you don't really like Eclipse. Is that a fair statement to make? I'd be interested to hear what you think about Eclipse as an IDE.

Michael Galpin: No, I have a lot of love for Eclipse, I really do! If you use something a lot, you become all too aware of its shortcomings while taking for granted its qualities, and this is true for Eclipse. To give an example, I'd love to see the JDT overhauled to be much more extensible for other languages. I know we've had issues tapping into it for VJET, and I've heard from Miles Sabin about all of the issues that he's faced in working on the Scala plugin for Eclipse. Often you have to resort to use AspectJ to monkey patch the JDT. It shouldn't be like that. I really hope that the open sourcing of IDEA will increase competition and spur innovation in all IDEs, especially Eclipse.

Here are the slides from Michael's talk on Eclipse @eBay: