Free Fridays: Building a Neural Network Query Optimizer
Free Fridays: Building a Neural Network Query Optimizer
Ever heard of ''Free Fridays?'' Let's find out what one person from Cockroach Labs does on her Free Fridays and if she finds them beneficial.
Join the DZone community and get the full member experience.Join For Free
Databases are better when they can run themselves. CockroachDB is a SQL database that automates scaling and recovery. Check it out here.
From the beginning, Free Fridays have been a part of the Cockroach Labs culture. It's a day where Roachers can make their own schedules: Should I come into the office or WFH? Should I send out these PRs or focus on my side project? Or should I just close my laptop and go on a weekend getaway? Quite frankly, it's up to them. They strike a chord with many as it's easy to wonder "Can a 4-Day Work Week Work?" Now, three and half years later, Free Fridays are still going strong and are a benefit that we can brag about.
Over the next few weeks, we'll be sharing how some of our employees spend their Free Fridays. I'm excited for you to see what my colleagues are working on in and out of the office. While every Free Friday can be different, I hope this highlights the range of ways our Roachers spend this time and the value that it has within our company culture.
Software EngineerWhat is your role at Cockroach Labs?
- I am a software engineer and am currently a member of the SQL Query Optimizer team, which is building a state-of-the-art cost-based query optimizer. Since the query optimizer is a new feature that has not yet released, we have the luxury of designing and building it from the ground up without worrying about production issues or backwards compatibility. In order to ensure the optimizer will have correct behavior and improve CockroachDB's performance, we are constantly learning about the subtleties of SQL scoping and semantics, reading papers about logically equivalent query transformations, and researching how to use statistics to correctly estimate the cost of query execution.
- As a member of the larger engineering team, I am also part of the on-call customer support rotation, so around every six months, I spend a week triaging production issues and answering questions from the community. All engineers, including myself, are also responsible for occasionally conducting technical interviews in order to share the load of Cockroach Labs' hiring process.
- Most of my time is spent writing code and reviewing the code written by other engineers on the SQL Query Optimizer team. I have very few meetings each week; I usually spend no more than two to three hours per week in meetings. When I’m not working on the query optimizer or in a meeting, I may be catching up on emails or slack messages or taking advantage of the many opportunities for learning and development at Cockroach Labs such as Lunch and Learns and career development workshops.
- As a recent Ph.D. graduate, I often use Free Fridays to maintain my connection to the academic world by reviewing papers, going to talks, or working on my own papers. Free Friday is also a day to catch up on the items on my to-do list that I didn’t have time to do earlier in the week. It’s a day to sleep in, meet friends for lunch, and catch up on personal errands such as visits to the dentist or laundry.
- My partner, Raul, is a postdoc at MIT and also studies database management systems (we met when I was at MIT for grad school). We are working on a project together to build a query optimizer based on neural networks or “deep learning.” Since traditional query optimizers (including the one we are building at Cockroach Labs) rely on many heuristics as well as statistics that are often stale and out of date, it is possible that a neural network could do a better job of choosing a query plan. It is too early to tell if our idea will work, but if it does, it would be a game-changing addition to Cockroach Labs’ query optimizer and a major boost for Raul’s academic career.
- Although I would like to spend most Fridays working on the neural network query optimizer, in practice, other tasks tend to take precedence. I am a member of the program committee for the three major academic database conferences this year (VLDB, SIGMOD, and ICDE), so I often need to spend my Free Fridays reviewing papers. Reviewing a single paper usually takes at least 3-4 hours (sometimes 6-8 if I’m not familiar with the topic), so often, that is all I have time for. I also spent several Free Fridays last fall working to get my own paper published with the last bit of work from my Ph.D. thesis. Finally, I often spend at least some time on Fridays doing personal errands, answering personal emails, or doing my normal day-to-day work of writing and reviewing code for the (non-neural-network) query optimizer.
- On Fridays, I typically sleep in until 8 or 9 and have a leisurely breakfast at home. I might start a load of laundry and then answer emails and catch up on news for an hour-or-so. By 10, I’m ready to start working, but my task for the day is different almost every Friday. This Friday, I am working on the blog entry you are now reading, but on any typical Friday, I could be reviewing a paper or doing any of the other academic, personal, and professional projects mentioned above. Sometimes, I meet a friend for lunch or coffee in the afternoon and return home to work for a few hours until around 6 pm.
- Although I have chosen a career in industry rather than academia, I hope that I can continue to stay connected to the academic community. By using Free Fridays to review papers, I am staying informed about the latest ideas in the research community and helping ensure that papers published in the major database conferences are relevant to the industry. As time goes on, I hope to spend more Free Fridays pursuing my own research as well, perhaps in collaboration with other members of the engineering team. I’d eventually like to help the team write a paper about the existing Cockroach database since I think there are many aspects of the system that the research community would find interesting. Publishing in academic conferences such as VLDB and SIGMOD will give us additional credibility in the research community and help attract top talent to the company.
Published at DZone with permission of Chelsea Lee , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.