For Agile on the Beach 2016 we had over 240 submissions. Thank you to everyone who submitted, I’m sorry we could have so few of you — 41 sessions if I remember correctly.
I find sending the “Sorry you weren’t accepted” emails the hardest part of organizing Agile on the Beach. That is especially true when I know the speaker personally, or I want the topic, or I’ve seen the talk before and think it would be great. But in the end I only have the same voting power as the other five committee members.
And in the end, knowing that the decision on who to accept, and who not to, was not mine alone helps me get through the “Sorries.”
Each year I tell potential speakers: “If you would like feedback on why your session was not accepted please email me, I can’t promise detailed feedback, or promise to do it promptly, but I’ll do my best.”
After All Agile is About Feedback, So Why Wouldn’t We Do This?
Well, its a lot of work — about five hours I think. Because it's a lot of work this year it has taken me nearly three months to find the time to do it, but I have just sent the last piece of feedback. Sometimes there isn’t much I can tell potential speakers. Reviewers can add comments with their score but they don’t have to.
(If you submitted something to Agile on the Beach 2016 and I’ve missed your request, or you only now decided you would like feedback then please email me.)
Many of the reasons why sessions didn’t score highly crop up again and again so I thought it would be worth capturing them here in a blog. For anyone thinking of submitting to Agile on the Beach 2017, or another conference, these reasons might be useful.
By the way, I have another blog post about the Agile on the Beach voting process which should be read together with this one, or better still, before this one!
So, the common themes in the feedback I’m giving submitters are:
- A lot of sessions were simply out-competed: reviewers liked them but they liked other sessions more.
- Some synopses were too short, light on detail, or did little to engage the audience — and certainly the reviewer.
- Some synopses were too long and had too much detail. Indeed I remember some that went on and on. Clearly there is an art to getting something long enough to include good details, but not so long the reader gets fed up reading it. (Remember, reviewers may have 240 of these to read.)
- Multiple submissions by one person is a high-risk strategy. If you are Adrian Howard or Seb Rose this is a high reward strategy, both of these speakers have in the past submitted strong proposals and the committee is left agonizing about which one, or perhaps two, of the submissions we will have. Seb and Adrian changed the debate. But for most submitters, reviewers notice multiple submissions and actively score one higher than the other. Unfortunately if another reviewer makes the opposite choice, multiple submissions effectively split the speaker's vote. So it is better to limit your number of submissions and play to your strengths.
- Multiple submissions of the same synopsis with different travel expense expectations or co-speakers really annoy reviewers. For example, a couple of people submitted a talk with expenses set to “Worldwide” i.e. a long haul flight, and then submitted the same with “Local” or “Will pay my own long haul.” Reviewers felt these people were trying to game the system and marked them down.
- Indeed, Worldwide travel or long-haul flights always present us with a problem, because such speakers are more expensive for us. We can usually find the money for one or two such speakers but that's the limit. This year we provided the option for speakers who were prepared to pay their own long-haul flights, and we have a couple of speakers doing this. While I accept this might not be fair, it is hard to see how we can be completely fair given our budget.
- Paid speakers: We don’t pay speakers for Agile on the Beach. We pay travel expenses but not speaking fees. If you are looking for a fee then please don’t submit.
- Keynotes: We don’t do an open call for keynotes. We usually select the keynotes and invite them before we even open the call for papers.
- Double sessions need to be twice as good. Accepting a double displaces two single sessions, which means we need to be convinced. Despite making extra space for doubles, this feels like an inevitable problem because it is about timing. It isn’t really fair; some topics genuinely deserve more time, especially if they are interactive. Perhaps the best advice is to only submit a double if you have been to the conference before and committee members are likely to remember you as a good speaker. Giving a double to an unknown speaker is a big risk.
- This year we had what seemed like a lot of submissions along the lines of “What Hiking taught me about Agile” or “Why paragliding is like software development.” While these are interesting metaphors, none of them scored highly. In general I think such metaphors only work if the field is well known. And this year, I for one got a bit fed up of reading yet another “Lessons from arctic exploration for Agilistas.”
- ThoughtWorks: This is one of those “nice to have” problems but still a problem. We have a great relationship with ThoughtWorks — largely thanks to Jim Barritt and James Lewis who put us on the TW map and encouraged a lot of TW people to attend. So we get a lot of strong proposals from Thought-Workers, after all TW only hire the best so we don’t get weak proposals from them. In 2015 we could have populated entire tracks with only TW consultants. They would be strong tracks but we don’t think that is good for conference variety. As a result, submissions from TW consultants have to compete against each other in addition to competing with everyone else. Sorry guys.
By the way, we don’t do anonymous submissions, we, certainly myself, tend to believe the speaker, their background, name, and experience is important. If we have been fortunate (or unfortunate) enough to see someone speak at a conference we take that into account. We also know there are many speakers we'd like to see return — and one or two who we don’t want to see again!
We have in the past been criticized for not including enough variety in our speakers: specifically: having too many men. Perhaps one reason for doing anonymous submissions would be to guard against this, but actually, we don’t see this as our problem. Or rather: yes we would like a better gender balance, but when we’ve looked at the male/female ratio of other comparable conferences we’ve found we our balance is better than most. That is not a reason to rest on our laurels, but it does imply it's not a problem to us specifically.