Estimation is not a new concept nor is it a bad word.
Estimation is not a new topic for anyone in the Drupal or open source community. We do it every day at our jobs. We even discuss estimation techniques at our conferences. We provide our clients with estimates when building and contributing back to open source project, yet we don't include estimations within our open source community and issue queues.
We all provide estimates to our clients - are we afraid to do it when it comes to Drupal and Open Source?
Before we take on this tough question, 'Are we afraid to estimate our work in Drupal and open source?', let's start off with a straightforward question: 'Why do we provide estimates to our clients?' The answer is just that our clients want to know how much something is going to cost and we want to know how much work is required to complete a project.
To give this discussion more context, let's begin with a very general definition of estimation
Estimation (or estimating) is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable.
Here’s my hypothesis:
The lack of estimation in open source is hurting the onboarding of new contributors and the ongoing sustainability of our projects.
The Science of guessing - Drupal estimation techniques from project managers
While researching estimation within the Drupal community, I found a bunch of great presentations about project management and estimation. To me, "The Science of guessing - Drupal estimation techniques from project managers" by Shannon Vettes (svettes), Jakob Persson (solipsist), and Mattias Axelsson (acke), was the most comprehensive and inspiring presentation. Feel free to watch this presentation. I am going to pull a few slides from this presentation to help move through this exploration.
Every presentation I watched focused on estimation concerning managing expectations for client/employer based projects. Yet, no one addressed estimation within our open source projects.
Open source projects are afraid to estimate because everyone is working for free
I know the above statement is a broad generalization but until only a few years ago one of most popular credos in the Drupal community came about when it came to the release of next version Drupal. It was/is…
It will be ready when it is ready.
I am glad to say this policy is no longer applicable to Drupal core because the community has committed to regular six month releases. Still, I feel we struggle to commit to specific deadlines. Yes, we have rough timelines for when core strategic initiatives might be completed, but the truth is these timelines are guesstimates because as a community we don't include any estimations within our issues queue or process. We don't have any accurate metrics tracking productivity, velocity or return on investment.
For example, a few years ago our community came together to create the Drupal 8 Accelerate Fund and it is worth asking some general questions about that collaboration.
Was the Drupal 8 Accelerate Fund successful? What was the return on investment? How much work was completed for 2,500 man hours at $100 per hour?
A general question for the Open Source community is...
If an open source project began to collaboratively and publically estimate tasks, would we be able to give accurate timelines and roadmaps?
Maybe we don’t know where to begin but we have to begin somewhere. To get answers, we have to ask the questions.
I want to emphasize that I am criticizing our process and not any individual, team, or organization. I am asking a tough question because I want us to solve the problem of sustainability. Building and sustaining an open source project has a cost and we need to know what that cost is.
How has not providing estimates impacted our open source projects?
Personally, I have seen how the lack of estimation has impacted the Webform module's issue queue. I have seen incredibly challenging tickets that lacked a clear estimate of the amount of work to be taken on by someone, only to see them become frustrated because they are unable to complete the task. I have patches in the issue queue for tasks that are so complex I don't have time to review the code. My failure to estimate specific tickets has led to another common problem that estimation helps to address, which is breaking down complex tasks into smaller more manageable steps.
If the Drupal community wants to get big and small businesses involved, we need to be able to provide realistic estimates. We need to have real and practical examples - to be able to say based on our community's history and these statistics, if you give X dollars, we can complete this initiative or task which will help benefit your business.
We don't need to be afraid to ask people to estimate as long as we all acknowledge that everyone's level of commitment is going to be different. Open Source projects generally employ no one. We would need to collectively discover the best estimation technique while factoring in a realistic level of commitment that a developer or business can make to completing an open source task. And yes, we also need to track how funding impacts an open source project.
How would estimation benefit open source projects?
The below chart from 'The Science of Guessing - Drupal estimation techniques from project managers' shows how estimating the value, effort, and risk helps everyone understand what work needs to be done and what probably does not need to be done
For me, the sweet spot, low-hanging fruit are easy to complete tasks, and an essential quadrant for onboarding new contributors and keeping existing contributors engaged, especially if they only have a few minutes available to contribute. Personally, I always start out my work week going after a few easy tasks before taking on the more challenging tickets which bring a lot of value to the Webform module.
Some great examples of low-hanging fruit are documentation, patch reviews, minor user interface tweaks, reviewing and improving test coverage. Realistically we need to break down complex tasks into simpler easier to complete tasks and user stories.
Having estimation in our open source issue queues will make it easier for people, organizations, and companies to get involved.
Businesses and organizations need to know how to plan, budget, and communicate what it costs to contribute to open source
I persuaded my main client Memorial Sloan Kettering to become one of the largest earliest adopters of Drupal 8. The Memorial Sloan Kettering website was built while Drupal 8 was still in alpha releases and launched on a beta release. Memorial Sloan Kettering did contribute code back to the Drupal community but it was difficult to plan and budget for funding any module upgrades from D7 or D8 because project maintainers tend not estimate how much is required to upgrade a module. We worked with guesstimates and managed to upgrade a few modules.
Imagine if we were able to provide reasonable estimates, based on historical information combined with a collaborative methodology, to organizations. Imagine being able to say to company A if you allocate X dollars to this specific open source initiative, project, or task, we can accomplish B features.
So let me put my dream into a real-world context; every organization in the EU needs to comply with the General Data Protection Regulation (GDPR) and make sure that they are protecting their users' data while making this data available and portable to each user, and ultimately deletable. This is a very broad generalization of the GDPR but to comply with these regulations, there are going to have to be several improvements made to the Webform module. These types of features could be broken down into individual tickets with collaborative estimates so that organizations can allocate resources and/or funds to help themselves and their community comply with the GDPR. BTW, Drupal core is facing the same exact challenges with GDPR.
How could we collaboratively estimate in our open source projects?
There are many different techniques for estimating. The only concrete requirement that I feel is a must is that we use collaborative estimation technique.
Personally, I like the concept of planning poker which is a form of the delphi method.
Planning poker, also called Scrum poker, is a consensus-based, gamified technique for estimating, mostly used to estimate effort or relative size of development goals in software development.
I also like using story points to scope out the overall level of effort required for a ticket.
A story point is an abstract measure of effort required to implement a user story. In simple terms, it is a number that tells the team about the difficulty level of the story. Difficulty could be related to complexities, risks, and efforts involved.
-- http://www.tothenew.com/blog/how-to-estimate-story-points-in-agile/
Even being able to categorize and label tickets as easy, medium, and hard could make a big difference. Currently, in our issue queues on Drupal.org we can apply these types of tags to tickets, but users have to use the advanced search to filter by any tag. It’s not reasonable to expect a new contributor looking for an easy ticket to use an advanced search.
We also need to collaboratively calculate the value of our tickets
So imagine if you have been using Drupal for a few years and finally wanted to contribute something back to the community, and started looking for something to do. You find a task that several people have marked as easy to do and also provides a lot of value. Even if you have questions or run into a problem with the task, you might find a mentor in one of the people who helped estimate that ticket.
What challenges should we begin to collaboratively estimate?
Every GDPR task is going to provide a lot of value to organizations in the EU. Personally, I feel it is also important that we start recognizing the inherent and ongoing value in improving accessibility in our software.
Frankly, there are probably feature requests in the Webform issue queue that I do not see the value in and people are not able to collaboratively communicate the need and importance for these features. For instance, I am not actively doing any headless Drupal work; I am not sure of the value of improving the Webform module's REST API's. I guess that specific organizations see a lot of value in using the Webform UI/UX to build form's which can be served via a REST or JSON API.
What is my key point?
We need to stop guessing the value and work required to complete a task. We should collaboratively determine the value of a task and estimate the amount of work necessary to complete the task. Understanding the value, effort and cost required to get things done will make it easier and more rewarding for individuals and organizations to get involved and stay involved in open source communities.
I knew I had to include the above slide for the presentation in this post because it portrays the challenge of getting people, especially developers to provide reasonable estimations. Developers tend to suck at estimating which is why the role the project manager exists. Project managers are the keepers of the process. My last blog titled 'Our journeys within our community,' I concluded with the realization that project managers are one of the key roles in our open source community. When it comes to addressing the challenge of estimation within our open source project they are the best qualified to help define and discover the best technique to estimate our work collaboratively
Good estimation happens when there is a good discussion followed by some agreement and action.
Conclusion
Since this blog post began with the hypothesis that, "the lack of estimation in open source is hurting the onboarding new contributors and ongoing sustainability of our project", it must end with a conclusion.
It's hard to conclude how the lack of estimation is hurting our projects because we don't have a concrete metrics to work with, except the fact that everyone is relying on and contributing to open source.
To improve open source sustainability we need to examine our process and how we collaborate to create open source software. Estimation is a key part of the process of building software.
We need to determine the best recipe to sustain and grow open source. My most recent blog posts have circled around the problem and challenges of sustainability. I feel the below formula/recipe could help us solve the problem of sustainability.
Project Management + Estimation + Mentorship = Sustainability
Examining this formula will be the topic for my next blog post.