Introducing the OW2 Project Market Readiness Levels

Introduction	Methodology Overview	Best Practices	Project Attributes	Market Capabilities	Market Readiness

Note: to discover MRL in action, you may directly look at OW2's projects page ("MRL" column in projects list).

In this page:

Introduction
Methodology
Defining the OW2 Market Readiness Levels
Collecting and processing project data
References
Request for Comments

Introduction

Open source software is a powerful enabler for collaborative innovation, however, making the source available does not automatically attract contributors or grant immediate market access. OW2 is developing a market readiness program to address what we call the delivery challenge of open source software development. Our aim is to facilitate the adoption of OW2 software by mainstream decision makers, i.e. those that are not necessarily open source software supporters.

Please check out the MRL introduction slide deck. The OW2 Market Readiness Levels (OW2 MRL) approach has also been presented at several events: OW2con'17, Cloud Expo Europe, FOSDEM, FOSS Backstage, OW2con'18, OW2con'19, SFScon 2020.

Since 2015 through the OW2 OSCAR quality program we have collected dozens of data points on code quality, licensing and community that we reported in the OW2 project dashboards, that can be considered as a first version of this methodology. We eventually realized the information contained in the dashboards was dense and complex to read. With the OW2 Market Readiness Levels our objective is to enhance the meaningfulness of this information. We do this by combining different data sources and by integrating them into one composite index: the OW2 Market Readiness Levels (OW2 MRL). Obviously the OW2 Market Readiness Levels are inspired by the well-known Technology Readiness Levels developed and made popular by the NASA.

Methodology

Our approach is both top-down and bottom-up. Top-down because we use our un-scientific business wisdom to pre-define the OW2 Market Readiness Levels and bottom-up because we use objective data collected from the projects to match them with the pre-defined levels. Given this approach we will gradually update the definition of the OW2 Market Readiness Levels, with the help of OW2 project leaders feedback and based on the data collection and analysis we will be able to produce.

Defining the OW2 Market Readiness Levels

The OW2 Market Readiness Levels (OW2 MRL) are a series of business-related situations that reflect the evolution of a project from initial software development to market leadership. The sequence must take into consideration the changes that must occur for a project to transform itself from a purely technology endeavour to a market powerhouse. We define the OW2 MRL by looking at the market hierarchy of open source projects - not a very scientific approach here, only our own modestly expert evaluation. And because we replicate the NASA TRL scale, the OW2 MRL scale also has nine levels emoticon_smile .

The exhibit below is the current status of the pre-defined OW2 MRL. For a more concrete overview, please visit examples of MRL applied to OW2 projects, such as Rocket.Chat or OCS Inventory. To find the MRL score for a given project, please visit OW2's project page.

OW2 MRL	MRL Rationale	Market Rationale	Technology Rationale	(NASA TRL)
9	Market Leader	Significant market share and global customer base. Properly financed and organized business support. Global active community.	Platform, Sub-Project Generation	“Flight Proven”
8	Established Outsider	Customer base of mainstream users. Appropriate financing. Active community support and contributions. Recognized software.	Building Range (Products/Features)	“Flight Qualified”
7	Established Business	Established product. Customer base of early and mainstream users. Stable financing. Open to community support and contributions.	Full Q&A	“Prototype in Space”
6	Sizable Adoption or Market	Proven product. Customer base of early users. Project fit for third party contributions. Implicit community governance.	Growing Interoperability	“Proven Demo”
5	Fair Adoption or Market	Some customers, recent market opening, Core team of developers, untested open source governance.	User Interface Maturing	“Relevant Envt Validation”
4	Usefulness Verified	Several users, project leadership well established.	MVP Defined	“Lab Validation”
3	Fledgeling Usefulness	One declared user (can be internal) with declared project leader.	Golden Scenario	“Proof of Concept”
2	Development	Basic R&D code developed with one demonstrated use case, some documentation.	POC	“Application Formulation”
1	Research	Basic R&D code developed.	Methodology & Algorithms	“Basic Principles”

Collecting and processing project data

This part will evolve from the OSCAR quality program previously implemented by OW2. Here we leverage OW2's participation in three EU-funded collaborative projects: RISCOSS, CROSSMINER and STAMP.

Project information is collected in three stages:

the first stage is a checklist of 50 best practices completed by project leaders,
the second one is a set of 15 data points automatically collected from the project's development environment,
the third stage is an expert assessment of the project market capabilities based on a structured expert interview with the project leader.

The list of best practices also serves as a blueprint to help project leaders implement better management of their projects. The list covers the following ten topic areas: Communication, Community, Documentation, Infrastructure, Management, Development, Testing, Release and Security.
You are welcome to check out this example of the MRL Best Practice form.

Data of different nature are normalised so as to be compared. They are also combined into aggregates and models that help derive meaning from raw data.

Data automatically collected is updated daily and manual data when justified by a significant event and at least two or three times a year.

As a result we have a combined graphic representation of the different scores as illustrated below from our beta version: a bar chart for the Best Practices score, a radar for the Project Attributes and a cursor for the Market Readiness Scale.

References

Countless different works have already been done on maturity and readiness models. As an introduction, here is a short list of useful links. Feel free to recommend more resources.

On the RFC vocabulary: https://tools.ietf.org/html/rfc2119
On comparable evaluation methodologies: https://en.wikipedia.org/wiki/Open-source_software_assessment_methodologies
On maturity models: https://en.wikipedia.org/wiki/Maturity_model
On the NASA TRL: http://www.spacewx.com/Docs/ISO_FDIS_16290_(E)_review.pdf
On TRL in a business context: https://serkanbolat.com/2014/11/03/technology-readiness-level-trl-math-for-innovative-smes/
On composite indicators: http://publications.jrc.ec.europa.eu/repository/bitstream/JRC31435/EUR%2022155%20EN.pdf (see appendix A)

Request for Comments

At OW2 we are convinced this tool must be collaboratively designed, built and maintained by those who will use it, the open source market participants and, in the first place, the leaders and the users of OW2 projects. We are not entirely sure how to organize it but we know this must be a transparent and open process.

Please feel free to comment! We encourage everyone to use the comment feature on this page to provide us with feedback, suggestions, references, advice, etc.

Thank you for your contributions.