Deployment frequency

What is deployment frequency?

Deployment frequency is how often code changes are deployed to production, simply by counting the number of deployments that are made in a particular timeframe, such as on a per-week or per-day basis. These changes can include bug fixes, new versions, enhanced capabilities, and new features. 

Deployment frequency is a “velocity” metric that has been long used in the software development space — it first came into prominence with the emergence of DevOps, and was then included as a key metric in the DevOps Research and Assessment (DORA) team’s framework, which aimed to help DevOps and engineering leaders improve the way they measured software delivery efficiency. DORA also raised the bar on what could be deemed an acceptable deployment frequency. 

Now, with many teams shifting to platform engineering and adopting AI technologies, deployment frequency remains a valuable and important metric across the SDLC. By better understanding how often changes are shipped, the organization can:

• More accurately predict future delivery timelines for new or enhanced features
• Gauge a team’s speed, efficiency, and agility
• Demonstrate how much value teams are delivering to end users

Your deployment frequency will vary, with some teams deploying code monthly, weekly, or even multiple times every day. However, deployment frequency doesn’t always equate to value delivered. Slowdowns in deployment frequency often correlate with issues in developer experience, developer productivity, and developer tools — which means that deployment frequency is often a canary in a coal mine for problems along your delivery pipeline. This concern should be at the front of your mind in the platform engineering space. 

How to measure deployment frequency

First, you need to define what constitutes a “deployment” within your organization. Software deployment incorporates all of the tasks required to make a software system or update available to its users:

• Releasing new code
• Delivering changes via CI/CD pipelines
• Updating applications
• Rolling out bug fixes or security patches

But organizations can take different approaches to measuring this. For instance, you could also count the number of pull requests (PRs) merged into the production branch over a given time frame, giving you the option of equating your PR merge count with deployment frequency, or if PRs are batched into releases, considering each batch as part of the deployment count. 

You can base deployments on the number of workflows run in GitHub Actions or Jenkins in a given month by automating the release process, and count whenever a code merge, tag creation, scheduled run, or other specific event occurs. 

There isn’t a hard or fast rule for calculating deployment frequency, but companies should be measuring and monitoring it. In a large engineering organization, different teams may not have visibility or awareness of deployments being made by other teams.

The ideal scenario is to have one place for everyone across the SDLC to be able to view deployment frequency, so that they can identify both technical and communication issues between teams. Once you've gathered your metrics and set your benchmarks—whether using the DORA benchmarks or your own—the next step is to put this information to practical use. 

‍

What is a ‘good’ deployment frequency?

The DORA team studied thousands of software engineering companies in a six-year program to better understand what metrics engineering teams should use to evaluate themselves, and to get to grips with the differences between high-performing and low-performing teams. 

The latest benchmarks for deployment frequency found in the Accelerate State of DevOps 2024 report are as follows:

• Elite performers: On-demand or multiple deploys per day
• High performers: At least once per week
• Medium performers: Once per week to once a month
• Low performers: Less than once every six months

High performers will make multiple deployments per day or be able to do so ‘on-demand’, while lower performing teams may only deploy once every six months. Deploying more often provides a number of significant benefits for organizations such as:

Speed to market: The most obvious benefit of higher frequency is speed to market; it means users will get new features, updates, and bug fixes faster. In turn, their customers are likely to be more satisfied with the service they’re getting. In turn, this provides the business a competitive advantage. 

‍Reduced risk: Smaller, incremental changes mean less risk because you introduce less complexity with each deployment. It also means that subsequent issues are easier to detect, fix, or rollback.

‍Encourages best practices: Engineering organizations that want to adhere to best practices for DevOps, agile, platform engineering and CI/CD will know that deploying more frequently is a move towards more maturity and automation. This makes the organization a better place to work for engineers, improving developer experience.

‍Improves developer productivity and morale: The reduction of bottlenecks, increased transparency and accountability over code, and a faster code-to-production path are all ways to improve developer productivity and morale. The knock-on effect is better employee retention; if you keep your best engineers, you’re going to be more successful as an organization.

It’s worth noting that each organization will have unique benchmarks based on:

• Their own operational goals
• The complexity of projects they work on
• The maturity of their engineering teams

You and your teams should work to develop a baseline in the context of your own software development pipeline that represents your current situation, and continually compare progress against that benchmark, as well as industry standards, to judge success. Engineering teams can implement and track DORA metrics using an internal developer portal.

Why deployment frequency is important

While deployment frequency is a rather easy metric to measure, it can provide teams with a lot of power.

Deployment frequency can be a metric you use to evaluate your team’s agility and responsiveness to feedback. But perhaps most importantly, deployment frequency can indicate whether there are issues with existing systems, processes, or even personnel. 

It provides a basis for engineering leaders to investigate further, and try to either fix issues or ensure that certain methods or systems are consistently used and applied because they provide good results. Once adopted as a metric, deployment frequency can serve as a gauge for the overall impact on or effectiveness of any changes made to the SDLC. 

In contrast, it can indicate when the environment is highly functional. More frequent deployments can also ensure the principles of continuous delivery are carried out, keeping your engineering team on track to deliver often and ultimately provide end users with the features, fixes, and updates that they need in a timely fashion. 

Real world examples of deployment frequency improvements

Below are two case studies illustrating significant improvements in deployment frequency:​

1. Xceptor improves deployment frequency by 24x

Xceptor, a data automation provider, faced challenges with siloed operations and limited visibility into software ownership and dependencies. These issues hindered its deployment processes, resulting in infrequent releases. By implementing Port's internal developer portal, Xceptor achieved:

• Enhanced developer autonomy: Developers gained the ability to manage and deploy their code more independently, reducing bottlenecks.​
  
  
• Efficient deployment processes: The abstraction of complex Kubernetes operations allowed for streamlined deployments.​
  
  
• Significant increase in deployment frequency: The average customer deployment frequency surged from once per year to 25 times per year—a 25-fold increase.
  

These improvements not only accelerated their deployment cycles but also reduced costs and improved overall developer efficiency. ​Read the full case study here.

2. Space International increases deployment frequency by 9x

Space International, a software development company, sought to improve its engineering metrics, particularly deployment frequency. By leveraging Port’s internal developer portal, they were able to:​

• Centralize information: A unified software catalog provided clear visibility into all projects and their statuses.​
  
• Implement scorecards: These tools allowed teams to define goals, set standards, and track progress effectively.​
  
• Introduce self-service actions: Developers could perform routine tasks without external dependencies, speeding up processes.​

As a result, Space International achieved a more efficient deployment pipeline, leading to increased deployment frequency and improved overall software delivery performance.

These case studies demonstrate that with the right tools and strategies, organizations can substantially improve their deployment frequency, leading to faster innovation and better responsiveness to user needs. Read the full case study here. 

{{cta-demo-baner}}

AI and deployment frequency

AI tools have the potential to make engineers more productive, but it’s not a guarantee; in fact, GenAI tools can also cause increased chaos leading to more vulnerabilities, bugs, and incidents. It’s for this reason that engineering leaders who have decided to adopt GenAI tools need to measure their true impact.

One of DORA’s two impact metrics is throughput, which refers to the time it takes committed code to make it to production, and is typically measured with deployment frequency and lead time for changes. Together, these metrics can give you insight into whether code makes it to production faster with AI, and how many changes your deployment pipeline can handle. An effective GenAI implementation should mean throughput increases across the board.

However, measuring GenAI effectiveness can be difficult because of the different types of metrics to track, and the various tools being used in the software development lifecycle. An internal developer portal can help organizations to measure throughput and stability, while also fostering an improved developer experience.

This is because portals unify every tool in the SDLC and bring together insights from across tools from code repos and CI/CD tools — which makes measuring deployment frequency easier and produces more accurate results. This helps you answer questions such as “has throughput really improved since rolling out AI tools?” and "how has our deployment frequency changed?” 

Learn more about how you can measure GenAI impact in an internal developer portal here.

{{cta_1}}

Deployment frequency pitfalls

There are some instances where you may see high deploy counts that don’t actually reflect a highly functional team or environment, but are the result of other inefficiencies or adoption antipatterns, such as:

• Gaming the system
  Developers who believe they’re being measured by their deployment frequency will feel the pressure to perform to what they perceive as leadership’s expectations, and may try to “game the system.” That means they may more frequently push code without proper checks, tests, or documentation.
• Inconsistent processes
  Inconsistencies among siloed teams who use different practices to break up their code may result in some teams shipping an unusually large number of small changes, or one team shipping many changes in a single file. Both approaches can cause issues with your change failure rate.
• Complexity dictates frequency
  Complex projects may require many bug fixes and deployments, but this doesn’t necessarily mean that code quality is high. The most important takeaway from implementing a deployment frequency metric in your organization is to understand that a higher deployment frequency does not always correlate with a well-functioning team.

Wrapping up

Deployment frequency is an important metric that many engineering teams currently use to help them measure developer productivity. But it is just one of four DORA metrics, and engineering teams ought to use at least all four to properly determine developer productivity and developer experience. 

Depending on the organization, there may be other objectives that the team wants to accomplish such as better standards compliance, tighter security, standardization, and meeting SLAs/SLOs. These objectives would require measuring and monitoring other types of metrics. Read more about other, related metrics in this post about engineering KPIs.