By Stephanie Sealy

How to Measure CMS ACO Programs

The Affordable Care Act established several programs administered through the Centers for Medicare and Medicaid Services (CMS), the aims of which were to incentivize and partner with providers to proactively coordinate patient care, maintain and improve the quality of healthcare delivered to beneficiaries, and reduce overall cost in providing such care. Several of these programs followed the “Accountable Care Organization” model, a concept that implies a change in incentives from purely fee-for- service delivery, to an approach where providers are incentivize to take financial risk in the limiting total cost of care.

Starting in 2012, CMS launched several variations of the ACO model, most of which remain active, and still more variations are planned for 2018. For example, the “Advance Payment” and “Pioneer ACO” models were launched in 2012, and concluded as planned in 2015 and 2016. Later programs, such as the “ACO Investment Model” and the “Next Generation ACO Model”, were built upon the learnings from their predecessors, and CMS continues to add new programs based on past learnings, with the goal of getting more providers accustomed to performance-based financial risk.

Current and Planned CMS ACO Programs

In 2017, there are 562 CMS-sponsored ACO’s across Shared Savings Tracks 1, 2 and 3, Next Generation and CEC, with the majority of those ACO’s in the MSSP Track 1 program. This represents roughly 12 million beneficiaries, and nearly 360,000 clinicians. In 2018, CMS is adding the Medicare ACO Track 1+ model as an encouragement for more ACOs to move out of Track 1 and toward taking more risk.

What these models have in common, despite having different levels of risk, different targeted geographies, patient characteristics or other attributes, is the simple notion of a financial benchmark used to establish bonus payments (or losses), and standards of quality that must be met to receive incentive payments. No matter how an ACO is organized, or its level of resources, all are judged by these “fundamentals” in the eyes of CMS programs. Performance against these fundamentals is publicly disclosed on a regular basis.

Current and Planned CMS ACO Programs

CMS has established an ostensibly simple financial benchmark that each ACO is measured against. All versions of the benchmark have the concept of “attributed beneficiaries”, or more simply, the population of patients for whom cost of care is measured. Each also measures the cost for a patient across multiple years, and then risk-adjusts these costs based on the health of each patient. Finally, there are “included” costs consisting of Part A and Part B claims, and “excluded” costs such as prescription drugs (Part D).

A beneficiary can be attributed to an ACO a number of different ways, depending on the type of ACO. The most prevalent approach to attribution is the use of claims history to assess who a patient’s primary doctor is, such that this provider can be measured against his or her ability to “quarterback” care journeys and cost. The attribution logic has more fine print to it, and the fine print does change, but the purpose of attribution is to assign responsibility where it makes most sense, in such a way that the assignment aligns with the ultimate aims of care coordination and value-based services.

The most commonly used cost benchmark is set by taking a three-year look-back at claims data for attributed beneficiaries:

  • Performance Year Minus 3 Years (2014, for example): 10% weight
  • Performance Year Minus 2 Years (2015): 30% weight
  • Performance Year Minus 1 Year (2016): 60% weight
  • Updated annually based on current ACO participants
  • Annual risk adjustment based on HCC (risk score)

These benchmarks are typically reported at the Per-Member- Per-Year (PMPY) level. An ACO’s ability to manage costs below their benchmark for attributed patients drives their likelihood of receiving incentive payments, and depending on the type of ACO, this might involve minimum savings or loss rates, upside risk, or downside risk in varying degrees. The largest ACO program by nearly any measure, the MSSP Track 1 ACO model, carries no downside risk and some upside opportunity if minimum savings are achieved.

The national average for MSSP Track 1 ACO’s is roughly $10k PMPY in total costs, and ACO management is targeted at maintaining quality while driving actual costs below the ACO’s specific benchmark. Some of the largest individual costs are short-term hospital stays, evaluation and management (E&M) visits, and in-office procedures. However, much of the purpose of an E&M visit is to quarterback the patient into healthier situations that don’t require a hospital stay. After 3 years, an ACO in the Track 1 program moves to a regional benchmark.

Confounding the effort to track costs, to a degree, is the fact that 3-5% of patients opt out of sharing data with their doctor, making their out-of- network costs (and some in-network costs) invisible to the ACO, but still a factor in the calculation. Other “invisible” numbers include the amount spent on mental health services or substance abuse treatment, reported only in the aggregate when CMS releases results. These 2 data challenges can introduce plus or minus 2- 3% uncertainty into actual costs, in a program where most ACO’s produce plus or minus 3% savings. Certainly, this is a challenge from a predictive standpoint.

Finally, the method in which the benchmark is set, of course, has its pros and cons. One weakness is that an ACO might be held responsible for up to 3 years of a patient’s cost even if that beneficiary only recently became attributed. In other words, in this case the ACO is inheriting “unmanaged” spend. The same 3-year lookback can also blunt improvements the ACO makes, such that full benefit takes multiple years to accrue. Any calculation or approach is bound to have limitations.

Quality Metrics and Reporting

A notional, but unspoken, concept in CMS cost-saving programs is the concern that attempts to lower costs in healthcare might result in poorer quality of care for beneficiaries, as providers attempt to “cut corners”. Fortunately, CMS established a method for holding providers accountable for maintaining levels of quality, reporting specifically-designed quality metrics to the public, and in most cases using the attainment of quality goals as a gate to receiving incentive payments.

While they have been changed a few times, the current set of quality measures represent a cross-section of objective and subjective measures, collected a few different ways. Of the 31 current quality metrics, 8 measures reflect patient satisfaction and are collected via opinion surveys sent to beneficiaries, 7 measures are derived directly from claims data and calculated by CMS, and the remaining 15 must be calculated by the ACO itself and submitted to CMS via approved systems. The final metric requires attestation that the ACO is making meaningful use of EMR technology.

The 23 objective metrics established by CMS are broken into 3 categories:

  • Care Coordination & Patient Safety (10 metrics) is intended to measure how well the ACO is “quarterbacking” the patient journey, and specifically when hospital stays are involved. Metrics include factors of admission and readmission, and effective practices for transitions of care post-discharge.
  • Preventative Health (8 metrics) is targeted at keeping beneficiaries healthy enough to avoid inpatient episodes or achieving early intervention. This category is heavily weighted toward various screenings for conditions like cancer or heart disease, and administration of low-cost interventions such as flu shots and statins.
  • At-Risk Population management (5 metrics) looks at how well the ACO is doing when managing diabetic patients, or beneficiaries with heart disease, high blood pressure, or depression.

Each of these metrics are basically graded on a curve, with points awarded based on where an ACO ranks on a percentile basis among all ACO’s, for each measure. Each measure can earn an ACO up to 2.0 points, although in 2015 the meaningful use EMR metrics was worth up to 4 points.

In addition, depending on the performance year for the ACO (for example, year 1, 2 or 3), certain metrics might simply be “reported”, versus “paid”. In the case of “reported”, this means that the ACO gets points merely for reporting their performance, regardless of its level of performance. By the end of the 3-year phase-in period, all metrics are “paid” based on actual performance.

While certainly a good set of metrics to track, there are certainly limitations here as well. One issue is that the heterogeneous nature of the network that Medicare patients travel makes it a challenge to stitch together all the data necessary to evaluate some metrics. Additionally, some of the metrics require quite a bit of clinical knowledge to determine if an individual patient is being treated as the metric would want them to be treated. As such, the process of calculating how an ACO is performing for some metrics can be expensive and time-consuming.


The CMS ACO program in general is growing, and CMS is investing in programs and new charge codes to incent provider networks to learn how to manage population health with a value approach, but without substantial up front risk. Each year, new programs are being introduced out of the CMS innovation initiative, targeted at the highest risk, highest cost populations. While the method in which an ACO is measured from a quality and cost standpoint can be less than ideal, the ultimate goal of these investments is to move the industry toward capitation in a thoughtful manner, with many trials along the way to discover what works.