Effects of the Ticket to Work Program: Return on Investment and Overall Assessment of Outcomes Versus Design

by
Social Security Bulletin, Vol. 83 No. 1, 2023

The Ticket to Work (TTW) program was established by 1999 legislation to expand access to vocational rehabilitation services for beneficiaries of Social Security Administration (SSA) disability benefit programs. We evaluate TTW and compare its outcomes with the intentions of its authorizing legislation. We also compare the program's costs with the benefit savings resulting from the reemployment of successful program participants. We analyze unadjusted descriptive statistics, then consider potential participation bias among individuals who received employment services by devising an econometric analysis that accounts for the seriousness of a participant's interest in work. We find that TTW improved employment outcomes and generated net benefit savings to SSA for many employment-service clients, but the savings did not fully offset program costs. However, these estimates should be regarded as lower bounds of TTW's positive effects because they do not account for higher service needs of TTW program participants.


Paul O'Leary is an economist with the Office of Retirement and Disability Policy, Social Security Administration. Emily Roessel is a senior policy researcher with the Social Security Advisory Board.

Acknowledgments: The authors thank Mark Trapani for his particularly insightful guidance on early drafts of this manuscript as well as helpful comments on the final draft. We also thank Kai Filion, Jackson Costa, David Weaver, Robert Weathers, and Jeffrey Hemmeter for useful feedback and suggestions.

The findings and conclusions presented in the Bulletin are those of the authors and do not necessarily represent the views of the Social Security Administration.

Introduction

Selected Abbreviations
BFW benefits forgone for work
CDR continuing disability review
DAF Disability Analysis File
DI Disability Insurance
EN employment network
GAO Government Accountability Office
OLS ordinary least square
SGA substantial gainful activity
SSA Social Security Administration
SSI Supplemental Security Income
STW (benefits) suspended or terminated because of work
SVRA state vocational rehabilitation agency
TPM Ticket Program Manager
TTW Ticket to Work
TWP trial work period
VR vocational rehabilitation

The Social Security Administration (SSA) has operated the Ticket to Work (TTW) program for 2 decades. TTW aims to enable SSA disability program beneficiaries to exit the program rolls and attain self-sufficiency through work by facilitating their access to and participation in employment training and support services. Many policymakers had high hopes for TTW when it was established and, for some, the program's results have been disappointing. This analysis examines the program's intentions, as described in the Ticket to Work and Work Incentives Improvement Act of 1999, and its outcomes through 2016. It also compares the program's costs with the benefit payments forgone for the beneficiaries the program served as intended.

TTW was based on the premise that many Supplemental Security Income (SSI) and Social Security Disability Insurance (DI) beneficiaries had the functional capacity and the desire to work but had too few options for employment services. Before TTW, all such services were provided by entities called state vocational rehabilitation agencies (SVRAs). It was thought that because SVRAs faced no competition for SSA disability program clients and received payments from SSA under a cost-reimbursement model, the existing vocational rehabilitation (VR) system did not give SVRAs incentives to provide the innovative solutions that could enable more of their clients to discontinue cash benefits through long-term employment. A Disability Policy Panel convened by the National Academy of Social Insurance in 1993 envisioned an expansion of employment service availability and choice by allowing public and private service providers, which would come to be known as employment networks (ENs), to compete with SVRAs for clientele. The panel also conceived a performance-based model for SSA payments to the service providers (Mashaw and Reno 1996). The existing payment system, entailing reimbursement for costs associated with providing employment services, would remain available; but providers would now have the option of receiving payments based on their clients' performance-based milestones and outcomes. A key feature of the new system would be the direct linkage of provider payments to beneficiary progress toward employment success and the resulting reductions in benefit payments. In this way, the program attempted to align the incentives of the employment service providers with those of SSA and the beneficiaries attempting to exit the disability rolls and achieve financial independence via work (Stapleton and Livermore 2003; Huynh and O'Leary 2003).

The Ticket to Work and Work Incentives Improvement Act, Public Law 106-170, was signed into law on December 17, 1999. A centerpiece of the act was the Ticket to Work and Self-Sufficiency Program, which was meant to comprehensively address longstanding issues that were seen as limiting employment for beneficiaries with disabilities. TTW created the new class of service providers, ENs, that the Disability Policy Panel had envisioned, and established the rules that aligned the SSA provider payments they received with the long-term success of beneficiaries. The legislation also protected beneficiaries from continuing disability reviews (CDRs, which affirm or deny ongoing eligibility for benefits) while they participated in TTW,1 extended access to Medicare or Medicaid coverage for beneficiaries returning to work, and allowed for expedited benefit reinstatement if the work attempt ultimately failed. As the Ticket to Work and Work Incentives Advisory Panel noted in their final report (2007), the act “represented the culmination of efforts by diverse stakeholders who recognized the importance of removing barriers to employment and labor market activity for individuals with disabilities.” The panel further noted that the legislation “demonstrated the evolution of thinking and attitudes about the interest in and capacity of people with disabilities to work, contribute to our nation's economy, and reduce their reliance on public benefits.” Radtke (2000) described the act as “one of the most significant changes in disability policy in the last 20 years.”

Despite early optimism, as the program unfolded and the results of SSA's evaluations were released, many came to regard TTW as a failure (Butler 2018; Government Accountability Office [GAO] 2011; Khazan 2015; Lankford 2017; Lawler 2014; SSA 2008). Eimicke, Cohen, and Miller (2017) noted that “by December 2005…a mere 1.8% of disability beneficiaries were participating in the TTW program in the 13 Phase 1 states” and that “as of early 2007, ENs as well as SVRAs were losing interest in the program.” Autor and Duggan (2010) described the TTW program as “ineffective” and noted that “fewer than 1,400 tickets (0.01 percent) of 12.2 million tickets issued in the first seven years of the…program led to successful workforce integration.”

It seems clear that the TTW program has not achieved the lofty changes many had anticipated for it. It is less clear, however, whether one should regard the program as a failure. Improving the rate at which beneficiaries leave the disability program rolls and become self-sufficient was an obvious goal of the program, but it was not the only goal, and one should consider all of TTW's objectives in assessing its level of success. That is a complex undertaking, as measures of program success are open to interpretation. SSA hired an independent contractor to evaluate TTW, which produced extensive findings on TTW activities, outcomes, and effects. In seven reports,2 the evaluation delves into topics ranging from SSA's implementation of TTW, participation by ENs and beneficiaries, the effect of regulatory changes implemented in 2008, crossover effects between SVRAs and ENs, program costs, and outcomes involving work attempts, earnings, and disability benefit receipt. Although the reports describe successes, failures, and adjustments, SSA did not design the evaluation to reach a single conclusion on TTW. Instead, the agency intended the evaluation to provide detailed findings on the various program aspects to inform decision makers and the public. As such, readers can reach a variety of conclusions from the reports.

To provide an overall TTW program assessment, we examine two basic research questions: Does the program achieve the stated objectives of its authorizing legislation, and has it produced benefits that exceed its costs? Although these questions are conceptually simple, overall costs and benefits can be difficult to measure.

The analysis is structured as follows. We start with an overview of expectations for the TTW program as outlined in the legislation that created it. We then review the findings from the TTW evaluation in the context of the legislative goals. We then extend those findings in two ways. First, we analyze service delivery, program costs, and employment and benefit outcomes without regard to what would have happened in the absence of TTW. Second, we assess employment outcomes using multivariate regressions, focusing on DI beneficiaries who have completed a trial work period (TWP)—during which a beneficiary can accrue earnings without risking benefit termination, under program rules—as a means of addressing selection bias among those who choose to use TTW services.

Overview of TTW Expectations and Perceived Results

The text of Public Law 106-170 stated that the primary purpose of TTW was

to establish a return to work ticket program that will allow individuals with disabilities to seek the services necessary to obtain and retain employment and reduce their dependency on cash benefit programs.

Subsequent passages, designated as “findings” 1 and 10–12 of the law's text, suggested additional goals of the program:

(1) It is the policy of the United States to provide assistance to individuals with disabilities to lead productive work lives.
(10) Eliminating…barriers to work by creating financial incentives to work and by providing individuals with disabilities real choice in obtaining the services and technology they need to find, enter, and maintain employment can greatly improve their short and long-term financial independence and personal well-being.
(11) In addition to the enormous advantages [this law's] changes promise for individuals with disabilities, redesigning government programs to help individuals with disabilities return to work may result in significant savings and extend the life of the Social Security Disability Insurance Trust Fund.
(12) If only an additional one-half of one percent of the current Social Security Disability Insurance and Supplemental Security Income recipients were to cease receiving benefits as a result of employment, the savings to the Social Security Trust Funds and to the Treasury in cash assistance would total $3,500,000,000 over the worklife of such individuals, far exceeding the cost of providing incentives and services needed to assist them in entering work and achieving financial independence to the best of their abilities.

Thus, although the primary goal to TTW was to reduce benefit payments by significantly increasing the rate at which benefits can be suspended or terminated because of work (STW), the act clearly included at least three other, though arguably secondary, objectives:

  1. Expand the services available to and used by beneficiaries that might allow them to obtain and retain employment.
  2. Improve the lives of beneficiaries by enabling greater self-sufficiency, with increased earnings allowing the reduction or elimination of dependence on cash benefits.
  3. Bring about benefit reductions in amounts that exceed the cost SSA pays to providers for employment services.

Since the completion of the TTW evaluation in 2013, stakeholders have tended to focus on the finding that the program has not made significant progress toward the primary goal of reducing the disability rolls. Less attention has been paid to the substantial evidence, presented below, that TTW has (1) expanded the range of services available to beneficiaries, (2) had a positive effect on participants, and (3) done so with costs that are no higher than those for the SVRA-only services it replaced, and are potentially lower than the benefit reductions that have ensued.

TTW Effects on Benefit Receipt

As noted above, SSA funded a comprehensive TTW evaluation, as required in the authorizing legislation. SSA contracted with the Lewin Group in 2000 to design the TTW evaluation, and with Mathematica (then known as Mathematica Policy Research) in 2002 to implement the evaluation. SSA also contracted with Mathematica in 2003 to conduct a survey to collect information needed for the evaluation that was not available in agency administrative data. Mathematica conducted the survey in four rounds from 2004 through 2010, collecting information from more than 20,000 beneficiary respondents.

The evaluation spanned 11 years and produced seven reports, each comprising from one to nine separately published studies, and several policy briefs. The fourth evaluation report (Stapleton and others 2008) and a study from the seventh report (Stapleton, Mamun, and Page 2013) specifically addressed TTW's net effect on the size of SSA's disability beneficiary population. If that outcome is measured by the rate at which TTW enabled DI disabled-worker beneficiaries to resume work at an earnings level that constitutes substantial gainful activity (SGA)—and thereby allows them to forgo cash benefits—then Table 1 appears to show that TTW had little or no effect.3 Although the benefit termination rates for SGA were marginally lower than 0.55 percent in years after 2001, the last full year before TTW Phase 1 implementation, the rates varied little. However, this simple measure does not account for differences over time in the beneficiary population or other environmental factors. The results in Table 1 are generally regarded as “gross outcomes,” as they do not attempt to account for related factors that could accentuate or offset any TTW effects. Results of analyses that attempt to remove or “net out” these related factors are generally regarded as “net effects.” To account for these other factors, more sophisticated statistical methods are needed. The evaluation contractor therefore attempted to measure the net increase in the rate at which beneficiaries reduced their dependence on disability benefits that was specifically attributable to the TTW program. Stapleton and others (2008) reported the results of the first of these analyses, and Stapleton, Mamun, and Page (2013) revised and updated those results using improved analytical techniques and additional data. The findings of these two analyses were consistent, and because the 2013 analysis was more robust, we focus on that latter report here.

Table 1. DI disabled-worker beneficiaries: Total and with benefits terminated because of SGA-level work, 2001–2016
Year Total With benefits terminated because of SGA
Number Percent
2001 5,268,039 29,000 0.55
2002 5,539,597 29,165 0.53
2003 5,868,541 27,926 0.48
2004 6,197,385 28,613 0.46
2005 6,519,001 36,263 0.56
2006 6,806,918 36,242 0.53
2007 7,098,723 33,381 0.47
2008 7,426,691 37,711 0.51
2009 7,788,013 32,445 0.42
2010 8,203,951 40,959 0.50
2011 8,575,544 39,813 0.46
2012 8,826,591 38,228 0.43
2013 8,940,950 31,591 0.35
2014 8,954,518 35,846 0.40
2015 8,909,430 39,652 0.45
2016 8,808,736 47,887 0.54
SOURCES: SSA, Annual Statistical Report on the Social Security Disability Insurance Program, 2001–2016 editions, Tables 1 and 50.

SSA implemented TTW in three phases, each encompassing a different group of states, from the beginning of 2002 through the end of 2004. The 9/11 attacks disrupted the Phase 1 rollout in New York, leaving only Phases 2 and 3 as viable for analysis. In each phase, SSA divided randomly selected beneficiaries into 10 “mail-month” groups. SSA mailed tickets, which program participants could exchange for employment services with an SVRA or an EN, to the first group in November of that phase's year, sent no tickets in December, and sent tickets to one of the remaining groups each month from January through September of the next year. Thus, a random subset of beneficiaries in each phase got their tickets as much as 11 months sooner than those in the latter mail-month groups. Stapleton, Mamun, and Page (2013) used this randomness to estimate the difference in TTW effects between the early and late mailing groups.4 However, because all mail-month groups in each phase received tickets within an 11-month span, the period available for measuring TTW effects was limited, which in turn limited the precision of this evaluation method.

Both the 2008 and 2013 analyses concluded that TTW had generated a statistically significant net increase in employment service enrollment with both ENs and SVRAs, but it did not increase, at statistically significant levels, the proportion of beneficiaries with disabilities who were able to transition from cash benefits to STW status—that is, to self-sufficiency. In particular, the evaluators found evidence that receiving a ticket 11 months sooner than other beneficiaries in Phase 2 increased the attainment of STW status by 7 percent, and the number of months in STW status by 5 percent, as of 48 months after the phase began. However, for Phase 3, the evaluators found no significant effect on STW status, and the effect on the number of STW-status months was similar to that for Phase 2 but in the opposite direction. They also found that TTW's effects on the completion of TWP months were insignificant for Phase 2, which cast further doubt on the positive Phase 2 STW-status findings because improved STW-status outcomes should be associated with improved TWP outcomes, as the latter is a precursor to the former. The evaluators concluded that the effects they measured were effectively random and therefore not statistically different from zero.

Although the evaluators' finding was disappointing, two points are worth considering. First, Congress designed the TTW program to supplement the cost reimbursement system that SSA had used for SVRAs since 1981. Under that system, SVRAs were reimbursed if a client attained an earning objective, with or without reaching STW status. TTW added ENs as eligible service providers and allowed ENs and SVRAs to choose whether to be reimbursed under the traditional system or under one of two alternatives: either a “milestone plus outcome” or an “outcome only” system, which tied payments to the client's STW status in the current month. Following standard practice, the evaluator analyzed the TTW's effects relative to the prior system. In other words, the net effect of TTW was the difference between the combination of EN and SVRA services and SVRA services alone, the services which had been available prior to TTW. The evaluation did not, and could not, measure the effect of TTW by comparing outcomes for beneficiaries receiving services from an SVRA or an EN to outcomes for beneficiaries who had no access to services.

The TTW evaluation did not attempt to measure the effect of employment services on the beneficiaries who used them, but other studies have, albeit with several limitations. For example, Dean, Dolan, and Schmidt (1999) and Dean and others (2001, 2014) used the earnings of SVRA clients to measure success, and compared annual earnings to service costs to assess their performance. Those studies found that beneficiaries generally increase employment and earnings after receiving SVRA services and that their earnings exceed the cost of services.5 Earnings relative to service costs, however, tell us little about effectiveness relative to disability benefit receipt. One analysis of earnings relative to SGA found that such earnings are not sufficient to discontinue disability benefits (GAO 2007). However, because SVRA-provided supports represent an ongoing national program, it is not possible to use an experimental design to study their effects. Researchers are thus left with only quasiexperimental methods that compare participants with characteristically similar nonparticipants such as VR service dropouts. Whether the comparison groups have been similar enough to participants to address the inherent selection bias implicit in completing VR services is unclear and this limits the strength of the findings in these studies. We will discuss selection bias in more detail later.

Second, the evaluation notably did not estimate TTW effects on participants only. By necessity, Mathematica evaluated TTW's effects on all beneficiaries (this is known as the “intent-to-treat” approach in the economic literature). Because TTW serves a relatively small proportion of DI beneficiaries and SSI recipients with disabilities, and many such beneficiaries and recipients forgo benefits for work without using employment services, TTW's effects on participants would need to be substantial to have a measurable effect on the larger population of beneficiaries and recipients. Mathematica did not find a large effect, but a small positive effect would have been undetectable because of the limitations of the methods available.

Both evaluation-design elements—the assessment of TTW effects relative to the prior system and the use of intent-to-treat methodology—made sense, given the way Congress required SSA to roll TTW out and the program's main goal of substantially increasing exits from the disability benefit rolls. However, these design parameters limited the methods available to the evaluator, and those methods limit the implications of the results. We know that TTW did not lead to a large increase in disability benefit exits, but we do not know if smaller effects were achieved. In fact, Stapleton, Mamun, and Page (2013) found that “the statistical power of the projections for STW [status] and the number of STW [status] months is insufficient to rule out the possibility that TTW had impacts of at least five percent on each outcome for Phases 2 and 3 pooled.” More precisely, their analysis found that TTW increased STW status incidence by less than 30 percent and the number of months in STW status by less than 9 percent, but they could not rule out smaller effects that approached those boundaries. This is not to say that the evaluation found evidence that positive outcomes occurred. Instead, we merely note that, given the inherent limitations of the evaluation methodology, effects of less than these magnitudes could have occurred without statistically significant detection. The methodological restrictions simply prevented a more precise result.

The evaluation also could not assess TTW's effect on other outcomes of interest. In particular, it did not address whether EN and SVRA services significantly increased employment among participants. Because employment by itself does not directly affect benefit eligibility, SSA does not track employment data. SSA tracks earnings, but only if they are high enough to affect benefits. Further, pre- and post-TTW earnings data are inconsistent because the quality of the earnings tracking changed after TTW implementation, as the program required SSA to improve its earnings data to calculate EN payments accurately. As such, the earnings and employment data for disability program beneficiaries that would allow a comprehensive TTW assessment are in some instances unavailable, or are available with gaps, especially for the pre-TTW comparison period. These limitations were not flaws of the evaluation design. Rather they reflect what was possible given the information available to the evaluator, the characteristics of the TTW program, and the way Congress required SSA to implement it.

Even the finding that TTW did not greatly affect the disability roll exit rate is more complicated than it first appears. The evaluation found that the success rate that resulted from combining EN and SVRA services was statistically no different from offering only SVRA services. To the extent that SVRA services had been effective prior to TTW, they continued to have a similar level of success after TTW, and the success rates for ENs were similar to the SVRA results. In fact, the success rate was higher for ENs, meaning that the new ENs were at least as effective as SVRAs. This does not necessarily mean, however, that ENs are better at serving beneficiaries than SVRAs. Because ENs can choose their clients while SVRAs cannot, adding ENs to the service mix under TTW may have simply shifted some of the easier-to-serve cases from SVRAs to ENs.6 Regardless, combined EN and SVRA services reach the same level of success that SVRAs alone had previously achieved. We explore this finding in further detail later.

Other Effects of TTW

As noted earlier, increasing the rate at which beneficiaries leave the disability rolls for work and thereby lower the amount of cash benefits paid by SSA was not TTW's only objective. Other objectives included expanding the availability and use of employment services, increasing the self-sufficiency of people with disabilities through expanded earnings, and spending less on employment services than the dollar value of benefits that would be paid if beneficiaries remained on the rolls.

TTW clearly increased consumer choice for beneficiaries. Further, under TTW, SVRAs could choose, case by case, whether to serve a client under either the existing cost reimbursement model or one of the new TTW payment options—in the latter cases, operating essentially as ENs. In addition to these SVRAs operating under both the new and old payment rules, approximately 600 ENs joined the ranks of employment service providers under TTW.7 Many of those ENs were VR vendors that had provided services under contract with SVRAs, but others offered new service models. These included “consumer-directed service ENs, which share a portion of the TTW payments they receive with their clients; employer ENs receiving TTW payments based on work activity of their TTW participant employees; [and] state workforce agency ENs comprising local workforce investment boards and One-Stop Career Centers” (Schimmel and others 2013). Beneficiaries have responded to the expanded choice by using EN services at an increasing rate since they were first offered, and this increase in choice also likely contributed to some of the increase in participation in return-to-work services that Mathematica found. Using data from SSA's Disability Analysis File (DAF), Chart 1 shows that although SVRAs have dominated the TTW ticket assignments, EN assignments have risen steadily since that option was fully implemented in 2005.8 Although SVRAs received 93 percent of ticket assignments in 2005, their market share had dropped to 72 percent by 2019. Further, while the numbers of tickets assigned to SVRAs have remained relatively constant since 2012 (with slight declines since 2015), EN assignments have increased by 80 percent since 2012, indicating progress toward Congress' intent of expanding service alternatives.

Line chart with tabular version below.
Show as table
Table equivalent for Chart 1. TTW participation: Tickets assigned, by provider type, 2005–2019 (in thousands)
Year ENs SVRAs (adjusted) SVRAs (unadjusted)
2005 4.769 66.172 63.583
2006 4.581 71.490 66.312
2007 4.522 77.710 69.944
2008 10.646 88.770 78.415
2009 14.748 93.707 80.763
2010 19.121 104.676 89.143
2011 18.998 107.234 159.010
2012 17.424 97.133 97.133
2013 22.519 87.032 . . .
2014 18.809 98.972 . . .
2015 18.742 106.992 . . .
2016 21.284 104.453 . . .
2017 24.022 100.014 . . .
2018 28.024 91.193 . . .
2019 30.679 90.934 . . .
2020 17.704 44.448 . . .
SOURCE: Authors' calculations based on 2020 DAF.
NOTE: SSA instituted stricter reporting requirements for SVRAs in 2011. The adjusted figures for SVRAs smooth the spike in “catch-up” assignments reported for 2011 by estimating likely annual flows had the reporting requirements been in place from 2005 forward.
. . . = not applicable.

Mathematica also found that TTW increased the level of interest in work among disability program beneficiaries. The National Beneficiary Survey (NBS), sponsored by SSA and conducted by Mathematica, includes questions that probe beneficiaries' expectations about future employment. Comparing results from the first NBS in 2004 (Thornton and others 2006) with those from the round completed in 2015 (SSA 2018a) shows an increase in beneficiary interest in substantial work across multiple measures:

This increase in interest in work is consistent with the net increase in participation in employment services noted above. Stapleton, Mamun, and Page (2013) found that employment service enrollment increased by about 15 percent over the 48-month observation period in the Phase 2 and Phase 3 states. Increases in reported interest in work and in employment service participation would seem to be a significant positive result, even without increases in disability roll exits. Congress intended TTW, at least partly, to improve opportunities for disability program beneficiaries and to change expectations about work, so it seems reasonable to regard higher interest in work and employment service participation as TTW successes.

Not all of the additional participants assigned their tickets to ENs: They distributed their tickets among both provider types, and the TTW evaluations found differences between participants who sought EN services and those who sought SVRA services (Thornton and others 2006; Stapleton and others 2008). For example, Stapleton and others found that

participants were more likely to assign their Ticket to an EN if they had low benefits, were African American, had minor children, or had been on the rolls for a year or less. Those with [intellectual disability, with] sensory or other nervous system disorders, [with] severe mental health problems, or in need of assistance to perform a daily activity were more likely to assign their Ticket to an SVRA than to an EN.

Thus, TTW likely led a broader range of beneficiaries to attempt employment and gave them greater chances of finding services that fit their situations. This, combined with the overall increase in employment service use, suggests that TTW prompted certain beneficiary groups to consider work and to pursue employment goals in greater numbers.10 The fact that participation increased without an increase in the proportion reaching STW status implies that these new participants were less successful overall than those who had participated before TTW implementation. This suggests that TTW broadened the range of beneficiaries interested in work, drawing participants from groups facing marginally greater challenges to work than those who had received SVRA services prior to TTW. These additional participants from the margin likely had less success than other beneficiaries. Moreover, even if we saw no increase in STW status for these beneficiaries, they may have improved their self-worth by attaining some level of work and likely improved their quality of life by adding some earnings to their benefit income.

Livermore, Hoffman, and Bardos (2012) found that most TTW participants were satisfied with the services they received during the program's early rollout. About 60 percent of EN users considered themselves “very” or “somewhat” satisfied with those services. Satisfaction was higher for SVRA clients, at about 70 percent. Regression-adjusted satisfaction results improved for both EN and SVRA users after SSA implemented modified TTW regulations in 2008, although only the increase for ENs was statistically significant, at 36 percent.11

Each year, many SVRAs do not have enough resources to serve all their applicants. When that occurs, those SVRAs enter “order of selection” status, wherein they must ration services and prioritize individuals with the most significant disabilities.12 SVRA clients include individuals who are not beneficiaries of SSA programs; and although SSA program beneficiaries tend to have the most significant disabilities among SVRA applicants, they do not constitute all the clients who are prioritized under orders of selection (Honeycutt and Stapleton 2013). Thus, by adding ENs as service providers, TTW has helped to expand the availability of employment services for beneficiaries despite tight fiscal resources among SVRAs. Without ENs, either SVRAs would need to increase the level of services they provide to absorb the demand currently fulfilled by ENs or unmet demand for employment support services would increase.

Although TTW has not significantly increased the number of beneficiaries who reach self-sufficiency relative to an SVRA-only system, it has increased the overall availability of employment services, consumer choice, and beneficiary participation, and beneficiaries are generally satisfied with the services they receive through TTW.

Measuring TTW Costs and Benefits

Assessing whether TTW's net effects were “substantial” is a matter of interpretation. The most policy-relevant question is whether TTW provided benefit savings (and other effects) sufficient to offset program costs.

TTW participants represent a relatively small proportion of disability beneficiaries. As a result, Thornton (2012) estimated that relatively modest program effects, on the order of a net increase of 2,000 to 3,000 disability roll exits, would be sufficient to offset the costs of the TTW program. Unfortunately, measuring such a small effect was precluded by the evaluation's limited precision:

The fact that we did not find statistically significant impacts on STW [incidence or months in STW status] does not by itself rule out the possibility that TTW under the initial regulations had impacts on these outcomes that were sufficiently large for the program to be “self-financing”—that is, for savings from a net reduction in benefits to be sufficient to pay for TTW payments to providers and all administrative costs attributed to the program. Thornton (2012) suggests that only a very small impact—an increase of 3,000 or so in the number of all beneficiaries experiencing STW [status] for the first time in each year—might be sufficient for the program to be self-financing. An annual impact on first-time [STW incidence] that is as small as 3,000 might correspond to such a small impact on STW [incidence] as of month 48 for new, young [DI] beneficiaries that the evaluation would be unable to differentiate between that impact and no impact at all (Stapleton, Mamun, and Page 2013).

Stapleton, Mamun, and Page estimated that 3,000 new instances of STW status in 2007 as a result of TTW would have represented a 5 percent increase in those cases. This leaves us with a rather ambiguous evaluation result. We know that the program did not lead to large-scale changes—TTW increased STW incidence by less than 30 percent and it increased the number of months in STW status by less than 9 percent—but we cannot rule out the possibility that it had effects that were of policy relevance. An increase as modest as 5 percent in DI and SSI exits would be sufficient to cover the cost of the program, but we cannot tell if such a change occurred.

We can look at cost in the broader sense as well. Because the employment service participation rate increased without a corresponding increase in the rate of disability roll exits, we would expect that the cost of providing such services rose. This in fact took place, but not necessarily because EN services were, on average, more costly. As we detail in Appendix A, ENs and SVRAs have different cost structures related to their different client populations, and this complicates the comparison of costs between the two provider types. Thornton (2012) found that the total provider payments per STW case were notably lower for ENs than for SVRAs. He estimated that SSA-financed return-to-work assistance per STW case was $11,500 (in 2009 dollars) for ENs and $14,035 for SVRAs.13 “The $2,535 difference represents the savings SSA receives when it pays for exits under the new TTW system, which makes outcome payments only while beneficiaries remain off of cash benefits, rather than under the traditional payment system” (Thornton 2012, 40). This result should be viewed with caution as it defines costs narrowly for both ENs and SVRAs, but it does suggest that costs could be higher if SSA reverted to an SVRA-only system and SVRAs served all beneficiaries currently receiving services through ENs.14

Even under Thornton's narrow definition of costs, the actual increase in SVRA costs relative to those for ENs would likely be less than the $2,500 per STW case noted above because the lower EN costs partly reflect the ability of ENs to choose to serve only lower-cost beneficiaries. If those beneficiaries used SVRAs, they would likely generate lower costs and SSA reimbursements, bringing down the average SVRA cost per STW case. Still, as we describe in Appendix A, it seems safe to conclude that TTW has expanded service availability without increasing costs over what would be required to serve the same number of beneficiaries under an SVRA-only system.

Comparison of TTW Costs and Benefits

Regardless of whether TTW expanded service availability and use or changed net disability benefit receipt, we would like to know whether TTW services are effective and provide efficient results for participating beneficiaries. In other words, did TTW services decrease SSA benefit payments relative to what participants would have received in the absence of those services, and were the costs of TTW services less than the net reductions in benefits? Stapleton, Mamun, and Page (2013) focused on whether TTW changed the rate at which disability program beneficiaries return to work relative to the SVRA-only system that preceded it. Their study did not assess whether either the SVRA or EN components of TTW led to reduced benefit payments relative to no services. This is an important distinction. We showed above that TTW expanded service delivery and had the same success as the SVRA-only program that preceded it. We now address whether services provided through SVRAs or ENs under TTW reduce reliance on cash benefits. This is a crucial question, as nothing is gained in expanding an ineffectual program.

Although TTW subsumed the national SVRA services program for SSA program beneficiaries with disabilities, many aspects of SVRA and EN services remain separate. We therefore explore the costs and benefits of these two provider types separately, assess their efficiencies, and compare their results.

To estimate the beneficial effects of TTW, we focus on the reductions in disability benefits that have resulted from participation. Ideally, we would estimate reductions in disability benefits that have been adjusted to account for the counterfactual: what would have happened if neither EN nor SVRA services were available under TTW. As noted in our discussion of the 2008 and 2013 evaluation reports, however, the methods available for estimating these effects have shortcomings. We therefore start by estimating gross benefit reductions without considering the counterfactual. Because gross benefit reductions are essentially an upper-bound estimate of TTW effects, they can be compared with costs to get a sense of whether net benefits exist at all. If costs are greater than any associated reductions in benefit payments, the program cannot pay for itself. Conversely, if the magnitude of benefit reductions grows relative to program costs, the opportunity for actual savings increases. So, although the outcome analysis cannot definitively show whether TTW provides savings, it can compare program costs to the benefits forgone for work (BFW) to provide an upper bound on TTW effects and thus set context for analyzing whether TTW may offset all or most of its costs.15

TTW Costs

The two components of TTW program-related expenses are the administrative cost of running the program and the SSA provider payments to ENs and SVRAs. Agency administrative data closely track the provider payments as part of managing the program. SSA's Office of the Inspector General reports these amounts by the year when the payments occur (SSA 2018b). This accounting method has problems because it is affected by budgetary and administrative priorities that are not directly relevant to TTW operations. Reporting cost by date of disbursement tends to show lower payment levels in years when staffing for SSA payment processing was tight, followed by higher payment levels in subsequent years as processing catches up with backlogs. Recording payments by the dates when they are due—when the service provider completes the documentation that supports that payment—would better represent the timing of the beneficiary's work activity that triggered the payment.

Table 2 shows the number and value of payments made to employment service providers from 2002 through 2016.16 The numbers of payments under the new TTW systems prior to 2006 were much lower than traditional-system reimbursements as SSA phased in the new program. The low figures in the early years also reflect the lag between the start of EN services and the point at which participants began working at a level sufficient to initiate a payment for the service provider. Payment frequencies under the TTW systems picked up starting in 2006 and then rose significantly after 2008 when SSA implemented new TTW regulations. In the later years, providers using the new milestone- and outcome-oriented payment systems established by TTW received far greater numbers of payments than did SVRAs using the traditional reimbursement model. This reflects the nature of the two payment structures, wherein traditional reimbursement is essentially a “lump-sum” payment after service completion while multiple payments under the newer TTW systems occur over time as beneficiaries reach employment milestones. By 2016, total annual new TTW-system payment amounts had risen to more than $63 million. Yet because the new TTW-system payment amounts are relatively small on average, their total value each year is still lower than that of traditional-system reimbursements. Payment amounts under the traditional system have averaged around $125 million annually, although payments in 2003, 2010, and 2011 were noticeably lower, likely because those years were affected by recessions. All payments under the traditional system went to SVRAs; most of the payments under the TTW's new systems went to ENs, but about 20 percent went to SVRAs (not shown).

Table 2. Annual TTW payments to employment service providers, by payment system, 2002–2016
Year of work activity Traditional reimbursement system (SVRAs only) New TTW payment systems (ENs and SVRAs)
Number Amount (2016 $) Number Amount (2016 $)
Total Average Total Average
2002 8,732 148,808,101 17,042 619 237,580 384
2003 5,882 92,822,208 15,781 2,785 1,194,173 429
2004 6,297 101,406,129 16,104 5,831 2,562,568 439
2005 7,510 110,149,079 14,667 8,366 3,535,649 423
2006 6,784 104,411,154 15,391 11,265 4,551,720 404
2007 8,329 125,976,989 15,125 13,642 5,528,360 405
2008 9,761 143,962,909 14,749 25,416 16,939,049 666
2009 8,755 134,450,407 15,357 36,576 26,202,851 716
2010 5,807 90,124,147 15,520 40,730 29,584,050 726
2011 5,255 84,098,895 16,004 46,766 33,029,082 706
2012 7,138 104,086,930 14,582 51,422 35,280,750 686
2013 9,385 135,296,335 14,416 62,839 43,739,871 696
2014 11,137 160,042,350 14,370 76,098 52,458,226 689
2015 12,482 184,826,959 14,807 92,408 62,857,333 680
2016 a 15,687 a 199,507,673 a 12,718 96,897 63,088,995 651
SOURCE: 2016 DAF.
NOTE: Annual payment calculations based on the payment-triggering month for ENs and the last date of continuous SGA in the payment claim for SVRAs.
a. Estimated based on 11,765 actual payments for $149,630,755 in the first 3 quarters of 2016.

Table 2 includes an interesting dynamic in that the total payment amounts for the new TTW systems relative to the traditional system climbed steadily over time, growing from nearly nothing in 2002 to nearly 40 percent in 2011. Since 2011, TTW-system payment amounts have apparently settled at about one-third the amount of traditional-system payments, although differences in payment lag times between the systems might skew the proportions for 2015 and 2016. By 2015, total provider payments had reached about $250 million, with indications that they would continue to rise.

In addition to the payment costs to providers, SSA has incurred administrative costs to run TTW. Some of these costs can be viewed as the continuation of the SVRA administrative costs that existed before TTW subsumed the old system. TTW was not funded under a separate appropriation so there is no line item in the budget that can be used to cite TTW administrative costs. Because there also is no administrative need to track TTW costs separately from those for related functions, SSA does not do so. Still, stakeholders overseeing TTW have expressed interest in quantifying those costs. In response, we used SSA data on staffing levels and estimated the hours spent on TTW-related tasks to compute TTW administrative costs for 2017 in an unpublished SSA report (O'Leary and Roessel 2018).17 For this analysis, we adjust those 2017 cost estimates to 2016 dollars so that they align with our data on EN and SVRA provider payments.

The absence of TTW cost estimates for all program years poses alignment issues for this cost-benefit analysis, which covers TTW operations and beneficiary participation beginning in 2006. We make the simplifying assumption that the administrative costs that occurred in 2006 are the only administrative costs for this cohort.18 Ideally, 2006 costs would have been estimated shortly after the close of 2006 and then adjusted to 2016 dollars for this analysis, but no such estimates are available, and recall issues would make new estimates for that year unreliable. Instead, we use the 2017 cost estimate, adjusted to 2016 dollars, which implicitly assumes that annual TTW administrative costs have been flat since 2006. This poses a limitation, which we discuss further below.

In this analysis, the TTW administrative cost estimates account only for SSA staffing, printing, mailing, and contract resources specifically dedicated to the TTW program. Costs are separately calculated for operations related to EN and SVRA ticket assignments. Staffing costs include those for headquarters, regional office, and field office personnel and are based on workload assessments for a representative sample of staff. The administrative cost estimates do not include costs for activities not specifically related to the TTW program. They also do not include any costs for information systems except those managed by the Ticket Program Manager (TPM) under the TPM contract. Further, they exclude any costs associated with beneficiaries who remain on the rolls because SSA did not conduct a CDR, under the TTW provision protecting participating beneficiaries from undergoing one (which might lead to benefit cessation).19 Finally, we also exclude any potential excess costs of unrecovered overpayments for TTW participants and associated administrative costs.20

We estimate total TTW administrative costs for 2016 of $18.8 million, of which $12.3 million were for operations related to EN ticket assignments and $6.5 million were related to SVRA assignments (Table 3). Based on the 2016 figures from Tables 2 and 3, the administrative costs related to SVRA ticket assignments under the traditional payment system represent about 3 percent of total costs of operating that part of the program ($6.5 million ÷ [$199.5 million + $6.5 million]), and administrative costs related to EN assignments under the TTW payment systems represent about 16 percent of total costs of operating that part of the program ($12.3 million ÷ [$63.1 million + $12.3 million]). Notably, the costs related to SVRA assignments do not account for the substantial SVRA funding provided by non-SSA federal block grants and state funds, which we discuss in Appendix A.21

Table 3. Estimated TTW administrative costs for 2016
Cost category Amount (2016 $)
Total 18,835,670
SVRA ticket assignments
Subtotal 6,504,129
Staff 1,980,802
TPM contract 4,018,330
Printing and mailing 504,996
EN ticket assignments
Subtotal 12,331,541
Staff 3,674,133
TPM contract 8,538,952
Printing and mailing 118,456
SOURCE: Authors' calculations based on O'Leary and Roessel (2018).

Benefits of TTW

Prior analyses of the utility of employment services generally measured success based on the earnings of the client after the completion of services (for example, Ashenfelter 1978; Couch 1992; Dean, Dolan, and Schmidt 1999; and Dean and others 2001, 2015). Although a client's postservice earnings are of interest to SSA, they are not an ideal measure of TTW's effectiveness and efficiency. To SSA, the more important question is whether these services lead to a decreased reliance on cash benefits (Dean and others 2014), and more specifically, whether the reduction in benefits linked to TTW services is greater than what SSA pays for those services. SSA recently developed BFW, a new measure that uses an algorithm to calculate the cash benefits given up over time when beneficiaries start working (Mathematica 2022). BFW has been incorporated into the DAF, enabling us to use it for this analysis. Linking TTW service data with the BFW data in the DAF allows us to compare the cost of these services with any subsequent decrease in cash benefits because of work.

We examine the effect of TTW services by following a client cohort over time. A successful progression through employment services generally begins with enrollment (ticket assignment), followed by service provision, then employment, possibly involving several attempts. Because the traditional cost reimbursement system and the newer TTW payment systems are all outcome-oriented, some level of employment success is necessary before SSA begins to incur costs in the form of employment service provider payments. For reimbursement under the traditional payment system, an SVRA's client must attain earnings of SGA level or higher for at least 9 months in a 12-month period.

The TTW payment systems are more complex.22 Recall that SVRAs that opt for payment under one of these systems are effectively ENs. ENs can qualify for a phase 1 milestone payment, a phase 2 milestone payment, and an outcome payment.23 An EN qualifies for a phase 1 milestone payment when the beneficiary reaches 1, 3, 6, and 9 months of earnings at the TWP level ($810 in 2016). An EN qualifies for a phase 2 milestone payment each month that the beneficiary earns at or above SGA level. Outcome payments require that the beneficiary be in STW status.

For an SSI recipient, earnings of more than $85 per month can trigger a benefit reduction. As such, a reduction can take effect rather quickly, although the dollar reduction is usually modest. For DI beneficiaries, by contrast, there are no partial reductions in benefit amounts, and they must complete a TWP before attaining STW status, although the benefit reduction amounts are typically more substantial. Because of these differences, it is unclear whether TTW is more cost-effective for SSI recipients or DI beneficiaries. DI beneficiaries are better candidates for employment because of their work histories, but they typically are also older, are more likely to have physical disabilities that are associated with poorer return-to-work outcomes, and must sustain employment longer to generate benefit reductions. Conversely, SSI recipients tend not to have extensive work histories, but they are younger, are more likely to have mental conditions that are associated with better return-to-work outcomes, and can generate benefit reductions sooner (SSA 2020).

Regardless of the payment system, positive employment outcomes take time to occur. Clients might take a year or two to complete employment services, with another year to find stable employment, and, for DI beneficiaries, still another year to complete their TWP. Only then does the beneficiary enter an extended period of eligibility during which benefits are suspended because of work. Thus, substantial benefit reductions are likely to take 3–4 years after enrollment to emerge, and even longer if the labor market sours or the participant does not quickly find stable employment. Further, because the cost of employment services can be high, and SSA pays at least a portion of those costs before the client accrues substantial work duration, a long time may pass before the value of benefits not paid because of work covers the cost of services.

Because it takes time to see whether TTW outcomes may have offset costs, we examine a 2006 cohort of TTW participants and follow them through receipt of employment services and any attainment of STW status that ensues through 2016.24 We selected the 2006 cohort because pre-2006 cohorts participated in TTW while its rollout was not yet complete, and the experiences of the 2007–2009 cohorts were more directly affected by the anomalous economic conditions of the Great Recession. Employment outcomes for the 2008–2011 cohorts were lower than those for 2006, but beginning with the 2013 cohort, inflation-adjusted outcomes again approached the 2006 levels. The 2006 cohort should thus provide representative results for a period of relative economic stability.25

Under both the TTW and traditional payment systems, some payments occur before a beneficiary maintains work for a substantial period. This means SSA pays providers for some beneficiaries whose work effort never reduces benefit payments enough to fully offset the service costs. However, service providers must submit payment requests with documentation, as appropriate, in a timely manner to be eligible to receive those payments. Because service providers do not always complete these payment-request steps within program-required limits, some beneficiaries receive services and attain employment that reduces SSA's benefit costs without generating SSA payments to ENs or SVRAs.26 We track TTW participants by the type of provider to which they assigned their ticket and by whether SSA made payments to the provider. Table 4 shows the extent to which 2006 DI beneficiaries or SSI recipients aged 18–65 in January 2006 participated in TTW services during 2006 and had their cash benefits suspended because of work that followed employment-service receipt.27

Table 4. TTW participation in 2006 and employment outcomes as of 2016
TTW participation and outcome Number Percent Attained STW status by December 31, 2016 (%) a
Total 11,122,282 100.00 --
Never enrolled in TTW (through 2016) 10,982,613 98.74 3.2
Enrolled in SVRA services 135,363 1.22 18.7
SVRA received provider payment by year-end 2016?
No 127,945 1.15 15.4
Yes b 7,418 0.07 76.3
Enrolled in EN services 4,306 0.04 28.6
EN received provider payment by year-end 2016?
No 3,191 0.03 16.7
Yes b 1,115 0.01 62.7
SOURCE: Authors' calculations using 2016 DAF.
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
-- = not available.
a. Applies to individuals who received neither DI benefits nor SSI payments in at least 1 month because of work.
b. Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.

SSA's DAF indicates STW status as well as the BFW measure mentioned above. SSA created STW status as a research variable because the agency's administrative data do not specifically identify work as a reason for a suspension or termination. SSA uses an algorithm to compare an individual's benefit status with information on work, earnings, other income, medical reviews, retirement, and death to determine whether benefit suspension or termination resulted from work or for one of these other reasons (Mathematica 2022).

Table 4 shows that very few beneficiaries enrolled in TTW in 2006. Of the 11.1 million beneficiaries on the rolls at some point in 2006, only 1.2 percent enrolled for SVRA services that year and only 0.04 percent enrolled for EN services. Yet those who enrolled for services had substantially better employment outcomes through the end of 2016 than those who did not. Of those who could have enrolled in TTW in 2006 but did not, only 3.2 percent reached STW status for at least 1 month by the end of 2016. For those who enrolled in SVRA services in 2006, the proportion was 18.7 percent and for those entering EN services in 2006, it was 28.6 percent. As might be expected, those who generated SSA payments to their service provider had even higher rates of STW status by the end of 2016. Of those generating SVRA payments as of year-end 2016, 76.3 percent attained STW status for at least 1 month, while of those generating EN payments, 62.7 percent reached STW status for at least 1 month.

Table 5 shows the values of BFW and the costs incurred for those beneficiaries under TTW. Table 4 showed that 3.2 percent of the 2006 cohort was able to reach STW status without the help of EN or SVRA services and Table 5 shows that this small share of beneficiaries nonetheless generated significant BFW. This is because 3.2 percent of such a large base (nearly 11 million) is a substantial number of (current and former) beneficiaries. Through 2016, those who were eligible for TTW services in 2006 but did not use them generated nearly $16 billion in gross BFW—benefits that were not paid because those beneficiaries were in STW status. Because those beneficiaries received no services, there are no service costs and the net BFW are this same $16 billion.

Table 5. TTW participation in 2006 and BFW and cost outcomes as of 2016
TTW participation and outcome Number BFW and costs through December 31, 2016 (millions of 2016 $)
Gross BFW Administrative costs a Payments to service providers Net BFW
Total 11,122,282 16,647.2 18.8 119.2 16,509.2
Never enrolled in TTW (through 2016) 10,982,613 15,881.3 0.0 0.0 15,881.3
Enrolled in SVRA services 135,363 711.7 6.5 108.9 596.3
SVRA received provider payment by year-end 2016?
No 127,945 454.5 6.1 0.0 448.4
Yes b 7,418 257.2 0.4 108.9 147.9
Enrolled in EN services 4,306 54.2 12.3 10.3 31.6
EN received provider payment by year-end 2016?
No 3,191 8.8 9.1 0.0 -0.3
Yes b 1,115 45.3 3.2 10.3 31.8
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
Totals do not necessarily equal the sum of rounded components.
BFW and costs for TTW participants exclude those resulting from any subsequent ticket assignments.
a. 2017 costs adjusted to 2016 dollars.
b. Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.

The smaller proportions of beneficiaries that used EN and SVRA services are reflected in the lower BFW results. Overall, each dollar spent by SSA on SVRA services resulted in more than $6 in gross BFW, while each dollar spent on EN services generated more than $2 in gross BFW. SVRA clients who enrolled in 2006 ultimately generated $711.7 million in gross BFW, $6.5 million in administrative costs, and $108.9 million in provider reimbursement costs, leaving $596.3 million in net BFW. Those who enrolled with ENs generated $54.2 million in gross BFW, $12.3 million in administrative costs, and $10.3 million in provider payments, leaving $31.6 million in net BFW.

Interestingly, Table 5 also shows substantial gross BFW in cases that never generated a payment to a service provider: $454.5 million for SVRA users and $8.8 million for EN users. This can occur for a variety of reasons, but the most likely cause is service providers losing track of some participants who find work. Unaware of their clients' employment success, the providers are not timely in their request for payment from SSA (Stapleton and others 2010). With minimal administrative costs and no provider reimbursements in these instances, the net BFW are nearly equal to the gross BFW for these SVRA users. For EN users, the administrative costs more than offset the gross BFW, leaving a net loss of $0.3 million in these cases. For clients who generate provider payments, combined administrative costs and provider payments reduce gross BFW by 42 percent for SVRA enrollees and by 30 percent for EN enrollees, leaving net BFW of $147.9 million and $31.8 million, respectively.

Although the total amounts show the scale of BFW, they are not useful for examining TTW's relative efficiency. Table 6 repeats Table 5 with per-beneficiary averages in place of total amounts. Beneficiaries in 2006 who never enrolled in TTW services generated a relatively modest average net BFW of $1,446. SVRA clients who generated no payments to their providers averaged $3,504 in net BFW. SSA's cost for EN clients who generated no provider payments exceeded gross BFW by $99 per beneficiary.

Table 6. TTW participation in 2006 and BFW and cost outcomes per beneficiary as of 2016
TTW participation and outcome Number BFW and costs through December 31, 2016 (2016 $)
Gross BFW Administrative costs a Payments to service providers Net BFW
Total 11,122,282 1,497 2 11 1,484
Never enrolled in TTW (through 2016) 10,982,613 1,446 0 0 1,446
Enrolled in SVRA services 135,363 5,258 48 805 4,405
SVRA received provider payment by year-end 2016?
No 127,945 3,552 48 0 3,504
Yes b 7,418 34,672 48 14,682 19,942
Enrolled in EN services 4,306 12,579 2,863 2,392 7,324
EN received provider payment by year-end 2016?
No 3,191 2,764 2,863 0 -99
Yes b 1,115 40,671 2,863 9,237 28,571
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
BFW and costs for TTW participants exclude those resulting from any subsequent ticket assignments.
a. 2017 costs adjusted to 2016 dollars.
b. Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.

The average net BFW for SVRA clients who generated a provider payment were nearly $20,000 and the average net BFW for EN enrollees who generated a provider payment were about 50 percent higher, at nearly $30,000, despite ENs' higher administrative costs. This is because average gross BFW are significantly higher for EN enrollees than for SVRA enrollees, while average EN provider payment amounts are significantly lower. For TTW enrollees overall, disregarding provider payment status, EN clients generated substantially higher average net BFW ($7,324) than SVRA clients ($4,405).

Although it is tempting to conclude from Table 6 that EN services are more expensive than SVRA services but attain significantly better outcomes, that is a misleading oversimplification. EN services are more costly to SSA but differences in the SVRA and EN cost structures and client populations, detailed in Appendix A, result in a significant portion of SVRA costs being borne by state and federal governments through block grants.28 Further, costs are not distributed evenly across TTW participants; those with greater employment success incur higher administrative and service costs than those with less success. Further still, because SVRAs and ENs are subject to different enrollment rules, SVRAs receive far more ticket assignments than ENs and are reimbursed for a much smaller fraction of their clients (5 percent, versus 26 percent for ENs). Because costs are linked to success, average costs for SVRAs are significantly lower than those for ENs. Although this poses no problem for comparing administrative and provider-payment costs with BFW, as in Table 6, simple averages cannot tell us what the expected costs for a typical EN participant would be under an SVRA assignment or vice versa. Would the typical EN (or SVRA) participant be more (or less) successful than the typical SVRA (or EN) participant and thereby generate costs to SSA that are higher (or lower) than those of the other provider type, assuming similar results?

In Appendix A we adjust for the differences in SVRA and EN clientele and show that the average costs by provider type are similar—even if we ignore the effects of block grants. These adjustments are imperfect, but they imply that shifting clients from SVRAs to ENs and vice versa is unlikely to significantly change the overall cost of providing employment services to SSA disability program beneficiaries.

Beyond this, the higher BFW values attained by EN clients also reflect differences stemming from the enrollment rules under which the ENs and SVRAs operate. ENs are allowed to choose which beneficiaries they will serve while SVRAs are required to serve anyone they are able to help and, if they are over capacity, must serve those with the most severe disabilities first. Thus, SVRAs could be expected to have a harder-to-serve clientele, on average, than ENs have. In the same way that one would not expect the average cost for one provider type to apply to the other, one would not expect the average success rate for one provider type to apply to the clientele of the other: an SVRA's (or EN's) success would improve (or decline) if it served the other provider type's clientele. Rather than comparing costs and outcomes between ENs and SVRAs, the point to be drawn from Table 6 is that participants in both ENs and SVRAs attain BFW that are significantly greater than their costs to SSA, and the results are far better than those for beneficiaries who get no SSA-funded services.

Although the outcome comparisons in Table 6 are interesting, the comparisons between service recipients and nonrecipients are likely to include significant bias because seeking services is itself a strong signal of interest in work. That bias would tend to make TTW outcomes look better than they really are because TTW participants represent a more motivated subpopulation among beneficiaries. We can adjust for this bias by limiting the analysis to DI beneficiaries who have completed a TWP. That group is not representative of all beneficiaries; it consists only of those who received DI benefits (either with or without concurrent SSI payments) and worked enough to earn at the trial work level (in 2016, at least $810 per month) for 9 or more months within a span of 12 consecutive months. The TWP duration and earnings criteria remove much of the bias in Table 6 in that they require the willingness and the ability to work at a substantial level, regardless of whether the beneficiary seeks employment services. However, a serious interest in work is not the only potential bias related to service use. Those who receive services presumably believe they need them. To the extent they are correct, one might expect these beneficiaries, in the absence of services, to have poorer employment outcomes than beneficiaries who forgo services, all else being equal. So, although we adjust for motivation toward work by restricting the analysis to beneficiaries who complete a TWP, the negative bias of service users needing those services likely remains.

Using TWP completion to control for work effort and aptitude narrows our examination to a subset of beneficiaries; therefore, any results for that group cannot be extrapolated for the larger group. Nevertheless, restricting to those with TWP completion allows us to intuitively account for bias and reveal more accurate net effects of TTW for at least this subset of beneficiaries. Table 7 shows the average BFW and costs for beneficiaries who were eligible for TTW services in 2006 and subsequently completed a TWP. Of the beneficiaries who never enrolled in TTW, only 3.0 percent (333,659) ever completed a TWP; and while only 3.2 percent of TTW nonenrollees overall ever attained STW status, 48.8 percent of those who completed a TWP did. Average net BFW also differed dramatically, at $1,446 for TTW nonparticipants overall (Table 6) and $27,387 for those who eventually completed a TWP (Table 7). Controlling for TWP completion also changes the STW-status and BFW numbers for TTW participants, although by significantly smaller proportions. Comparing Tables 4 and 7 shows that STW status increases from 18.7 percent for SVRA participants overall to 50.2 percent for those who completed a TWP, and from 28.6 percent to 60.4 percent for EN participants. Among those who completed a TWP, average gross BFW for SVRA clients are $17,622 and for EN clients they are $31,812. Although the average gross BFW for all SVRA clients in Table 7 are lower than those for TTW nonparticipants, the average gross BFW for TTW participants whose service providers received SSA payments are much higher than those for nonparticipants ($43,742 for SVRA clients and $53,410 for EN clients). Accounting for TTW costs drops the average net BFW further below the level for nonparticipants (who had no service costs) for both SVRAs and ENs overall, but for cases that generated SSA payments for services, the average net BFW still exceed those for nonparticipants.

Table 7. TTW participation in 2006 and employment outcomes for beneficiaries who completed a TWP as of 2016
TTW participation and outcome Number Attained STW status by December 31, 2016 (%) a Per-beneficiary BFW and costs through December 31, 2016 (2016 $)
Gross BFW Administrative costs b Payments to service providers Net BFW
Never enrolled in TTW (through 2016) 333,659 48.8 27,387 0 0 27,387
Enrolled in SVRA services 23,431 50.2 17,622 48 2,457 15,117
SVRA received provider payment by year-end 2016?
No 19,400 43.5 12,195 48 0 12,147
Yes c 4,031 82.7 43,742 48 14,285 29,409
Enrolled in EN services 1,303 60.4 31,812 2,863 5,837 23,112
EN received provider payment by year-end 2016?
No 621 45.6 8,094 2,863 0 5,231
Yes c 682 73.9 53,410 2,863 11,152 39,395
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTES: Includes DI and DI/SSI beneficiaries who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who had entered STW status before 2006, or who enrolled in an SVRA or an EN before or after 2006.
BFW and costs for TTW participants exclude those resulting from any subsequent ticket assignments.
a. Applies to individuals who received neither DI benefits nor SSI payments in at least 1 month because of work.
b. 2017 costs adjusted to 2016 dollars.
c. Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.

Table 7 may also indicate the effect of differing levels of need for services in the difference between outcomes for ENs, which can choose whom they serve, and SVRAs, which cannot. Thus, the poorer outcomes for SVRAs could reflect a clientele that not only needs services to succeed, but also needs more intensive services than those provided by ENs. These findings suggest that, even when we restrict our analysis to beneficiaries who completed their TWP, significant differences remain between those who do and do not receive services, and between those who receive services from ENs versus SVRAs.

Table 8 compares the characteristics of DI beneficiaries who completed a TWP for each of the three types of employment service use (none, SVRAEN). EN clients were the quickest to start working and had higher average initial earnings than SVRA clients. Although it took SVRA clients longer than EN clients to start work after assigning their tickets, the median time differed by only 6 months.

Table 8. Characteristics of DI beneficiaries on the rolls in 2006 who completed a TWP by December 2016, by type of employment service received in 2006
Characteristic None SVRA EN
Number of beneficiaries 333,659 23,431 1,303
Average age 43.9 37.9 42.6
Average monthly benefit (2016 $) 985 908 979
Benefit type (%)
DI only 85.7 82.6 91.1
DI and SSI concurrently 14.3 17.4 8.9
Sex (%)
Men 48.2 53.9 47.0
Women 51.8 46.1 53.0
Impairment type (%)
Musculoskeletal system disorder 25.2 11.8 19.3
Psychiatric disorder 33.2 39.8 44.3
Intellectual disability 6.1 11.8 5.4
Sensory system disorder 2.0 13.7 4.1
Organ systems disorder 15.1 8.0 10.7
Nervous system disorder 5.3 5.8 5.7
Injuries 4.1 4.6 2.8
Blood disorders or infectious diseases 3.4 1.8 2.9
Neoplasms 3.9 1.0 3.1
Congenital anomalies 0.2 0.5 0.2
Skin disorders 0.3 0.1 0.2
Unknown 0.1 0.1 (X)
Other 1.3 0.8 1.5
Most recent work before starting benefits (%)
Did not work 4.4 5.9 2.7
2 or fewer years 25.4 20.0 23.9
3–4 years 2.8 1.9 1.8
5 or more years 1.0 0.7 1.2
Average number of months from disability onset to December 2006 93.4 98.2 102.0
As of December 2016, percentage who died a 8.8 4.8 8.9
Of those with any earnings as of December 2016—
Total earnings (thousands of 2016 $) 108,738 103,347 134,896
Median number of months to first earnings
From January 2006 21 13 9
From ticket assignment . . . 9 3
Median first-month earnings (2016 $) 582 493 573
Average number of months in STW status 18.9 18.6 26.7
SOURCE: Authors' calculations using 2016 DAF.
NOTES: Includes DI and DI/SSI beneficiaries who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who had entered STW status before 2006, or who enrolled in an SVRA or an EN before or after 2006.
Rounded components of percentage distributions do not necessarily sum to 100.0.
(X) = suppressed to avoid disclosing information about particular individuals; . . . = not applicable.
a. Includes only deaths occurring while receiving disability benefits or in STW status. Omits deaths occurring after conversion to retirement benefits or while benefits are suspended or terminated for reasons other than work.

Relative to the other service-type groups, service nonusers were significantly more likely to have musculoskeletal- or organ-system impairments, SVRA clients were significantly more likely to have intellectual or sensory-system disabilities or to receive concurrent DI and SSI benefits, and EN clients were less likely to have disabilities from injuries or to have no work experience prior to starting benefits. SVRA clients were also about half as likely to die during the observation period as were those in the other service-type groups, which likely reflects the younger average age of SVRA users. Consistent with Table 7, which shows that EN clients tend to attain higher BFW and are more likely to reach STW status than nonparticipants and SVRA users, Table 8 shows that EN clients also maintain STW status for a longer period.

Multivariate Analyses

Differences in beneficiary characteristics are likely to account for the differences in TTW outcomes that remain after controlling for TWP completion. We can account for these differences by using the following simple ordinary least square (OLS) multivariate regression:

(1)
Y=α+βX+δT+γε

where:

Y is our outcome,

X is our vector of observable characteristics,

T is a dummy indicating whether the beneficiary received TTW services, and

ε is our error term.

Table 9 shows the regression results for total earnings, STW status attainment and duration, and gross BFW for DI beneficiaries who completed a TWP, controlling for the observable differences.

Table 9. OLS regressions relating DI beneficiary characteristics and employment service use to selected positive employment outcomes: Beneficiaries on the rolls in 2006 who completed a TWP by December 2016
Characteristic Aggregate earnings (thousands of 2016 $) Attained STW status (%) Months in STW status Per-beneficiary gross BFW (2016 $)
Estimate Standard error Estimate Standard error Estimate Standard error Estimate Standard error
Intercept 109,256*** 1,518 0.854*** 0.005 40.91*** 0.29 40,971*** 450
Per year of age -1,882*** 29 -0.009*** 0.000 -0.62*** 0.01 -973*** 8
Per dollar of average monthly benefit 93*** 1 0.000*** 0.000 0.01*** 0.00 31*** 0
Benefit type (%)
DI only (reference category) . . . . . . . . . . . . . . . . . . . . . . . .
DI and SSI concurrently -42,116*** 812 -0.094*** 0.002 -8.76*** 0.15 -13,631*** 241
Sex (%)
Men 4,381*** 575 0.030*** 0.002 0.96*** 0.11 1,206*** 170
Women (reference category) . . . . . . . . . . . . . . . . . . . . . . . .
Impairment type (%)
Musculoskeletal system disorder (reference category) . . . . . . . . . . . . . . . . . . . . . . . .
Psychiatric disorder -11,561*** 773 -0.022*** 0.002 -1.83*** 0.15 -2,694*** 229
Intellectual disability -24,138*** 1,326 -0.138*** 0.004 -10.11*** 0.25 -10,077*** 393
Sensory system disorder 3,625** 1,788 -0.089*** 0.005 -3.41*** 0.34 -5,308*** 530
Organ systems disorder -2,418*** 914 -0.001 0.003 0.54*** 0.17 543** 271
Nervous system disorder -2,832** 1,330 -0.040*** 0.004 -1.06*** 0.25 -2,020*** 394
Injuries 30,569*** 1,483 0.037*** 0.004 6.07*** 0.28 9,450*** 440
Blood disorders or infectious diseases 11,992*** 1,635 0.049*** 0.005 4.00*** 0.31 3,797*** 485
Neoplasms 82,693*** 1,556 0.148*** 0.005 16.23*** 0.29 27,444*** 461
Congenital anomalies -4,070 5,641 -0.070*** 0.017 -4.73*** 1.06 -8,197*** 1,671
Skin disorders 3,977 5,445 -0.017 0.016 1.02 1.03 309 1,613
Unknown -491 11,916 -0.010 0.035 -0.66 2.25 1,293 3,532
Other 13,642*** 2,547 -0.013* 0.008 1.57*** 0.48 3,363*** 755
Most recent work before starting benefits (%)
Did not work (reference category) . . . . . . . . . . . . . . . . . . . . . . . .
2 or fewer years -7,506*** 645 0.009*** 0.002 0.38*** 0.12 -507*** 191
3–4 years -11,419*** 1,687 -0.009* 0.005 -0.79** 0.32 -2,188*** 500
5 or more years -22,143*** 2,745 -0.044*** 0.008 -2.50*** 0.52 -3,407*** 814
Per month from disability onset to December 2006 -27*** 3 0.000*** 0.000 0.00 0.00 0 1
TTW participation
Did not participate (reference category) . . . . . . . . . . . . . . . . . . . . . . . .
Assigned ticket to SVRA -4,808*** 1,144 -0.008** 0.003 -1.62*** 0.22 -10,601*** 339
Assigned ticket to EN 24,214*** 4,578 0.107*** 0.014 7.07*** 0.86 3,381** 1,357
R2 0.08 0.04 0.06 0.01
SOURCE: Authors' calculations using 2016 DAF.
NOTES: Includes DI and DI/SSI beneficiaries who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who had entered STW status before 2006, or who enrolled in an SVRA or an EN before or after 2006.
. . . = not applicable; * = statistically significant at the 90 percent level; ** = statistically significant at the 95 percent level; *** = statistically significant at the 99 percent level.

Given the large dataset (358,393 beneficiaries), it is not surprising that many of the parameter estimates are statistically significant. We therefore focus on those that are both significant and of greater magnitude. Across all four models, intellectual disability is significantly negatively associated with positive employment outcomes, while neoplasms have a significant positive relationship. Disability because of injury has significant positive effects on earnings, months in STW status, and BFW, and a modest positive effect on attaining STW status. Concurrent-benefit status is negatively associated with employment outcomes in all models but the magnitude of the effect on attaining STW status is small.

Longer periods between last employment and first benefit are negatively associated with earnings and BFW, although only the earnings magnitudes are large. Disabled-worker beneficiaries must by definition have work histories, so DI beneficiaries without prior earnings must qualify either as disabled widow(er)s or disabled adult children with eligibility based on the work history of a spouse or parent who is disabled or has retired or died. These results suggest that, among beneficiaries who completed a TWP, surviving spouses and adult children tend to have the highest earnings; but among those with work histories, recency of work is associated with higher earnings. For the two STW-status regressions, the signs in the results are more in line with expectations in that those who worked within 2 years of benefit commencement had positive STW-status outcomes, but the magnitudes are small. Further, among TWP completers, both age and sex have limited effects on outcomes, although results are consistent with other research, as employment success declines with age and men are more likely than women to attain employment.

After accounting for these characteristics, our results are roughly consistent with those of the simple outcome analysis in Table 7. Table 9 also shows that those who received EN services attained $24,214 more in total earnings than those who received no services. Further, they were 10.7 percent more likely to reach STW status, accrued 7.07 more months in STW status, and accounted for $3,381 more in gross BFW through the end of 2016 (versus the $4,425 difference indicated in Table 7). Relative to those who did not receive employment services, SVRA clients had lower total earnings, attained and retained STW status at about the same levels, and accounted for $10,601 less in gross BFW (compared with the $9,765 difference indicated in Table 7).

We can also use TWP completion status to model gross BFW on the full DI population using a Heckman two-stage selection model. The Heckman model was developed to produce regression estimates—for example, of wages—for an entire population when input data are available for only a subset of the population—for example, those who work (Heckman 1974, 1976). Without such a model, any estimate based on only the wages of workers would be biased and tend to overstate wages for nonworkers. Whether individuals work is related to wage rates: They will work only if they can earn above a particular amount. The Heckman model uses information on the work decision (which, unlike that for wages, is available for everyone) to adjust the wage equation for those who do work. It solves the bias problem by estimating a first-stage equation on the probability of work (that is, the probability of having wages) based on characteristics that drive the work decision. From this equation, one can calculate the inverse Mills ratio, λ, which provides a measure of the selection bias driving whether an individual works, and thus has earnings, for the wage equation. By including λ in the second-stage wage equation, the Heckman model can produce unbiased wage estimates for the entire population.

Although Heckman developed this technique in the context of wages and work, it can apply to other situations where outcome data are only available for a subpopulation that is defined by a participation decision. For our purposes, we can use it to estimate dollar outcomes (BFW) that are subject to exceeding a threshold value.

We use the following model to estimate BFW:

(2)
Y=α+βX+δT+γA+ρN+ε

where:

Y is the BFW outcome,

X is the vector of observable characteristics,

T is a dummy indicating whether the beneficiary receives TTW services,

A is aptitude for self-supporting work, and

N is need for assistance in attaining self-supporting work.

A and N are unobservable, yet they determine whether we have data on Y. If we run our analysis on the full population and do not account for these unobservable variables, we will have biased results. In the regressions in Table 9, we adjusted for A by including only those who had completed a TWP. Under the Heckman model, we include all DI beneficiaries and modify equation 2 as follows:

(3)
Y=α+βX+δT+ρN+μ
(4)
B= 1, if γX+υ00, otherwise

We drop the unobservable quantity A from equation 3 and add equation 4, the selection equation, with B as the threshold dummy variable indicating whether we see outcome Y. For our analysis, only a person with sufficient aptitude and desire for self-supporting work generates BFW; that is, when γX+υ0. We substitute T, the completion of the TWP, for B as our selection variable. Both B and T are indicators of self-supporting aptitude; but whereas B indicates that self-supporting work has occurred, T indicates the broader condition that self-supporting work is possible and the beneficiary is sufficiently motivated to attain it.29 Another advantage of using TWP completion is that it is an SSA program variable (meaning that administrative data track the count precisely) while BFW is a constructed variable, constituting an estimate generated from an algorithm (Mathematica 2022).

Also, note that our selection equation only addresses selection that is due to aptitude for self-supporting work. The variable N, the need for assistance to get to self-supporting work, remains unobservable because our Heckman model does not have a proxy for it, so it is omitted and our results will retain some bias. Note also that A and N work in opposite directions: omitting A will tend to overstate the effect of TTW services, δ, and omitting N will tend to understate δ.

Equation 4 is our first-stage selection equation on the probability of completing the TWP, and equation 3 is our second-stage estimate of BFW, adjusted for those who do complete their TWP and thus are unlikely to have nonzero BFW. Notice that we use the vector of independent variables,X, in the selection equation rather than X. To implement the Heckman approach properly, we need exclusion restrictions both to minimize correlation between λ and the vector of independent variables and to facilitate model identification.30 To do so, we want to include independent variables in stage one that predict selection into the TWP-completion group but do not predict the BFW, at least not directly. Although equations 3 and 4 include many of the same independent variables, only the selection equation includes whether the beneficiary died before the end of the observation period. This variable is meant to proxy for health (as distinct from disability) in that it seems likely that many beneficiaries with short life horizons will be aware of that likelihood and will be less likely to pursue work at substantial levels as a result. We also exclude the two variables that, based on Table 9, are related more to earnings (stage 2) than to the decision to work at a substantial level (stage 1): most recent work before benefits and benefit type (DI only versus concurrent DI/SSI). The Heckman model results are shown in Table 10.

Table 10. Heckman selection model regressions relating beneficiary characteristics and employment service use to selected positive employment outcomes: Beneficiaries on the rolls in 2006 with outcomes as of December 2016
Characteristic Stage 1: Probit regression on TWP completion Stage 2: OLS regression on gross BFW (including inverse Mills ratio from stage 1)
Estimate Standard error Estimate Standard error
Intercept 0.046*** 0.004 65,938*** 650
Per year of age -0.030*** 0.000 147*** 22
Per dollar of average monthly benefit . . . . . . 31*** 0
Benefit type
DI only (reference category) . . . . . . . . . . . .
DI and SSI concurrently . . . . . . -13,509*** 241
Sex
Men -0.048*** 0.002 3,550*** 185
Women (reference category) . . . . . . . . . . . .
Impairment type
Musculoskeletal system disorder (reference category) . . . . . . . . . . . .
Psychiatric disorder -0.017*** 0.002 -1,773*** 243
Intellectual disability -0.331*** 0.004 2,027*** 469
Sensory system disorder 0.079*** 0.005 -7,824*** 565
Organ systems disorder 0.011*** 0.003 2,366*** 287
Nervous system disorder -0.166*** 0.004 4,338*** 430
Injuries 0.007* 0.004 9,409*** 465
Blood disorders or infectious diseases 0.276*** 0.005 -4,693*** 543
Neoplasms 0.227*** 0.005 23,588*** 494
Congenital anomalies -0.413*** 0.017 7,434*** 1,785
Skin disorders 0.018 0.017 147 1,715
Unknown -0.157*** 0.033 6,776* 3,704
Other -0.151*** 0.007 9,370*** 795
Most recent work before starting benefits
Did not work (reference category) . . . . . . . . . . . .
2 or fewer years . . . . . . -704*** 190
3–4 years . . . . . . -2,509*** 496
5 or more years . . . . . . -3,672*** 807
Per month from disability onset to December 2006 -0.002*** 0.000 56*** 1
Died by December 2016 -0.476*** 0.003 . . . . . .
TTW participation
Did not participate (reference category) . . . . . . . . . . . .
Assigned ticket to SVRA . . . . . . -10,692*** 339
Assigned ticket to EN . . . . . . 3,544** 1,351
λ . . . . . . -42,374*** 754
R2 . . . 0.11
SOURCE: Authors' calculations using 2016 DAF.
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who had entered STW status before 2006, or who enrolled in an SVRA or an EN before or after 2006.
. . . = not applicable; * = statistically significant at the 90 percent level; ** = statistically significant at the 95 percent level; *** = statistically significant at the 99 percent level.

In the first-stage (probit) regression, congenital anomalies and intellectual disability have strong negative correlations, relative to musculoskeletal system impairments, with completing a TWP. The same is true, to a lesser degree, for nervous system impairments. Not surprisingly, death before the end of the observation period is also strongly and negatively associated with TWP completion. Alternatively, blood disorders/infectious diseases and neoplasms are positively, and strongly, associated with TWP completion. Being male, older, and with longer time on the rolls are all negatively associated with TWP completion, but the effects are relatively small.

In the second-stage (OLS) regression, recipients of concurrent DI/SSI payments have significantly lower gross BFW than DI-only beneficiaries, by $13,509 on average. Relative to musculoskeletal impairments, sensory system disorders and blood disorders/infectious diseases all significantly reduce gross BFW. Injuries and congenital anomalies are both associated with higher BFW relative to musculoskeletal impairments, but neoplasms are associated with the highest relative increase in BFW at $23,588. Higher monthly benefit amounts are also associated with higher BFW: An increase of just $100 in the monthly benefit is associated with a $3,100 increase in BFW. Given that DI benefits generally vary between $1,000 and $2,000 per month, this effect can be substantial. Not surprisingly, men are more likely to attain higher BFW levels than women, but the relative difference is modest ($3,550). Similarly, not having worked since 5 or more years before starting benefits has the expected negative effect, but again it is modest relative to those with no prior work ($3,672).

The Heckman model results of greatest interest are the effects of SVRA and EN services. EN service use increased gross BFW over the period by $3,544 per beneficiary (relative to TTW nonparticipants) while the effect of SVRA service use under TTW was to reduce BFW over the period by $10,692. Both estimates are very close to the Table 9 OLS regressions on BFW ($3,381 and −$10,601, respectively). The parameter estimate for the inverse Mills ratio (λ) in the Heckman model is significant and, as expected, negative. The magnitudes of TTW effects estimated with the Heckman model are much smaller than those that would have resulted from running the regression without any selection adjustment for serious interest in work.31

We also find that the simple estimations of effects (Table 7) are close to the net effects resulting from the Heckman regression controlling for seriousness toward work and observable characteristics. In Table 10, those who received EN services attained $3,544 more on average in gross BFW through the end of 2016 than nonparticipants (compared with $4,425 more in Table 7), and those who received SVRA services attained $10,692 less in gross BFW per beneficiary than those with no services (compared with $9,765 less in Table 7). The simple analysis overstates the effect of TTW, but not by much.

The costs shown in Table 7 are for beneficiaries completing a TWP. However, the Heckman estimate is for all recipients of DI or concurrent DI/SSI benefits regardless of TWP status. For that larger group, the TTW costs were $5,680 per EN client and $854 per SVRA client (not shown). Combining those figures with the gross BFW results in Table 10 suggests that neither EN nor SVRA services reduced SSA's costs through BFW in the years following a beneficiary's TTW participation. For EN clients, BFW through the end of 2016 recouped about two-thirds of costs and for SVRA clients, with negative net BFW, per participant costs sum to $11,546.32

However, as noted above, this is not the full story. Our Heckman model adjusts for serious interest in work, but it does not adjust for the fact that many who seek employment services do so because they think those services are necessary for them to succeed. If this self-assessed need is reasonably accurate, our regression—which ignores that assessment—will underestimate the effects of service use. The fact that SVRA service use has a negative, large, and significant effect for beneficiaries who are serious about work and complete their TWP clearly seems to indicate that SVRA clients may have greater barriers to employment success than other TWP completers do.

Also noteworthy is that the stage 2 regression model has a low R2 value, indicating that much of what drives these effects is not accounted for. This, in turn, means that even after limiting our analysis to those who have completed their TWP, significant quantities of information remain unobserved in the model. At least part of this missing information is likely due to the service needs of those who enroll in employment service programs.

Charts 2A and 2B track the earnings and employment outcomes of 2001 DI awardees to further examine potential bias related to beneficiaries' employment-service need. We shift the study cohort from 2006 beneficiaries to 2001 awardees to provide enough lead time for beneficiaries to have sorted themselves into the three types of employment service use: none, EN, and SVRA. We define the user groups by their status as of 2006, thus allowing beneficiaries 5 years from benefit start to decide whether to seek services, followed by a 10-year observation period that lets us track and compare their earnings and employment outcomes. To provide visual comparisons, Charts 2A and 2B also track earnings and employment outcomes for service nonusers who did not work until after 2006. However, our discussion focuses only on EN users, SVRA users, and service nonusers who worked as of 2006.

Line chart with tabular version below.
Show as table
Table equivalent for Chart 2A. Average earnings of 2001 DI awardees among those with earnings as of 2006 or later, by type of employment service use as of 2006, 2001–2016 (in thousands of 2016 dollars)
Year EN SVRA None (working by 2006) None (not working by 2006)
2001 7.725 5.583 8.916 . . .
2002 7.770 5.493 7.886 . . .
2003 6.732 6.460 9.021 . . .
2004 8.706 7.453 9.679 . . .
2005 10.459 8.322 9.981 . . .
2006 11.668 9.269 10.442 . . .
2007 13.352 10.469 11.611 4.398
2008 13.595 11.132 12.127 6.240
2009 14.552 11.868 12.764 7.750
2010 14.811 11.935 12.758 7.853
2011 14.883 12.133 12.955 8.238
2012 15.087 12.436 13.397 8.794
2013 14.460 12.789 14.009 9.192
2014 14.363 13.159 14.590 9.510
2015 14.639 14.011 15.450 10.257
2016 15.571 14.497 15.900 10.618
SOURCES: Authors' calculations using 2016 DAF; SSA (2020, 22).
Annualized SGA level = $13,560. Annualized TWP level = $9,720.
. . . = not applicable.
Line chart with tabular version below.
Show as table
Table equivalent for Chart 2B. Employment rate among 2001 DI awardees, by type of employment service use as of 2006, 2001–2016 (in percent)
Year EN SVRA None (working in 2006) None (not working in 2006)
2001 46.8 49.6 73.6 . . .
2002 32.3 39.6 36.1 . . .
2003 40.8 40.8 32.9 . . .
2004 49.0 42.7 32.8 . . .
2005 53.5 43.9 32.6 . . .
2006 56.8 44.0 32.8 . . .
2007 52.8 43.5 30.5 2.2
2008 48.4 41.4 28.6 2.9
2009 40.2 36.8 25.6 2.8
2010 38.1 35.1 24.6 3.2
2011 35.6 33.9 24.0 3.3
2012 33.7 33.4 23.8 3.5
2013 33.3 33.2 23.6 3.7
2014 33.4 33.5 23.6 4.1
2015 34.0 33.9 24.4 4.4
2016 32.8 34.6 25.0 4.9
SOURCES: Authors' calculations using 2016 DAF; SSA (2020, 21).
NOTE: Some beneficiaries who had not worked as of 2006 attained work in later years.
. . . = not applicable.

The charts cover 2001–2016; but because some 2001 awardees had prebenefit earnings in 2001, our first year of interest is 2002. As might be expected, beneficiaries who worked and did not seek services in the first 5 years on the benefit rolls had the highest average earnings in 2002, slightly above those of EN clients and substantially higher than those of SVRA clients (Chart 2A). For service nonusers who worked, earnings increased steadily over the observation period, and they had the highest average earnings, by a small margin, in the last few observation years. Although service users started the period with lower average earnings, EN clients reached and sustained higher earnings over time. SVRA clients started with average earnings well below the other groups and, over time, they reduced the earnings gap with service nonusers who worked, but did not equal them. By 2005, earnings for EN clients had surpassed those of service nonusers who worked, and by 2008, EN users' average earnings eclipsed the annualized SGA level. SVRA clients had closed about half the earnings gap with service nonusers who worked by 2006 and although they didn't further close the gap thereafter, their average earnings continued to climb, finally edging above SGA level by 2015.

All three of the service-use groups that Chart 2B tracks before 2007 had similarly low proportions of beneficiaries working in 2002. For service nonusers who worked, the rate was 36 percent in 2002 and declined thereafter. That downward trend likely reflects the workers who were unable to sustain employment, and as they dropped out, the culling effect likely played a significant part in the increase in earnings for service nonusers who work (Chart 2A), in that the beneficiaries who continued to work were pared down over time to only the highest earners in the group. Among EN users, only 32 percent worked in 2002 (Chart 2B), but the rate rose steadily and significantly through 2006. SVRA users had a higher employment rate in 2002 (41 percent) and the rate rose steadily, but less sharply than that of EN users, through 2006. More importantly, both the EN and SVRA users had earnings and employment-rate increases: Each year from 2003 to 2006, more enrollees worked, and they had higher earnings. Although the employment rates declined after 2006, they were, for both groups, about 10 percentage points higher in 2016 than the rate for service nonusers who had worked as of 2006.

Although the evidence is not definitive, Charts 2A and 2B support the hypothesis that beneficiaries who seek EN and (especially) SVRA services do so because they believe that they need those services, without which, such beneficiaries would be expected to have poorer employment outcomes than those who never seek services. The charts also suggest that EN services can fill clients' needs and SVRA services can at least ameliorate them.

Limitations

This analysis focuses on the costs and effects of the TTW program and does not attempt to identify or account for any costs and effects related to other provisions of the authorizing legislation. Examples of the latter include costs associated with the elimination of work activity as a basis for reviewing an individual's disabled status, expedited reinstatement of benefits after a failed work attempt, outreach and support for the TTW Work Incentives Planning and Assistance initiative, and extended access to medical benefits, along with the previously mentioned protection of TTW participants from CDRs and the exclusion of unrecovered overpayments. Some of these costs are not specific to TTW and could be modified or eliminated irrespective of the program. As such, we treated them as general SSA costs that apply to all beneficiaries who work. Similarly, we did not account for computer system development costs that were related to or prioritized by TTW implementation, other than those included in the TPM contract. Although we do not attempt to account for these general SSA costs, we acknowledge that some of them may be used disproportionately for TTW participants.

We have reliable data on costs for services associated with beneficiaries who assigned their tickets in 2006 but we do not have 2006 data on administrative costs. We therefore substituted 2017 administrative costs adjusted to 2016 dollars. This substitution assumes that annual TTW administrative costs have been flat since 2006, which is not accurate; but we think it is reasonably close after we remove the nonsteady-state costs associated with TTW startup. To the extent that our estimates are lower than the true costs, program effects will be overstated.

Although our Heckman analysis adjusted for selection bias based on work aptitude and interest, it had no adjustment for service need. The negative need-for-service bias is likely to be substantial based on three findings in the analysis. First, we found that SVRA participation had a large, negative, and statistically significant effect on BFW. This defies common sense. Although it seems possible that helping someone build skills and find work has no discernible effect, it seems unlikely that a beneficiary who receives such services is worse off than one who receives no services at all. Perhaps some SVRA clients delayed a work start without getting anything useful from the SVRA during the delay. However, given that average gross BFW for SVRA clients are fully one-third less than those for service nonusers, and the median time from ticket assignment to work start is only 6 months longer for SVRA clients than for EN clients, it is hard to believe that the difference is due only to an employment delay caused by SVRA participation.

Second, we cannot determine how much of the difference between SVRA and EN outcomes is attributable to the success of the new service models under TTW and how much is due to bias from client self-sorting, based on need and ability, into ENs or SVRAs. We know from the contractor's evaluation that EN and SVRA services differ (Schimmel and others 2013) but we also know that ENs can choose which beneficiaries they serve while SVRAs cannot. Given that EN payments follow a fixed payment structure while SVRA payments use a reimbursement model, it seems likely that the beneficiaries with the greatest need end up at SVRAs. We cannot quantify differences in service need among TTW participants, but those unmeasured differences in client needs and abilities are likely to be the primary factors driving the differences between EN and SVRA outcomes. This seems evident in the earnings profiles of beneficiaries in the years after benefits begin, as those who seek SVRA services start out with significantly lower average earnings than either EN clients or service nonusers. Although SVRA clients' earnings improve significantly over time, and they narrow the gap with service nonusers, they remain the group with the lowest average earnings across the entire study period.

The selection of beneficiaries based on service need also suggests a limitation of the Heckman model for this analysis. The model assumes we can capture bias through the first-stage regression, but this works only if a single threshold variable can be used to capture the bias entirely. That seems unlikely in this instance, where the bias moves in two directions: increased aptitude for self-supporting work makes success more likely, while the need for assistance makes it less likely.

We have not attempted to build a comprehensive model of the factors affecting a beneficiary's likelihood of returning to work or leaving the disability benefit rolls. Instead, our analysis uses the available administrative data from the DAF on variables that are thought to be potential correlates with this behavior. Yet SSA data do not track several characteristics that could illuminate the analysis. These missing beneficiary characteristics include educational attainment, job characteristics prior to benefit start (such as industry, occupation, and job tenure), the stage at which benefits were awarded (initial determination, reconsideration, appeal), residual functional capacity, proximity to services, and access to transportation. Because SSA introduced an automated application processing system called the Electronic Disability (eDib) System, some of this information is available for beneficiaries applying from 2008 forward, but there are concerns about the completeness of the eDib data. In attempting to use eDib data for this analysis we encountered differences between eDib and DAF data, the causes of which we could not determine. Although we did not use eDib information in our analysis, its breadth remains an attractive extension for future study.

Another limitation is that our Heckman model excludes SSI recipients who do not receive concurrent DI benefits. This is because we use TWP completion as our threshold variable, and the TWP does not apply to SSI-only recipients. An alternative would be to use the STW-status variable, which applies to both SSI and DI and is conceptually similar to TWP completion. As noted earlier, however, we chose TWP completion because it is a program variable while the STW-status variable is constructed for the DAF. In one sense, STW status is a better choice in the Heckman setup: If one's benefits are suspended for work, BFW must be nonzero, in the same way that if work occurs, earnings are nonzero under the Heckman theoretical framework (Killingsworth 1983). However, STW status is a higher threshold than TWP completion for DI beneficiaries and a lower one for SSI. For DI, STW status implies TWP completion and at least 1 month of SGA after the end of the TWP. For SSI, STW status does not necessarily indicate a sustained work effort because it can occur for any month in which earnings are approximately double the SSI payment amount.33 Further analysis of the use of STW status as the threshold variable is worth examining in future research.

We made several assumptions in estimating TTW administrative costs. As noted in Appendix A, those costs are spread out over several years, so we approximate a single year's costs associated with the enrollee cohort. Although costs have changed over time, we use 2017 as a representative year for administrative costs because it is the best available estimate and represents a year in which TTW was operationally mature. It is likely that actual costs for 2006 were higher, but we think that many of those costs are attributable to TTW's startup status. We do not think the range of possible steady-state cost values for 2006 would materially affect our basic findings.34

Summary

The TTW program was implemented with high hopes that it would significantly broaden access to employment services, efficiently fund those services, reduce dependency on cash benefits, and increase the proportion of beneficiaries who are able to exit the disability benefit rolls and live self-sufficiently, thereby generating savings to the Social Security trust funds (for DI) and the general fund of the treasury (for SSI).

Although some observers regard TTW as a failure, that conclusion is based narrowly on findings that TTW did not substantially increase disability program exits. Broader evidence shows that TTW accomplished other goals. It expanded the service options available to SSA disability program beneficiaries and increased enrollment in employment services overall. There is also evidence that TTW participation, for both EN and SVRA clients, has yielded positive earnings results, improving lives and reducing dependency on cash benefits. In addition, long-term net-outcomes analysis shows substantial BFW for TTW participants.

Finding that the TTW's effects were not as substantial as its designers had hoped does not mean that new EN services added under the program had failed. Because TTW superseded a preexisting SSA program to provide VR services, Mathematica's evaluation compared TTW's effects with those of that prior program—it did not assess the TTW effects relative to a no-services counterfactual. The evaluation found that TTW expanded service provision and participation without the decline in disability program exit rates that might have been expected as more beneficiaries (perhaps with poorer work prospects) enrolled for services. This means that services provided by ENs, combined with those offered by the preexisting SVRAs, were no less effective than those that were previously available to fewer potential clients, presumably with better median employment prospects, from SVRAs alone. Also of note: Although Mathematica did not find that TTW had a large effect on the disability roll exit rate, a smaller effect, not measurable with the methods available, may have occurred. Specifically, TTW increased STW status by less than 30 percent and the number of months in STW status by less than 9 percent—but the evaluation could not determine, with the available data, how closely the actual figures approached those ceilings.

Further, separate models assessing EN and SVRA services found better outcomes for ENs. TTW's introduction of ENs thus expanded service options and availability and reshuffled the service landscape while achieving the same benefit exit rate as the prior SVRA-only system. With increased service availability, beneficiaries were able to choose the providers that best met their needs.

The Mathematica evaluation examined TTW's marginal effect, but not the overall effect of the employment services. We therefore examined the net outcomes—program-associated costs and savings—for all beneficiaries. Analyzing program outcomes, without adjustments to account for selection biases, we found that TTW participants using either EN or SVRA services had much better employment results and, on average, accrued much higher BFW than nonparticipants. Then, adjusting for bias related to work interest and capability among employment service users, we limited the analysis to DI beneficiaries who had completed a TWP. Among this work-motivated group, we found that EN and SVRA clients accrued high average gross BFW, but average net BFW, after adjusting for average service costs, were lower for TTW service users than for TWP completers who received no services.

We next examined the effect of TTW services using more complex regression analyses to account for differences between service users and nonusers. Our Heckman model found that TTW employment service use, relative to nonuse of any services, increased gross BFW per beneficiary by $3,544 for EN users and reduced per-beneficiary gross BFW by $10,692 for SVRA users over the 2006–2016 study period. We also found that the simple adjusted outcomes, when restricted to DI beneficiaries who were TWP completers, were close to the more complex regression results. This is a useful finding: Because the simple model is intuitive and easy to explain, and provides similar results, it serves as a proxy for the more accurate and complex econometric models and can illustrate the results to a broader audience.

Conclusions

Although this analysis clarifies TTW's effects, significant questions remain. We show that simple outcome measures can be misleading because they do not account for the biases inherent in who does and does not seek services. Restricting the analysis to DI beneficiaries who completed a TWP and applying the Heckman model both seem to work reasonably well in adjusting for serious interest in work. However, neither of these approaches account for bias related to the differences in service needs between service users and nonusers, so the actual effects are likely greater.

By adjusting for interest in work but not for self-assessed need of services, our estimates appear to represent a lower bound for TTW effects. Unfortunately, this finding does not address whether savings from benefit reductions cover TTW costs: accounting for the beneficiary's need for service would likely increase the estimated benefits of TTW, and would likely erase the apparent negative outcomes of SVRA services, but perhaps not enough to fully offset EN and SVRA service costs. The absence of clear findings after nearly 2 decades of observation, evaluation, and analysis underscores the difficulty in estimating definitive effects without a well-designed randomized control trial. The quasiexperimental methods used thus far simply do not seem capable of capturing the complex biases inherent in the return-to-work decisions facing beneficiaries with disabilities.

Still, our analysis finds that EN services, for which service need is less likely to be a limiting factor, significantly boost client earnings, STW status, and BFW for DI beneficiaries. This, along with Mathematica's finding that the combined employment outcomes for EN and SVRA services under TTW were similar to the pre-TTW SVRA-only outcomes, suggests that SVRAs are likely more successful than the Heckman analysis implies: DI beneficiaries who use EN services would have been as successful under the old system as they are under TTW. If this were not the case, then overall employment success would have improved when SSA switched from SVRA services only to the hybrid SVRA/EN model under TTW. It may be that pre-TTW SVRAs succeeded only with clients who have self-sorted into using ENs under TTW, but that seems unlikely. The likelier scenario is that SVRAs have a range of successes across the continuum of beneficiaries they serve and that SVRA clients with lower service needs have positive outcomes like those attained by EN clients. Other SVRA clients with greater service needs probably have poorer outcomes, but it seems unlikely that those outcomes are worse than they would have been with no services (as the Heckman model implies) when the SVRAs seem capable of achieving significant positive results for other clients. Further, the Heckman model's large negative finding for SVRA users suggests that the service-need bias that remains in the model could be quite substantial and that removing the bias could lead to significantly larger estimated effects for both ENs and SVRAs.

We also find that it seems unlikely for TTW or any alternative return-to-work program to significantly improve on the TTW's results. TTW offered many innovations to address perceived shortfalls of previous programs:

Combined, these TTW innovations affected employment service participation, but they have not led to large increases in disability program exits. It is hard to imagine how further changes in incentives, or the removal of additional barriers, could both remain within the current benefit rules and result in significantly more program exits than we have seen under TTW.

The 2008 change in TTW regulations is also instructive. As discussed in Livermore, Hoffman, and Bardos (2012), the new regulations increased TTW participation and seem to have drawn beneficiaries with greater work challenges on average than those who participated before 2008. Schimmel and others (2013) found that these newly induced participants were less successful in finding work than prior TTW participants had been, which led to an overall decline in STW status:

The recession almost certainly explains some and possibly most of the adjusted decline in the percentage [of participants] with at least one [STW-status] month [−39 percent], but other explanations are possible. In particular, the regulatory changes increased incentives to serve beneficiaries with a lower likelihood of sustaining high levels of earnings.

The authors found similar results for BFW.

Together, these findings suggest that TTW may have reached the maximum number of beneficiaries who are able and wish to work and can be served by employment service providers. This does not mean that employment programs for disability program beneficiaries cannot be improved, but that doing so is likely to have only marginal effects, and it is hard to imagine that a large new program addressing these same incentives and barriers would dramatically move the needle in the way that the TTW legislation envisioned. Instead, significant changes in beneficiary work activity would seem to require structural changes to the disability program rules that would change beneficiary incentives for work.

Still, this analysis shows that TTW has had some significant positive outcomes. It has improved choice and induced more beneficiaries to try to improve their lives with work. TTW employment services have produced definitive improvements in earnings, STW status, and BFW, at least for DI beneficiaries who are served by ENs; it likely had similar effects for SVRA clients as well. At a minimum, these benefit reductions offset some employment service costs, although further research is needed to determine the extent of the net effects. These are all important and valid achievements that SSA should build on. Although TTW has not brought the sea change its designers hoped for, there is substantial evidence that it has, and will continue to have, a significant positive effect on the lives of the people it serves.

Appendix A: Alternative Average Cost Calculations for EN and SVRA Services

Table 6 presents estimated costs per EN and SVRA enrollee, but comparing the average costs of providing EN and SVRA services under TTW is not straightforward for many reasons, of which these two are among the more prominent:

  1. Because SVRAs are also funded by block grants, SSA provider payments cover only part of their costs. In 2017, block grants funded $3.97 billion of SVRA payments, with $3.12 billion (78.7 percent) representing the non-SSA federal share and $0.85 billion (21.3 percent) representing the state share. According to our review of Case Service Reports from the Department of Education's Rehabilitation Services Administration, beneficiaries of SSA disability programs make up at least 30 percent of SVRA enrollees. Therefore, at least some of these federal funds are spent on SSA beneficiaries, even though SSA does not pay them.
  2. Although ENs are allowed to choose which beneficiaries they will serve, SVRAs are required to serve anyone they are able to help. As a consequence, many SVRA clients are not expected to return to work (VR services may target functional improvement rather than employment as a goal), and these clients are not considered to be TTW participants who are likely to generate SSA reimbursement for the provider. Although SSA counts these clients as TTW participants while services are ongoing, SSA funds are highly unlikely to be used for them.

For these and other reasons, SVRAs and ENs use different approaches to enrolling and serving beneficiaries, which likely drives some of the differences between EN and SVRA outcomes seen in Tables 4 and 5. For example, Table 5 indicates that SVRAs receive provider payments for only about 5 percent of the beneficiaries they enroll while ENs receive payments for 26 percent of their clients. This is significant because the different approaches SVRAs and ENs use are reflected in their costs.

Some TTW administrative costs involve services that are available to all beneficiaries. These include outreach and support (for example, the telephone help line provided under the TPM contract) and training for field and regional office staff about the TTW program. Some costs, such as those for managing ticket assignments and unassignments, relate only to TTW participants. Other costs apply only to currently active EN and SVRA clients who are actively participating in TTW; examples include benefit planning queries and other questions handled at field and regional offices. Lastly, the cost of administering TTW provider payments to ENs and SVRAs relates only to TTW participants who generate payments. Because of operational differences, ENs and SVRAs have different proportions of the TTW subpopulations to which these costs apply, so simply averaging the total cost among a single subpopulation, such as all enrollees, might provide misleading comparative results. We therefore seek estimates that account for these differences, to let us compare the cost for beneficiaries who received services from one provider type with the hypothetical cost they would generate had they used the other provider type and attained similar results.

We cannot sort these subpopulations into mutually exclusive categories; but summary statistics compiled from Tables 3, 4, and 5 on total enrollees, enrollees who reach STW status, and enrollees who generate provider payments can capture important differences in the SVRA and EN clientele and associated SSA costs (Table A-1). In Table A-2, we use the subpopulations and cost estimates shown in Table A-1 as the basis for illustrating five alternative methods of calculating average costs for EN and SVRA services. Each calculation method includes a caveat.

Table A-1. TTW 2006 enrollee cohort and related costs
Provider type Number of 2006 enrollees Costs (millions of 2016 $)
Total Attained STW status as of December 2016 Generated provider payments as of December 2016 Annual administrative costs a Cumulative SSA provider payments
SVRA 135,363 25,313 7,418 6.5 108.9
EN 4,306 1,232 1,115 12.3 10.3
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.
a. 2017 costs adjusted to 2016 dollars.
Table A-2. Alternative average cost calculations for SVRAs and ENs
Provider type Average costs (2016 $) Percentage change in SVRA block grants that would equalize SVRA and EN provider payments Caveat
Total Administrative costs a Provider payments
1. Administrative costs and provider payments per enrollee
SVRA 853 48 805 15 An imprecise measure because most enrollees do not generate a provider payment. It artificially increases the participation base and lowers average costs, especially for SVRAs.
EN 5,248 2,856 2,392 . . .
2. Administrative costs and provider payments per STW exit
SVRA 4,559 257 4,302 9 An imprecise measure because many enrollees who attain STW status do not generate a provider payment. It artificially increases the participation base and lowers average costs, especially for SVRAs.
EN 18,344 9,984 8,360 . . .
3. Administrative costs and provider payments per enrollee generating payment
SVRA 15,557 876 14,681 1 An imprecise measure because many administrative costs cover overall operations, not just for those who generate payments. It artificially decreases the participation base and raises average costs, especially for ENs.
EN 20,269 11,031 9,238 . . .
4. Administrative costs per STW exit, provider payments per enrollee generating payment
SVRA 14,937 257 14,681 3 Potentially more accurate, but this measure likely divides administrative costs by too few cases because many costs attributed to ENs are for outreach and training, which affect all enrollees. It may thus overestimate average EN costs.
EN 19,221 9,984 9,238 . . .
5. Administrative costs per enrollee, provider payments per enrollee generating payment
SVRA 14,729 48 14,681 -9 Potentially more accurate, but this measure may divide administrative costs by too many cases because costs are likely to be higher for enrollees generating payments. It may thus underestimate average EN costs.
EN 12,094 2,856 9,238 . . .
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.
. . . = not applicable.
a. 2017 costs adjusted to 2016 dollars.

Table A-2 also includes a column that addresses the fact that SVRAs receive some non-SSA funds to serve SSA program beneficiaries. With those beneficiaries accounting for at least 30 percent of SVRA clients, one might presume that SSA payments represent a similar share of total SVRA funding. Because we cannot determine the proportion of block grant funds that SVRAs use to serve SSA program beneficiaries, we instead identify the percentage of SVRA block grant funding that would equalize average EN and SVRA costs under each of the five cost calculations. For example, the total EN cost per enrollee in Table A-2, calculation 1, exceeds the SVRA cost by $4,395. If, on average, SVRAs spent this difference of $4,395 from their block grant funds on each of their 135,363 enrollees who were SSA program beneficiaries in 2016, that cost would have totaled about $595 million, which is equal to about 15 percent of the $3.97 billion in block grants they received for that year.36 Under average-cost calculation 1, if SVRAs spend more than 15 percent of their non-SSA resources on SSA program beneficiaries, then SVRA services are more expensive on average than EN services. If the figure is less than 15 percent, then SVRA services are less expensive than EN services. Among the five calculation methods, the greater the difference between the EN and SVRA average costs, the higher the percentage of block grant spending that would be needed to equalize these costs. For calculation 5, average costs are lower for ENs than for SVRAs, so the percentage becomes negative and EN services are less expensive regardless of what proportion of the non-SSA funds are spent on SSA program beneficiaries. In no  case, however, does the block grant percentage approach 30 percent, so unless SVRAs consistently spend considerably less than 30 percent of their non-SSA resources on the 30 percent of clients who are SSA program beneficiaries, SVRA services are more expensive on average than EN services when all SSA payments, federal block grants, and state block grant shares are included.

We explored the five alternative calculations to find an allocation method that reasonably adjusts for service intensity, such that providers whose clients require more intensive services get a higher share of the costs. The caveats in Table A-2 address potential strengths or limitations of each allocation method. In particular, the variability in the calculations shows that averaging all costs by a single group likely leads to imprecise estimates because administrative costs and provider payments are focused on different subsets of the beneficiary population. Averages per enrollee overall (calculation 1) likely underestimate both administrative costs and provider payments while averages per STW exit (calculation 2) likely overestimate administrative costs and underestimates payments.

Calculations 4 and 5 provide the best comparative average cost estimates for ENs and SVRAs by using different populations for averaging administrative and payment costs. Only in calculation 5 are the average total costs lower for ENs than for SVRAs. Still, neither of these estimates is entirely accurate and we conclude the true costs likely lie somewhere between calculations 4 and 5.

Regardless of calculation method for average costs, total administrative costs are greater for ENs than for SVRAs. Table A-3 looks at the types of costs that account for the differences. Administrative costs for headquarters operations and provider payments are similar for the two provider types. Regional and field office operational costs are higher for ENs because those services entail more TTW training for SSA staff and more inquiries from providers and participants than SVRA services do. TPM contract costs also diverge widely, mostly because the larger of the two components of TPM costs—for provider outreach/recruitment and timely completion of client progress reviews—is specific to EN services. The other TPM costs, for operating the TTW call center and managing ticket assignments, are higher for SVRA services because they are allotted by the relative size of the EN and SVRA participant populations.

Table A-3. TTW administrative costs, by category: SVRAs and ENs, 2017
Cost category Costs (2016 $) Difference (EN minus SVRA)
Total SVRA EN 2016 $ Percent
Total 18,835,670 6,504,129 12,331,541 5,827,413 31
Staff
All 5,654,934 1,980,802 3,674,133 1,693,330 30
To administer—
SSA headquarters operations and provider payments 2,939,856 1,378,063 1,561,793 183,730 6
Regional and field office operations a 2,715,078 602,739 2,112,339 1,509,600 56
TPM contract
All 12,557,283 4,018,330 8,538,952 4,520,622 36
To administer—
Provider outreach/recruitment and participant progress reviews 7,534,370 0 7,534,370 7,534,370 100
Ticket assignments and TTW call center operations 5,022,913 4,018,331 1,004,583 -3,013,748 -60
Printing and mailing 623,452 504,996 118,456 -386,540 -62
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTE: Totals do not necessarily equal the sum of rounded components.
a. Most of these costs are related to training and responding to inquiries from service providers and participants.

Some of these costs are not strictly administrative in that they are not necessary to run the TTW program. For instance, responding to participant queries and completing timely progress reviews can respectively be viewed as broader beneficiary services and program integrity workloads that are not central to TTW operations. To estimate how “administrative” costs that are not strictly administrative in nature could affect the difference between average EN and SVRA costs, in Table A-4 we reconsider calculations 4 and 5 from Table A-2, identifying the EN costs that, if removed, would essentially eliminate the differences. Calculation 6 in Table A-4 shows that if $7 million in EN costs are not essential to TTW operations37 and we remove those costs, EN and SVRA costs equalize, even though this method adopts calculation 4's denominator (enrollees who reach STW status), which likely overstates EN average costs. This is not to say there are $7 million in EN costs that are not essential to TTW operations. TTW mandated many costs, and if $7 million of those costs were unnecessary, EN and SVRA costs would be the same.

Table A-4. Alternative average cost calculations for SVRA and EN services isolating "nonessential" administrative costs
Provider type Average costs (2016 $) Percentage change in SVRA block grants that would equalize SVRA and EN provider payments Caveat
Total Administrative costs a Provider payments
6. Administrative costs per STW exit, provider payments per enrollee generating payment
SVRA 14,937 257 14,681 0 Removes $7 million in TPM contract costs for EN services. Administrative costs are likely still to be overestimated because some beneficiaries who receive TTW services do not attain STW status.
EN 14,937 5,700 9,238 . . .
7. Administrative costs per enrollee, provider payments per enrollee generating payment
SVRA 14,831 150 14,681 -6 Administrative cost estimates are adjusted to try to account for different levels of TTW activity among service providers of each type.
EN 13,238 4,000 9,238 . . .
SOURCE: Authors' calculations using 2016 DAF and O'Leary and Roessel (2018).
NOTES: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
Includes some individuals who generated payments to both an SVRA and an EN under a single ticket assignment. These cases most likely reflect a TTW initiative called Partnership Plus that enables a client to transfer an ongoing assignment from one provider type to the other.
. . . = not applicable.
a. 2017 costs adjusted to 2016 dollars.

Using the methodology of calculation 5 in Table A-2 and a rough estimate of the likely distribution of administrative costs among service providers with high and low TTW activity, calculation 7 in Table A-4 shows a “best guess” of average costs and the actual provider payment amounts. Even in this scenario, costs are lower for ENs than for SVRAs (ignoring block grant resources), although the differences are smaller than those for calculation 5 in Table A-2.

This appendix shows that when we analyze TTW costs in a manner that addresses the inherent differences in how each provider type operates, it is reasonable to conclude that average EN and SVRA costs are similar. It also shows how the simple average can be misleading. Table A-5 shows the simple average costs for 2016 of $835 for SVRAs and $5,204 for ENs. If all EN clients had been served by SVRAs, total cost, as estimated using simple averages, would be $116.6 million. This suggests that moving clients from ENs to SVRAs would significantly lower costs. However, these EN clients are more successful than the average SVRA client so the SVRA cost for serving these clients would be higher than the simple SVRA average cost. If we use only SVRA costs for both EN and SVRA enrollees but compute average administrative costs using all enrollees and provider payments using only those enrollees generating payments (as in calculation 5), the estimated total cost is $129.3 million. This higher estimate adjusts for the higher success rate for EN clients and the higher costs associated with that success. This estimate is slightly lower than the combined EN and SVRA costs ($135.5 million) because it assumes that none of the $3.97 billion in non-SSA funds were used to serve these SVRA clients.

Table A-5. Alternative cost calculations for SVRA and EN services assuming SVRA costs for both provider types (in 2016 dollars)
Provider type Number of 2006 enrollees Average administrative costs a per enrollee Number of enrollees who generated a provider payment Average payment per enrollee who generated a payment Simple average costs Total costs (in millions)
Actual Assuming SVRA-level costs for both provider types
Based on simple average Based on adjusted average
TTW 139,669 135 8,533 13,671 970 135.5 116.6 129.3
SVRA 135,363 48 7,418 14,367 835 113.1 113.0 113.1
EN 4,306 2,863 1,115 9,041 5,204 22.4 3.6 16.2
SOURCE: Authors' calculations and O'Leary and Roessel (2018).
NOTE: Includes DI and DI/SSI beneficiaries and SSI recipients who were aged 18–65 in January 2006 and were on the rolls at any point in 2006. Excludes beneficiaries who enrolled in an SVRA or an EN before or after 2006.
a. 2017 costs adjusted to 2016 dollars.

Because EN and SVRA costs are so similar when properly calculated, removing either the EN or SVRA side of TTW services would not materially change the overall cost of providing employment services to SSA disability program beneficiaries. The costs would simply shift to the other provider type and the net effect would be minimal. In other words, if SSA were to stop supporting EN services, SVRA costs would go up by an amount similar to the resulting reduction in EN costs, as the ENs' former clients moved to the SVRAs. The only way for costs to decline in this case would be if the change resulted in an overall decrease in service provision.

Notes

1 CDRs are suspended while an individual is enrolled in TTW. In the absence of this protection, CDRs generally occur every 3 to 7 years, depending on the severity of the beneficiary's disabling condition.

2 Each report consists of from one to nine individual “studies.” All of the reports and the component studies are available at https://www.ssa.gov/disabilityresearch/twe_reports.htm#twe.

3 SGA is an annually adjusted monthly earnings threshold with which SSA determines whether an individual is disabled. SSA generally regards individuals with monthly earnings at or above the SGA threshold to be capable of working at a self-sustaining level and thus not disabled. There are two thresholds depending on the nature of the disability. For 2023, the monthly SGA thresholds are $2,460 for individuals who are statutorily blind and $1,470 for other individuals.

4 Because this method relies on random groups with differing access to services, it is regarded as an experimental design.

5 For example, Dean, Dolan, and Schmidt (1999) examined SVRA case closures from 1980 and found that both men and women increased their earnings by about $5,000 in total over the 8 years after SVRA participation (our calculations based on that study's Table 6) and had “an aggregate [earnings to service cost] ratio of 2.61 for 12,031 women and 2.43 for 14,075 men.” The authors concluded “that the [SVRA] program returns roughly $2.50 for each dollar spent.”

6 Further complicating the comparison, states often do not have sufficient resources to serve all eligible individuals. When this occurs, the state's SVRAs enter “order of selection” status, under which they are required to prioritize services for individuals who are deemed by the state to have the most significant disabilities. In such circumstances, SVRAs are significantly disadvantaged relative to ENs in terms of their clients' employment prospects. We will revisit this circumstance later.

7 The number of entities that have registered as ENs is higher, but the number that actively participate in TTW at a given time has generally hovered around 600 (Prenovitz, Bardos, and O'Day 2012).

8 The TTW rollout began in 2003 but the program was not available in all states until the end of 2005.

9 A chi-square test of independence showed statistically significant increases from 2004 to 2015 in four of the five measures of work interest. Only the change in “Sees oneself working for pay in the next 2 years” was not significant.

10 Stapleton and others (2008) also found that older beneficiaries were relatively more likely to assign their ticket to an EN. Schimmel and others (2013) added that “there is no indication that the [573 percent] increase in [EN milestone-only (MO) payment-method] assignments [from 2007 to 2010] simply reflects a shift from SVRA acceptance of assignments under the traditional payment system toward more acceptance under the MO system. Indeed, the pace of growth of assignments to SVRAs under the traditional system was about the same as that of eligible beneficiaries during this period, with 0.6 percent of beneficiaries making assignments to the traditional payment system in each year from 2005 through 2010.”

11 The regression-adjusted satisfaction estimates for ENs rose from 50 percent to 68 percent. We use regression-adjusted estimates for this metric because the composition of participants before and after the implementation of the 2008 regulations differed.

12 Honeycutt and others (2015, Table 6) found that 31 of the 51 SVRAs they examined were under an order of selection. GAO (2009, Table 7) found at least one SVRA under an order of selection in 31 states in fiscal year 2008.

13 Thornton could not directly measure exits because of STW status for SVRAs. Instead, he used SSA payments to SVRAs for successful employment rehabilitation (meaning work at the SGA level for 9 months within a 12-month period) as a proxy. Such rehabilitation outcomes do not necessarily entail STW status.

14 SVRA and EN costs are structured differently, which complicates comparisons. For example, for ENs, Thornton (2012) combined the sum of the milestone payments (which SSA pays in full when a client reaches STW status) plus at least one outcome payment, plus additional outcome payments according to the proportions of clients that remain off the disability rolls over time. For SVRA costs, Thornton used average cost reimbursement paid by SSA, assuming that all clients who generate a provider reimbursement reach STW status (which is an oversimplification). It is an imperfect measure that ignores administrative costs (which are higher for ENs than for SVRAs) and provider payments funded by state VR block grants. It also excludes payments to ENs for beneficiaries who never reach STW status. Thornton accounted for both administrative costs and provider payments for EN clients who never reach STW status in calculating what it would take for EN savings to cover EN costs, but not in his comparison of EN and SVRA costs. Our Appendix A provides alternative cost estimates but its findings are similar to Thornton's.

15 We examine costs and benefits only from the perspective of the SSA disability programs. As such, BFW reflects only DI and SSI benefits and ignores potential effects on other programs such as Medicare and Medicaid. With additional data, a broader analysis of costs and benefits could be completed from the perspective of the entire federal government, state governments, and the taxpayer. Any potential TTW-related administrative costs incurred by other federal agencies and state governments, such as those involved in removing former beneficiaries from Medicaid or food stamps or processing income tax returns, would likely be negligible and probably contribute to net positive results. As such, the non-SSA effects of TTW would at worst be zero. To the extent that TTW has positive net effects beyond SSA, then the net effects on SSA are a lower bound that would imply greater net savings or lesser net costs if they were included in the analysis.

16 Because service providers typically must request reimbursement to be paid, and providers sometimes delay (or neglect) filing a request, Table 2 does not fully account for reimbursable services provided.

17 In our 2018 report, we estimated TTW administrative costs as the sum of SSA staffing and contract costs related to the TTW program. Contracting costs were provided by SSA's Office of Employment Support, the office that operates TTW, and SSA's Office of Budget.

18 Total costs in 2006 would have included tracking and payment functions for prior years' participants and recruitment activities that led to participation in subsequent years. We assume that the values for preceding and subsequent years essentially cancel each other out and avoid these computational complexities.

19 Estimating the cost of protecting TTW participants from CDRs is beyond the scope of this analysis. The Congressional Budget Office (CBO) estimated that the CDR exemption would enable SSA to save $10 million a year by conducting fewer CDRs and add $25 million a year in benefits paid to individuals who would have been found ineligible in a CDR, for a net annual cost of $15 million (CBO 1999). However, that study was overly optimistic about TTW participation, assuming that 7 percent of beneficiaries would participate, nearly triple the actual rate of less than 2.5 percent (Altshuler and others 2013). Proportionally reducing the CBO estimate to align with actual participation rates yields a net cost to SSA of $4.7 million a year ($7.4 million in 2016 dollars). For the Ticket Act's 2008 amendments, SSA estimated that the cost of the CDR deferrals would be $287 million over 10 years. This translates into $320 million in 2016 dollars, or $32 million per year, which is 4.3 times the adjusted CBO estimate. More recently, the GAO found that including CDR costs took the TTW program from $2.03 billion in net savings to $529 million in net costs for the period August 2008 through December 2018 (GAO 2021, Table 7). Adjusted to 2016 dollars, GAO's estimated cost of the CDR exemption is $2.4 billion, or about $230 million per year, which is 31 times the adjusted CBO estimate and 7 times the SSA estimate. Calculating the CDR cost of TTW is clearly unsettled so we omit these estimates from our analysis.

20 We regard overpayments as a general administrative cost rather than a TTW-specific cost. Adding this cost to TTW unfairly penalizes the program for succeeding in getting beneficiaries back to work.

21 Because SVRAs are also funded by a combination of federal block grants and state funds (see https://rsa.ed.gov/about/programs/vocational-rehabilitation-state-grants), SSA pays only part of the cost of their operations. The Department of Education disburses federal grants authorized by the Rehabilitation Act of 1973, as amended by Title IV of the Workforce Innovation and Opportunity Act. The grants assist states in operating VR programs for individuals with disabilities. The federal funds are provided via a block grant formula that requires state matching funds. Currently, the federal funds provide 78.7 percent of the VR operating costs and states provide the remaining 21.3 percent. The combined funds cover both administrative and service-provision costs. In 2017, non-SSA expenditures on VR services totaled $3.97 billion, of which $3.12 billion came from federal grants. Because SSA program beneficiaries constitute no less than 30 percent of SVRA enrollees, at least some of these federal grants are spent on them.

22 For details, see https://yourtickettowork.ssa.gov/program-operations/payments.html.

23 The EN payment milestone phases have no connection with the TTW program rollout phases.

24 In our calculations, we stop attributing to the 2006 cohort any service provider payments and any BFW that result from a subsequently assigned new ticket.

25 We compared postenrollment employment outcomes for the 2006 and later enrollee cohorts. Using a 3-year postenrollment follow-up period for each cohort, we found the employment outcomes for post-2006 enrollees to be similar to, but generally lower than, those for 2006 enrollees. For example, the proportion of participants with at least 1 month of STW status in the 3 years after assignment was 11.8 percent for the 2006 cohort, 8.3 percent for the 2009 cohort, and 10.5 percent for the 2013 cohort. Similarly, average BFW were $2,328 for the 2006 cohort, $1,547 for the 2009 cohort, and $2,132 for the 2013 cohort. Because the deviations for the 2009 cohort seem to be driven largely by the Great Recession, the 2006 cohort outcomes can be regarded as reflecting TTW assignments during a relatively robust economic period.

26 Stapleton and others (2010) found that “providers received [new TTW-system] payments in approximately 40 percent of the months in which…participants were off the rolls for work during the period from 2002 through 2006.” We found similar results for SVRA participants.

27 Our analysis excludes individuals who participated in TTW only in years other than 2006. It also excludes any costs or BFW outcomes that result from a new ticket assigned after 2006 by a 2006 enrollee. We assume that costs and outcomes occurring after a subsequent ticket assignment are attributable to that subsequent assignment and not the 2006 assignment. Because we omitted individuals who participated in TTW in years other than 2006, the comparison group in our analysis—TTW nonparticipants—never participated in TTW at any time during the observation period.

28 It is also possible that some ENs receive state or non-TTW federal funding through programs such as the Workforce Investment Act. SSA does not collect such data and we have not attempted to account for funding from these potential sources.

29 Although we could have used STW status instead of TWP completion, we chose the latter because it follows 9 months of work above SGA level, thereby providing the earliest indication of work ability and serious motivation—and seriousness about work is what we most want to capture. Using TWP completion also seems akin to including the self-employed in the standard age/education Heckman framework with the selection equation based on hours of work. Positive hours of work indicate serious work effort the same way that TWP completion does, with the possibility that the effort could result in zero or even negative earnings, just as BFW can be zero despite TWP completion.

30 Technically, we do not have a true identification problem between the two stages of the Heckman model because stage one is a nonlinear probit model and stage two is a linear OLS regression.

31 In fact, ignoring selection bias attributable to unobservable work aptitude and effort results in significantly higher effects for both EN services (to $10,603) and SVRA services (from a negative value to $2,351).

32 Note that BFW continued to accrue after 2016 and thereby have since risen relative to costs. Administrative costs accrue only in the year of assignment under our cost methodology, and SSA payments to service providers beyond 2016 are very small. Although only a small proportion of beneficiaries who exit the benefit rolls remain independent permanently, their employment success raises their cumulative BFW each month until they die or reach retirement age.

33 More precisely, STW status occurs for SSI recipients when the benefit payment drops to $0 because the recipient's earnings exceed the combined general and earnings exclusion of $85, and the benefit is reduced to $0 by the $2 reduction in payment for every $1 in earning above the exclusion. Thus, for an individual with an $800 monthly SSI benefit, $1,685 in earnings in a month would put the recipient in STW status.

34 We completed a back-of-the-envelope administrative cost estimate using a combination of historical data and level-of-effort information from O'Leary and Roessel (2018) and found that 2006 costs may have been about $28 million (in 2016 dollars) rather than the $19 million we used in this analysis. However, about half of the difference ($4.5 million) is attributable to TPM contracting costs and the other half covers SSA headquarters staff time to implement and manage TTW. These costs rose from 2006 to 2012 then declined thereafter. The TTW rollout ended in 2006, new regulations were implemented in 2008, and significant changes to payment processing occurred through 2011, all of which contributed to higher administrative costs over this period. From 2012 to 2017, administrative costs declined, as new assignments rose by an average of 1.7 percent per quarter and in-use tickets increased from 392,823 to 462,000 (compared with 214,428 for 2006, based on authors' calculations using DAF data). Together, this suggests that a significant portion of the higher 2006 costs reflects startup activities that are not part of the mature TTW program, leading us to conclude that 2017 data better represent steady-state administrative costs.

35 TTW's expansion of payment models may have had less effect than was expected because the SVRA reimbursement model that predated TTW can itself be seen as outcome-based in that provider payments are predicated on the client reaching SGA in 9 of 12 months. The milestone payment model is a departure from a pure outcome system yet it functions much like reimbursement in that it pays for services based on milestones toward STW status rather than fully attaining STW status. Because outcomes did not change dramatically with the addition of EN service options and the new payment systems, the SVRA payment trigger of reaching SGA in 9 of 12 months seems to indicate the same level of success as reaching STW status under the new TTW payment rules.

36 This percentage calculation uses the number of SVRA clients that we used in calculating average administrative costs (that is, either total enrollees, the number reaching STW, or the number who generate provider payments) as specified for each average-cost calculation method in Table A-2.

37 Using figures from Table A-2, calculation 4, we subtracted the difference between EN total cost and SVRA total cost ($19,221 − $14,937, or $4,284) from the average administrative costs per STW exit for EN clients ($9,984). We then multiplied the result ($5,700) by the number of 2006 EN enrollees who attained STW status as of December 2016 in Table A-1 (1,232) to estimate $7,022,400 in TPM contract costs for EN services.

References

Altshuler, Norma, Sarah Prenovitz, Bonnie O'Day, and Gina Livermore. 2013. Provider Experiences Under the Revised Ticket to Work Regulations. Washington, DC: Mathematica Policy Research.   https://www.ssa.gov/disabilityresearch/documents/Process%20report%20errata.pdf.

Ashenfelter, Orley. 1978. “Estimating the Effect of Training Programs on Earnings.” Review of Economics and Statistics 60(1): 47–57.

Autor, David H., and Mark Duggan. 2010. Supporting Work: A Proposal for Modernizing the U.S. Disability Insurance System. Washington, DC: The Center for American Progress and Brookings Institution Hamilton Project.

Butler, Joanne. 2018. “Tightening Social Security Disability Will Obviously Lead To Higher Unemployment.” The Daily Caller, January 8. https://dailycaller.com/2018/01/07/tightening-social-security-disability-will-obviously-lead-to-higher-unemployment/.

[CBO] Congressional Budget Office. 1999. “Pay-As-You-Go Estimate: H.R. 1180, Ticket to Work and Work Incentives Improvement Act of 1999.” Washington, DC: CBO.

Couch, Kenneth A. 1992. “New Evidence on the Long-Term Effects of Employment Training Programs.” Journal of Labor Economics 10(4): 380–388.

Dean, David H., Robert C. Dolan, Robert M. Schmidt. 1999. “Evaluating the Vocational Rehabilitation Program Using Longitudinal Data: Evidence for a Quasiexperimental Research Design.” Evaluation Review 23(2): 162–189.

Dean, David H., Robert C. Dolan, Robert M. Schmidt, Paul Wehman, John Kregel, and Grant Revell. 2001. “A Paradigm for Evaluation of the Federal-State Vocational Rehabilitation Program.” In Achievement and Challenges in Employment Services for People with Disabilities: The Longitudinal Impact of Workplace Supports Monograph, edited by John Kregel, David H. Dean, and Paul Wehman (1–40). Richmond, VA: Virginia Commonwealth University Rehabilitation Research and Training Center for Workplace Supports.

Dean, David, John V. Pepper, Robert M. Schmidt, and Steven Stern. 2014. “State Vocational Rehabilitation Programs and Federal Disability Insurance: An Analysis of Virginia's Vocational Rehabilitation Program.” IZA Journal of Labor Policy 3(7): 1–19.

———. 2015. “The Effects of Vocational Rehabilitation for People with Cognitive Impairments.” International Economic Review 56(2): 399–426.

Eimicke, William, Steven A. Cohen, and Alison Miller. 2017. The Ticket to Work Program: Helping the Disabled to Achieve Self-Sufficiency. New York, NY: Manhattan Institute.

[GAO] Government Accountability Office. 2007. Vocational Rehabilitation: Earnings Increased for Many SSA Beneficiaries After Completing VR Services, but Few Earned Enough to Leave SSA's Disability Rolls. GAO-07-332. Washington, DC: GAO. https://www.gao.gov/assets/gao-07-332.pdf.

———. 2009. Vocational Rehabilitation Funding Formula: Options for Improving Equity in State Grants and Considerations for Performance Incentives. GAO-09-798. Washington, DC: GAO. https://www.gao.gov/new.items/d09798.pdf.

———. 2011. Social Security Disability: Ticket to Work Participation Has Increased, but Additional Oversight Needed. GAO-11-324. Washington, DC: GAO. https://www.gao.gov/assets/gao-11-324.pdf.

———. 2021. Social Security Disability: Ticket to Work Helped Some Participants, but Overpayments Increased Program Costs. GAO-22-140431. Washington, DC: GAO. https://www.gao.gov/assets/gao-22-104031.pdf.

Heckman, James J. 1974. “Shadow Prices, Market Wages and Labor Supply.” Econometrica 42(4): 679–694.

———. 1976. “The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models.” In Annals of Economic and Social Measurement, Volume 5, Number 4, edited by Sanford V. Berg (475–492). Cambridge, MA: National Bureau of Economic Research.

Honeycutt, Todd, and David Stapleton. 2013. “Striking While the Iron Is Hot: The Effect of Vocational Rehabilitation Service Wait Times on Employment Outcomes for Applicants Receiving Social Security Disability Benefits.” Journal of Vocational Rehabilitation 3(2): 137–152.

Honeycutt, Todd, Allison Thompkins, Maura Bardos, and Steven Stern. 2015. “State Differences in the Vocational Rehabilitation Experiences of Transition-Age Youth with Disabilities.” Journal of Vocational Rehabilitation 42(1): 17–30.

Huynh, Minh, and Paul O'Leary. 2003. “Issues Affecting Alternatives to the Ticket to Work Incentive Structure.” In Paying for Results in Vocational Rehabilitation: Will Provider Incentives Work for Ticket to Work?, edited by Kalman Rupp and Stephen H. Bell. Washington, DC: Urban Institute Press.

Khazan, Olga. 2015. “Why Is It So Hard to Find Jobs for Disabled Workers?” The Atlantic, March 30.

Killingsworth, Mark R. 1983. Labor Supply. New York, NY: Cambridge University Press.

Lankford, James. 2017. “I've Got a Ticket to Work.” Federal Fumbles: 100 Ways the Government Dropped the Ball 3: 38. https://www.lankford.senate.gov/imo/media/doc/Federal_Fumbles_2017.pdf.

Lawler, Joseph. 2014. “Social Security's Futile Attempt to Help Disabled People Go Back to Work.” Washington Examiner,  August 15. http://www.washingtonexaminer.com/social-securitys-futile-attempt-to-help-disabled-people-go-back-to-work/article/2552063.

Livermore, Gina A., Denise Hoffman, and Maura Bardos. 2012. Ticket to Work Participant Characteristics and Outcomes Under the Revised Regulations. Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/documents/Final%20TTW%20Outcomes.pdf.

Mashaw, Jerry L., and Virginia P. Reno, editors. 1996. Balancing Security and Opportunity: The Challenge of Disability Income Policy. Washington, DC: National Academy of Social Insurance.

Mathematica. 2022. Disability Analysis File 2020 (DAF20) Documentation: Data from January 1994 through December 2020. Volume 3: Tips for Conducting Analysis with the DAF20. Washington, DC: Mathematica. https://www.ssa.gov/disabilityresearch/documents/daf/V03.%20DAF20_Tips%20for%20Conducting%20Analysis.pdf.

O'Leary, Paul, and Emily Roessel. 2018. Administrative Costs of SSA's Ticket to Work Program: Design and Estimation of 2017 Costs. Unpublished report. Washington, DC: SSA.

Prenovitz, Sarah, Maura Bardos, and Bonnie O'Day. 2012. Ticket to Work After the Release of the 2008 Revised Regulations: Progress and Prospects. Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/documents/FINAL%20Revised%20Regulations.pdf.

Radtke, Jean, editor. 2000. Effective Strategies to Improve the Employment of SSI/SSDI Participants. Menominie, WI: University of Wisconsin-Stout Vocational Rehabilitation Institute.

Schimmel, Jody, David Stapleton, David R. Mann, and Dawn Phelps. 2013. Participant and Provider Outcomes Since the Inception of Ticket to Work and the Effects of the 2008 Regulatory Changes. Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/documents/TTW-NSTW%20Report-Final-072513.pdf.

[SSA] Social Security Administration. 2008. Ticket to Work and Self-Sufficiency Program Cost Effectiveness. A-02-07-17048. Washington, DC: SSA, Office of the Inspector General. https://oig-files.ssa.gov/audits/full/A-02-07-17048_0.pdf.

———. 2018a. National Beneficiary Survey: Disability Statistics, 2015. Washington, DC: SSA. https://www.ssa.gov/policy/docs/statcomps/nbs/2015/index.html.

———. 2018b. The Social Security Administration's Programs and Projects That Assist Beneficiaries in Returning to Work. A-04-18-50600. Washington, DC: SSA, Office of the Inspector General. https://oig-files.ssa.gov/audits/full/A-04-18-50600_0.pdf.

———. 2020. DI & SSI Program Participants: Characteristics & Employment, 2015. Washington, DC: SSA. https://www.ssa.gov/policy/docs/chartbooks/di-ssi-employment/2015/index.html.

Stapleton, David C., and Gina A. Livermore. 2003. “A Conceptual Model and Evaluation Strategy for the Empirical Study of the Adequacy of Incentives in the Ticket to Work Program.” In Paying for Results in Vocational Rehabilitation: Will Provider Incentives Work for Ticket to Work?, edited by Kalman Rupp and Stephen H. Bell (71–100). Washington, DC: Urban Institute Press.

Stapleton, David, Gina Livermore, Craig Thornton, Bonnie O'Day, Robert Weathers, Krista Harrison, So O'Neil, Emily Sama Martin, David Wittenburg, and Debra Wright. 2008. Ticket to Work at the Crossroads: A Solid Foundation with an Uncertain Future. Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/ttw4/TTW_Rpt4_508_vol1r.pdf.

Stapleton, David, Arif Mamun, and Jeremy Page. 2013. Initial Impacts of the Ticket to Work Program for Young New Social Security Disability Awardees: Estimates Based on Randomly Assigned Mail Months. Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/documents/Ticket%20Impact%20Report.pdf.

Stapleton, David, Jody Schimmel, Miriam Loewenberg, and Sarah Prenovitz. 2010. Work Activity and Use of Employment Supports Under the Original Ticket to Work Regulations: Time That Beneficiaries Spend Off the Rolls Due to Work and the Payments Generated for Employment Networks. Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/documents/TTW5_8_LDW.pdf.

Thornton, Craig. 2012. Can the Ticket to Work Program Be Self-Financing? Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/documents/TTW%20Financing%20508.pdf.

Thornton, Craig, Thomas Fraker, Gina Livermore, David Stapleton, Bonnie O'Day, Tim Silva, Emily Sama Martin, John Kregel, and Debra Wright. 2006. Evaluation of the Ticket to Work Program: Implementation Experience During the Second Two Years of Operations (2003–2004). Washington, DC: Mathematica Policy Research. https://www.ssa.gov/disabilityresearch/twe_reports/twe2.htm.

Ticket to Work and Work Incentives Advisory Panel. 2007. Building on the Ticket: A New Paradigm for Investing in Economic Self-Sufficiency for People with Significant Disabilities. Final Report to the President and Congress, Year Eight of the Panel. SSA Publication No. 63-011. Washington, DC: SSA.