Ticket to Work Evaluation Report (February 2004)

This is an archival or historical document and may not reflect current policies or procedures.
Table of Contents      List of Tables      List of Figures      References


CHAPTER 1

Overview of the Ticket to Work Program and Its Evaluation

The Ticket to Work and Work Incentives Improvement Act of 1999 (Ticket Act) established the Ticket to Work program (TTW) to increase access to, and the quality of, rehabilitation and employment services available to Social Security disability beneficiaries, and ultimately to increase the number who become economically self-sufficient. Currently, very few beneficiaries leave the rolls due to work. TTW tries to help more beneficiaries exit due to work by changing the way the Social Security Administration (SSA) pays for employment services. It also changes some program rules in order to let beneficiaries explore work opportunities without jeopardizing their benefit status.

The TTW program operates with the apparent paradox of trying to promote work among a group of individuals judged incapable of substantial work. People who receive disability benefits from either SSA’s Disability Insurance (DI) or Supplemental Security Income (SSI) programs have been judged to have a medically determinable impairment that is expected to last at least 12 months or result in death and that renders them unable to engage in substantial gainful activity. The vast majority of beneficiaries will not attempt any work once they are on the rolls. Only about 2.5 percent of any enrollment cohort will ultimately leave the rolls due to work and less than 0.5 percent of all beneficiaries on the rolls at a point in time eventually leave due to work (Newcomb et al. 2003; Berkowitz 2003).

The low employment rates among disability beneficiaries have proven difficult to increase substantially. There is no evidence that the many work incentives that SSA has instituted prior to TTW have increased work-related program exits (Newcomb et al. 2003). Furthermore, two prior SSA demonstrations to test employment support programs, Project Network and the Transitional Employment Training Demonstration, had low participation rates: about 6 percent of eligibles participated (Kornfeld et al. 1999; and Decker and Thornton 1995). In addition, while both demonstrations’ interventions generated a large proportional increase in participant earnings, those increases were small in absolute terms. These small absolute increases translated into negligible reductions in benefit payment as most working participants had earnings below the thresholds that would result in losing their benefits.

Nevertheless, it is well-known that many people with medical conditions that would make them eligible for disability benefits do in fact work, and advances in technology and rehabilitation techniques make it feasible for many people with very severe disabilities to obtain and hold jobs (for example, Bond et al. 1997). This has generated a continuing interest in promoting employment among SSA’s disability beneficiaries, which in turn has led to a consensus that no person with a disability should be denied the right to participate fully in society, including work, because of external barriers that can be removed with reasonable efforts. The main issues for SSA are therefore: what are the best methods for addressing barriers? How many beneficiaries will seek to take advantage of new opportunities? Will the programs enable many beneficiaries to earn enough to leave the rolls? And what will be the net cost or savings to the government?

There are essentially four major types of barriers that disability beneficiaries face when they want to obtain substantial employment:

  • Benefit Policies That Reduce Gains from Employment. Cash benefit programs, including DI and SSI, generally contain provisions to reduce or stop benefits as a beneficiary’s earnings increase. This can create a substantial disincentive for beneficiaries to work since a beneficiary’s total income (benefits plus earnings) may rise slowly, or in some cases even fall, as earnings increase. Also, while health insurance benefits through Medicare and Medicaid are available even to beneficiaries who are no longer receiving cash benefits, many beneficiaries may nevertheless be concerned about losing those benefits if they attempt to work.

  • Limited Beneficiary Knowledge of the Service System. Both the DI and SSI programs contain provisions designed to encourage work among beneficiaries. However, many beneficiaries are unfamiliar with these provisions and have an incomplete picture of how working will actually affect their benefits. Beneficiaries may also be concerned that the provisions, which are often fairly complex, may not be implemented fully or accurately. In either of these cases, beneficiaries are likely to under use the provisions and be less inclined to work.

  • Inadequate Employment-Related Skills or Workplace Accommodations. Beneficiaries may lack the full set of skills and attitudes required for successful employment, including knowledge of the labor market and how to search for a job as well as more fundamental job skills or basic education. Beneficiaries may also require accommodations to help them overcome impairments that might prevent them from being productively employed. Furthermore, some beneficiaries may have become discouraged about work and withdrawn completely from the labor force. While beneficiaries could, in theory, purchase training and job placement services, they often lack sufficient financial resources or access to loans to purchase the training or services that would enable them to obtain substantial employment.

  • Employers’ Misimpressions. Persons with disabilities may also face barriers created by employers’ misimpressions of their abilities or, in some cases, discrimination. Thus, even when they want to work, they can have a difficult time getting a job offer.

The TTW program and other elements of the Ticket Act try to provide the means to help beneficiaries overcome these barriers. In particular, the TTW program introduces a new financing system for providers and gives beneficiaries a choice over which provider to use. The new financing system adds two payment options to the traditional payment system that SSA has used in the past to pay state vocational rehabilitation agencies (SVRAs) for rehabilitation services provided to beneficiaries. The traditional system reimburses an agency’s costs, up to a limit, if a beneficiary obtains earnings of at least the substantial gainful activity level (currently set at $810 per month for most individuals) for nine months. The new options have substantially stronger performance incentives because they require a beneficiary to exit cash benefit status by reason of increased earnings for 60 months before the provider receives full payment. The first new option, the outcome-only payment system, provides higher payments but makes no payments until a beneficiary leaves the rolls. The other new option, the milestone-outcome system, provides smaller outcome payments, but can provide up to four larger milestone payments while a beneficiary is still receiving benefits, if a beneficiary achieves intermediate earnings targets.

TTW increases choice by greatly expanding the types of organizations that it will pay to assist beneficiaries’ work efforts. Beneficiaries can choose between a range of public and private providers other than SVRAs, called Employment Networks (ENs), that have signed a contract with SSA. ENs cannot use the traditional payment system, they must elect to be paid under either the outcome-only or milestone-outcomes payment systems. SVRAs can act as ENs by using the new payment systems, but they can also decide to serve beneficiaries under the traditional system.

In addition, TTW gives service providers and beneficiaries considerable flexibility to choose the services that will be provided. In fact, providers and beneficiaries must agree on an individualized employment plan before a ticket can be put into use. This plan could, in theory, include a wide array of services designed to help beneficiaries overcome barriers related to their knowledge of the system and labor market, their employment-related skills, and even employer misperceptions of their abilities.

Service delivery is constrained, however, by providers’ desire to limit service expenditures to a level that fits within the payments they expect to receive and by providers’ assessments of whether the services they can provide are likely to result in a beneficiary leaving the rolls. Participation in TTW is completely voluntary for beneficiaries and providers, so providers can refuse to serve beneficiaries whom they think have a low probability of leaving the rolls due to work (thereby triggering outcome payments).

The TTW legislation also introduces other changes that try to reduce the policy barriers that can make work unattractive to some beneficiaries. In particular, the Ticket Act contained the following provisions:1

  • Expedited reinstatement that allows beneficiaries who have left the rolls because of work to have their benefits (and any associated health insurance) reinstated without a new application. This option is available for five years following termination of a beneficiary’s eligibility for disability status due to work.

  • Suspension of medical disability reviews used to assess whether a beneficiary’s impairments are still present and still sufficient to preclude substantial gainful activity. This suspension lasts as long as a beneficiary is actively using a Ticket by pursuing the goals established in his/her individualized work plan.

  • Removal of work activity as a trigger for a medical disability review for long-term DI beneficiaries, which means that these beneficiaries (regardless of participation in TTW) can seek work without fear that their engaging in work will lead SSA to conduct a review of their disability status.

  • Extended eligibility for Medicare continues coverage for DI beneficiaries who return to work for an additional 54 months beyond what was available before (from 39 to 93 months). In addition, when that extended coverage expires, beneficiaries can purchase Medicare coverage.

  • Medicaid Buy-In program provisions in the Ticket Act make it easier for states to establish programs that let people with disabilities purchase Medicaid coverage on a sliding-fee basis. Currently 28 states have such programs.

Thus, the TTW program seeks to increase the rate at which disability beneficiaries exit the rolls due to work by adding two payment systems with stronger performance incentives that expand beneficiaries’ choices for service providers and reduce some of the work disincentives. TTW is being rolled out across the country in three phases, beginning in February 2002 and continuing through September 2004. As of September 2003, the program was well underway, operating in 33 states plus the District of Columbia. At that time, SSA had mailed tickets to almost 5.3 million beneficiaries, 784 providers had registered as ENs in addition to 50 SVRAs, and almost 25,000 beneficiaries had assigned their ticket to an EN or a SVRA (Social Security Administration 2003). Even at this relatively early point in its development, TTW has become one of the biggest operations ever fielded by SSA. It has required SSA to develop many new systems that were not particularly important when its mission focused primarily on paying benefits. In particular, new procedures had to be developed to recruit and register ENs, to inform and recruit beneficiaries, and to track monthly eligibility status and work activity in sufficient detail to support the milestone and outcome payments. Given the magnitude of these changes and complexity of program interactions, it will take a while before all components are working smoothly and the program can achieve its maximum effectiveness. 

 

A. EVALUATION COMPONENTS: GOALS, SCHEDULES, AND PRODUCTS

Given the importance and complexity of this new program, Congress mandated that SSA conduct a comprehensive evaluation to provide important short-term information about program implementation that will help SSA refine program operations. This feedback is particularly important to SSA as it proceeds through the difficult initial stages of program implementation. The evaluation will also provide information about the long-term effects of TTW on beneficiaries’ employment, earnings, and benefit receipt, which in turn will help SSA and Congress assess the extent to which TTW meets its goals.

In addition, the evaluation will provide important information for SSA’s ongoing policy development. The evaluation will analyze the records of millions of beneficiaries and survey thousands of beneficiaries and TTW participants during the next five years. In doing so, it will provide detailed information about the work behaviors and attitudes of beneficiaries, and identify the ways they get information about Social Security programs and the labor market. This information will enable SSA to tailor the TTW program and future initiatives to more effectively reach beneficiaries and help them to achieve their full employment potential. At the end of May 2003, SSA contracted for the full evaluation, although substantial evaluation activity had occurred under an earlier design contract. In particular, the full evaluation follows the design developed by Stapleton and Livermore (2002) and builds on the preliminary process analyses done by Livermore et al. (2003).

As specified in the design, SSA has established seven major priorities for the TTW evaluation (shown in Table I.1). The evaluation will use three types of data to address these priority questions: (1) extensive qualitative data about TTW operations to be collected through document review, on-site interviews, telephone interviews, and focus groups; (2) longitudinal SSA administrative data for millions of beneficiaries plus Rehabilitation Services Administration data that will be matched to SSA records; and (3) a set of surveys that includes both repeated cross-section surveys of disability program beneficiaries and longitudinal surveys of TTW participants. In addition, the process analysis will help to identify ways to improve TTW operations and will also provide information that will help interpret findings from the participation, impact, and adequacy of incentives analyses.

The evaluation will conduct the following four analyses.

1. Process Analysis The process evaluation will rely on both administrative and survey data combined with qualitative data from site visits, telephone interviews, and focus groups with the TTW Program Manager (a contractor hired by SSA to help implement TTW), SSA, ENs, and other providers who choose not to become ENs. It will document how TTW is being implemented, assess how the program affects the market for employment-related services, and provide contextual information to help interpret impact analysis findings. It will also assess the implementation and ongoing operations of TTW over the 2003–2007 period, building on the information collected earlier by Livermore et al. (2003). The broad issues to be addressed include the following:

  • How is TTW being implemented and what are the issues and/or problems faced by SSA, the Program Manager, ENs, beneficiaries, and other stakeholders?

  • What changes in the program policies and operations have been made since inception?

  • Does TTW improve the supply of employment-related services to beneficiaries and, if so, in what manner? If not, why not? Also, what factors influence service providers’ decisions to participate in the TTW program?

  • Does TTW expand or change the use of employment-related services by beneficiaries and, if so, in what manner? If not, why not?

  • Are intermediate program outcomes and goals being achieved? What external factors affect outcomes? How might outcomes be improved?

Table I.1:  Evaluation Priorities, Components, and Data

 
Analyses
Data Sources
Process
Participation
Impact/ Outcomes
Adequacy of Incentives
Qualitative Data
Survey Data
Administrative Data
Priority Questions for the Evaluation
 
1.     Did TTW significantly reduce dependence on SSA benefits through increased employment and earnings?
   
  
Check
   
  
  
Check
2.     What was the impact of TTW on earnings, employment duration, SSA benefits, and beneficiary income?
  
  
Check
   
  
  
Check
3.     Did TTW produce net SSA program costs or savings? How much? What are costs and benefits of the TTW program to SSA?
  
  
Check
   
  
  
Check
4.     Who did and did not participate in TTW?
  
Check
   
  
  
Check
Check
5.     What groups were adequately served under the TTW program, and what groups were underserved?
  
  
  
Check
Check
Check
Check
6.     What aspects of the program improved or reduced program success?
Check
   
  
  
Check
Check
Check
7.     Did TTW produce net social costs or benefits? What were the social costs and benefits of the TTW program?
  
  
Check
   
Check
Check
Check

 

2. Participation Analysis The participation analysis will rely on administrative and survey data to answer the broad questions: how many beneficiaries participate in TTW, what are their characteristics, and what are their reasons for nonparticipation? More specifically, the analysis will address the following questions:

  • How do beneficiaries learn about the TTW program? Do they generally understand the opportunities it offers?

  • How many beneficiaries participate in the TTW program?

  • What are the characteristics of individuals who do and do not participate in the TTW program and how do these characteristics relate to participation?

  • How do potential participants’ attitudes and perceptions of the TTW impact whether or not they participate in the program?

  • What are the most effective sources of information about TTW for potential participants?

  • How common is involuntary nonparticipation, that is, instances where a beneficiary seeks services but their ticket is refused by an EN?

3. Impact/Outcome Analysis This evaluation component will address SSA’s three top evaluation priorities as highlighted in Table I.1:
  • Did TTW significantly reduce dependence on SSA benefits through increased earnings?

  • What was the impact of TTW on earnings, employment duration, SSA benefits, and beneficiary income?

  • Did TTW produce net SSA program costs or savings? How much? What are costs and benefits of the TTW program to SSA?

The evaluation will estimate program impacts using a design that compares outcomes for TTW participants with outcomes for similar beneficiaries who do not participate. The major challenge of the evaluation is to select comparison beneficiaries who behave as the participants would have in the absence of the TTW program. To meet this challenge, the evaluation will use a variety of analytic approaches, each using a specific comparison group and statistical methodology to assess the extent to which observed differences between participants and the comparison group members are attributable to TTW. These approaches include comparisons within states of similar beneficiaries before and after TTW rollout, contemporaneous comparisons during the rollout period between beneficiaries in states where Tickets are available and those where Tickets are not yet available, and within-state comparisons between beneficiaries who receive Tickets in early mailings and those who receive Tickets in the last rounds of mailing (Stapleton and Livermore 2002).

The impact analysis will be conducted within the context of a general model that will be flexible enough to accommodate all reasonable and defensible analytic approaches. This model will generate impact estimates that, in effect, will be weighted averages of the estimates that would be generated by the specific approaches. These will constitute the evaluation’s “benchmark” estimates of the TTW program’s impacts. By placing restrictions on the general model, we will also be able to produce impact estimates that correspond to each specific analytic approach. This approach provides a set of plausible estimates, rather than a single inherently uncertain estimate. In doing so, it tries to give policymakers a good sense of the effects TTW produces, particularly when the impact estimates are interpreted in the context of the detailed operational and contextual information gathered in the process evaluation and participation analysis.

Most of the estimates will be based on analysis of SSA administrative data. Survey data will be used to measure outcomes that are not measured in the administrative data (e.g., hours worked, wage rates, fringe benefits, and satisfaction with EN services). Because the number of people included in the surveys will be far fewer than those captured in the administrative data, and because we could not conduct the surveys before TTW’s rollout, we will not be able to estimate impacts on such outcomes; instead, we will focus on describing outcomes and trying to understand beneficiaries’ perspectives of the TTW program and work.

4. Adequacy of Incentives Analysis The adequacy of incentives analysis will draw on the process, participation, and impact/outcome analyses. In essence, the evaluation will examine many of the issues previously described, focusing specifically on the subgroups of beneficiaries that are expected to have a particularly difficult time accessing services through the TTW program. This subgroup, as defined in the Ticket Act, includes individuals who need ongoing support and services in order to maintain employment, individuals who require high-cost job accommodations, individuals who earn a sub-minimum wage, and individuals who work and receive partial cash benefits. The Ticket legislation requires SSA both to assess whether the program includes sufficient incentives to encourage ENs to work with these groups of beneficiaries and to consider program modifications that might improve services for these individuals. The evaluation will focus on the first of these requirements and provide SSA with information that will help the agency address the second. Some of the key questions for this evaluation component include the following:

  • How many beneficiaries are in these four Adequacy of Incentives (AOI) groups, and what are their characteristics and program benefits?

  • Are ENs willing and able to work with beneficiaries in the AOI groups?

  • What employment services do beneficiaries in these groups receive, with and without Tickets?

  • To what extent does TTW affect employment and program outcomes for these beneficiaries?

  • To what extent does TTW affect net program costs for these beneficiaries?

  • To what extent do non-SSA programs serve these beneficiaries, and how does TTW affect those programs?

B. PURPOSE AND ORGANIZATION OF THIS EVALUATION REPORT

This is the first report from the evaluation. Written using data collected within four months after the effective date of the contract, its primary goal is to provide SSA with formative information on program implementation. The issue of program effectiveness will be addressed in subsequent reports. This report begins by reviewing the basic structure of the TTW program and its legislative and programmatic context (Chapter II). It then turns to the early TTW implementation experience, examining the many procedures and policies SSA developed for this new program and the early operation experience of SSA, the Program Manager and ENs (Chapter III). Then the report presents current statistics on the number of tickets that have been mailed, accepted by ENs or SVRAs, and the characteristics of beneficiaries who are using their Tickets (Chapter IV), which is followed by case studies of experienced ENs that have served substantial numbers of beneficiaries under the TTW program and that were interviewed earlier in the preliminary process analysis (Chapter V). The report then turns to the important issue of the adequacy of incentives, using a mixture of updated process information and statistics on ticket assignments (Chapter VI). The report concludes with two chapters that identify the operational issues that deserve further consideration (Chapter VII) and describe the remaining data collection, analysis, and reporting activities of the evaluation (Chapter VIII). Appendices provide more detailed statistics on ticket activity in the 33 states where TTW is currently operational, more details about the case-study ENs, and details about how we identified beneficiaries in the AOI groups.

Notes:

1Other than the suspension of medical disability reviews, these provisions are available to beneficiaries regardless of whether they are using a Ticket. Return to text.