Disability Research Home

What's New

Major Activities

Demonstrations

Projects & Studies

Research & Data

Occupational Information System Project

Work Incentive Policy

Public Use Files

Completed Projects

More Information

About Us

Red Book

BPQY Handbook

Organization Chart

Related Links

Questions & Comments

CHAPTER 6
Adequacy of Incentives Study

In passing the Ticket Act, Congress acknowledged that the TTW program might not be equally accessible to all disability beneficiaries. Of particular concern was the possibility that the performance-based payment system might lead providers to serve mainly beneficiaries who are most ready to return to work while largely ignoring beneficiaries requiring more intensive or long-term support if they are to become successfully employed. Such client selection practices could create an efficient program in the sense that payments made to ENs could be offset by savings from selected beneficiaries exiting from the disability rolls. But the practices could also create an inequitable program in the sense that some beneficiaries who want to work may be unable to obtain TTW-financed services that would enable them to succeed.

To address the equity issue, Congress mandated an Adequacy of Incentives (AOI) study to evaluate how the TTW program can be used to increase employment among beneficiaries with significant support needs. The Ticket Act specifies four groups of beneficiaries that could find it difficult to obtain services in the performance-based TTW environment:

1. Individuals with a need for ongoing support and services
2. Individuals with a need for high-cost accommodations
3. Individuals who earn a subminimum wage
4. Individuals who work and receive partial cash benefits

The statute requires SSA to identify and implement a payment system that would allow the four groups of beneficiaries to participate in the TTW program. The Commissioner is mandated to report to Congress on recommendations for a method or methods of adjusting payment rates to ENs to ensure equitable participation among the above groups. Furthermore, the Commissioner must implement the necessary adjusted payment rates before full implementation of the TTW program, which is to occur by October 2004.¹

The AOI analysis will draw on all aspects of the TTW evaluation to address the following topics and questions:

• Process. How do the structure and operations of TTW affect access to TTW-financed services for the four AOI groups? In particular, to what extent are participating ENs capable of serving AOI groups, to what extent do they adopt beneficiary screening procedures that would likely exclude many in the AOI groups, and are beneficiaries in the AOI groups effectively informed of their options?
• AOI Group Characteristics. What are the characteristics of the AOI groups, particularly those that might influence an EN’s willingness to accept a Ticket assignment from them? How do the characteristics of the AOI groups compare with those of beneficiaries outside the AOI groups?
• Relative TTW Experience. Does the rate of Ticket assignment, ultimate work success, long-term benefit receipt, and other aspects of the AOI groups’ experiences with TTW differ from those of other beneficiaries?
• Relative TTW Effect. Does the extent to which TTW changes beneficiary outcomes differ between members of the AOI groups and other beneficiaries? If so, do clear factors account for the differential effects?

At this early stage of the evaluation, we can already see ways in which some beneficiaries may be excluded from full participation in the TTW program. In particular, the fact that many of the most experienced providers are losing money on their TTW activities suggests that the current system provides little incentive to serve job-ready beneficiaries, let alone those who need ongoing supports or high-cost job accommodations. In addition, considerable anecdotal evidence indicates that many ENs are using screening criteria that exclude beneficiaries who are interested in receiving partial benefits while they work.

We have also begun to analyze administrative data on beneficiaries in the AOI groups. The early analysis uses SSA administrative information about beneficiaries’ primary disabling conditions to identify beneficiaries in two of the AOI groups: those who need ongoing supports and those who need high-cost accommodations. This preliminary definition is clearly an approximation because accurate classification requires information on individuals’ functioning, expectations, educational background, and previous work activity; it cannot be based entirely on their disabling conditions. Nevertheless, this preliminary definition allows us to initiate the analysis based on the currently available administrative data.

Accordingly, we find that beneficiaries in the AOI groups constitute a majority of all eligible DI and SSI beneficiaries. The finding is not surprising given that the preliminary definition places all beneficiaries with mental illness, mental retardation, or other mental disorder into the AOI groups and that beneficiaries with those diagnoses account for about 38 percent of all beneficiaries. What is surprising, however, is that beneficiaries in our preliminary AOI groups also account for the majority of beneficiaries using Tickets and that they have higher participation rates than those beneficiaries not included in our AOI groups. Most of these beneficiaries are being served by SVRAs under the traditional payment system, thereby underscoring the agencies’ obligation to try to serve all applicants. It also reflects the substantial diversity among beneficiaries with the same disabling conditions and illustrates the need to refine our definitions further as more detailed data become available.

We discuss the findings in more detail starting with information primarily from the preliminary TTW process analysis (Livermore et al. 2003) and from our follow-up interviews with eight experienced ENs. We then turn to a statistical description of the characteristics and experiences of the two AOI groups that can be identified using our preliminary definition.

The most obvious process analysis conclusion about the adequacy of incentives is that the most experienced ENs do not appear to be earning a profit from their TTW activities. As noted in Chapter V, all eight of the experienced ENs we interviewed for the report indicated that they were losing money. Without other sources of income or changes to the payments and payment process, these ENs seem likely to contract their operations further or drop out of the program altogether. If these ENs are indicative of all ENs, then the current TTW system clearly does not provide enough of an incentive for ENs to remain in the program in general, let alone serve individuals perceived as requiring costly, long-term supports. SSA has taken steps to make the payment system more efficient, but it remains to be seen if those steps alone will be sufficient to keep the current ENs participating. Thus, it seems likely that the current system does not provide enough of an incentive for ENs to serve those beneficiaries who require substantial supports or who currently earn subminimum wages.

In addition, the available process information suggests that many ENs, as well as some SVRAs acting as ENs, have established screening and intake procedures that could exclude individuals in the AOI groups from participating in the TTW program under the new payment options. Such screening is entirely consistent with the incentives established by the program. ENs can only recoup the costs of serving Ticket holders and possibly make a profit if the beneficiaries they serve succeed in the labor market. Specifically, beneficiaries must achieve employment at levels that will generate a payment stream from SSA that exceeds the costs of assisting them. For beneficiaries requiring high-cost services, employment success means that they work full time at a job above minimum wage and remain employed well past the point at which cash benefits go to zero. Thus, we expect ENs to use various screening mechanisms to identify eligible beneficiaries who will (1) cost relatively little to serve and (2) demonstrate the best chance of meeting their employment goals.

Chapters III and V presented several examples of ENs’ screening practices intended to identify the most promising candidates. Some ENs ask beneficiaries questions such as, "Are you interested in full-time employment?" or "Are you interested in going off cash benefits?" A "no" to either question might lead an EN to decide against serving a beneficiary. Such screening questions clearly try to exclude members of the fourth AOI group—those who want to work while continuing to receive partial cash benefits.

In addition to a beneficiary’s willingness to work full time and move off cash benefits, ENs may consider several other factors in determining whether to accept an individual’s Ticket. Given that a beneficiary must be in zero-cash benefits status for up to 60 months in order for the EN to receive the maximum payment under either the outcome-only or milestone-outcome payment system, some ENs hesitate to accept Tickets from beneficiaries who may have difficulty maintaining employment for extended periods without intensive ongoing supports. For example, some ENs have indicated reluctance to serve individuals with cognitive or psychiatric disabilities, traumatic brain injuries, or other conditions that the ENs perceive as decreasing the person’s odds of remaining in zero-cash benefits for a substantial period. Such selectivity probably has its greatest impact on members of the first AOI group, those with a need for ongoing supports and services.

Some ENs are also reluctant to accept Tickets from individuals in the second AOI group, those perceived as needing high-cost accommodations. Beneficiaries who require expensive assistive technology, for example, may have relatively more difficulty locating an EN that will accept their Tickets. Because ENs appear to have difficult generating profits from their Ticket activities, they are likely to be very reluctant to accept Tickets from beneficiaries that may require above average resource to place on jobs (which could be the case if the EN had to pay for an expensive accommodation) or that appear to have a below average chance of obtaining employment (which could be the case if the EN thought potential employers would have to pay for the accommodations). Thus, it is not surprising that many ENs appear to target their services to beneficiaries whom they believe are able to enter employment without requiring high-cost accommodations.

Some ENs also carefully consider a beneficiary’s educational history and employment experience when deciding whether to accept a Ticket. Individuals whose work history is exclusively or primarily limited to subminimum wage jobs—those in AOI group 3—may be viewed as lacking the productive capacity to achieve sufficient earnings to generate Ticket payments.

Initial rejection by an EN does not necessarily mean that an AOI group member would never be able to participate in the TTW program. It appears that non–SVRA ENs commonly refer candidates that they perceive will require extensive services to SVRAs, where they are more likely to be served. Indeed, of the eight experienced ENs we recently interviewed, the Oklahoma Department of Rehabilitative Services (DRS), an SVRA, was the only one with a policy of accepting Tickets from all interested beneficiaries. A major factor influencing the Oklahoma SVRA’s policy of accepting AOI group members was its substantial level of funding from outside the TTW program. As explained in Chapter V, Oklahoma had a source of operating revenues that helped it cover service costs for clients who might not generate Ticket payments for quite some time (if at all). Most non–SVRA ENs are not in the same financial position.

The process study did reveal that one experienced non–SVRA EN, Marriott’s Bridges Program, focuses on serving beneficiaries in one of the AOI groups, specifically clients with ongoing support needs. The Marriott EN had relied primarily on funding from the Workforce Investment Act and saw TTW milestone payments as a way of modestly supplementing the types of services it had long provided to its target clients. In this case, TTW funding did not appear to increase the number of AOI group members served by the organization. Rather, when beneficiaries managed to generate one or two milestone payments, the funds enabled Bridges to provide additional services it had not previously offered. Although this EN’s effort to serve beneficiaries in an AOI group is encouraging, readers should recall that Bridges was experiencing substantial financial difficulties, with revenues falling far short of costs.

In summary, early process information shows that the TTW program can give SVRAs and other providers that have outside funding an additional incentive to serve disability beneficiaries, including in some cases, those in one or more of the AOI groups. But ENs that try to rely solely on TTW payments, the new payment systems do not appear to provide much incentive to serve beneficiaries in general, let alone those in the AOI groups. After all, if ENs are currently experiencing financial difficulties while service clients who have been screened as relatively easy to serve—those who appear job ready and not in need of costly accommodations or supports—then it would seem unrealistic to expect that they would make a concerted effort to serve beneficiaries in the AOI groups.

B. CHARACTERISTICS OF BENEFICIARIES IN TWO AOI GROUPS

The AOI analysis depends on the ability to define those beneficiaries who might be accurately categorized into a specific AOI group. Only then can the process analysis interviews, site visits, and focus groups concentrate on the relevant beneficiaries and quantitative analyses to investigate properly the participation and outcomes of these beneficiaries.

In developing an analysis of the AOI groups, we have to solve two issues. First, the Ticket Act does not define the specific AOI groups in a way that allows accurate identification of beneficiaries using SSA administrative data. Second, SSA administrative data do not enable us to differentiate between those who do not participate because they are not interested in doing so (voluntary non-participants) and those who would like to participate but cannot find and EN willing to accept their Ticket (involuntary non-participants). If many beneficiaries in the AOI groups possess such severe health conditions or impairments that they feel work is not possible or desirable under any reasonable circumstances, then the evaluation will substantially overstate the extent to which the needs of AOI beneficiaries are not met by TTW. Fortunately, the evaluation surveys, which will start in January 2004, will provide more detailed information for identifying AOI groups and assessing their knowledge, attitudes, and expectations about work.

Until the survey data become available, the evaluation must face the challenges involved in using administrative data or other empirical approaches to identifying the AOI groups. Using available SSA data and the definitions developed in the evaluation design (Stapleton and Livermore 2002), we developed preliminary definitions of the first two AOI groups: beneficiaries who require ongoing supports and those who require high-cost accommodations. These preliminary definitions are based solely on beneficiaries’ primary disabling conditions as recorded in the SSA data and are mutually exclusive. In particular, the definition of the group that requires ongoing supports includes impairments that are likely to result in:

• A frequent need for personal assistance or coaching (e.g., cognitive disabilities, autism, other developmental disabilities, traumatic brain injury, other severe cognitive disorders, quadriplegia)
• A tendency to be able to work only episodically (e.g., psychiatric disorders)
• Possible disruptions of a person’s work activity (e.g., uncontrolled seizure disorders)
• Gradual reduction of an individual’s functional capacity over time so that long-term employment retention may be difficult (e.g., multiple sclerosis, degenerative arthritis)

The intent of the definition of the group that requires high-cost accommodations is to include beneficiaries who require supports such as assistive technologies, workplace modifications, job coaching, personal assistance services, and interpreter or reader services. It includes impairments that result in the inability to use two or more limbs, severe neurological impairments (e.g., spinal cord injuries), deafness and severe auditory impairments, and blindness and severe vision impairments. Appendix C details the relevant sections from SSA’s lists of impairments used to construct these definitions, along with the associated SSA impairment codes.

Using administrative data about other beneficiary characteristics such as educational and employment history, we will eventually refine and expand our preliminary definitions. For example, we could use longitudinal SSI earnings data or even historical FICA tax reports to assist in the identification of individuals who have worked for subminimum wages or otherwise had very low earnings once those SSA records become available to the evaluation in 2004.² We will also use the survey data to develop more precise definitions. In addition, although the groups identified in accordance with the preliminary definition are mutually exclusive, it is likely that a more detailed analysis will find some types of beneficiaries who are in more than one AOI group.

Despite the limitations of the preliminary definitions, they enable us start the analysis and provide a basis for making some initial observations about the extent to which TTW services are available to two of the AOI groups.

First, the preliminary definitions indicate that beneficiaries in the first two AOI groups constitute a majority of all Ticket beneficiaries (Figure VI.1). Of the nearly 5.1 million Ticket-eligible beneficiaries in the Phase 1 and 2 states, about 54 percent can be defined as needing ongoing support and services, and another nearly 8 percent can be defined as needing high-cost accommodations. As noted, the finding is not particularly surprising given that the definition includes disabling conditions that account for a large share of SSA beneficiaries. In particular, by including all beneficiaries with mental illness, mental retardation, and other mental impairments in the group requiring ongoing support, the definition places 38 percent of all beneficiaries in the first AOI group. Similarly, the definition of beneficiaries in the second group includes all blind and deaf beneficiaries as well as those with severe neurological impairments. Beneficiaries with these impairments account for almost 8 percent of eligible beneficiaries.

Click here for Figure VI.1

Second, analyses of the beneficiaries identified with the preliminary definitions indicate some important differences between AOI group 1 (those who require ongoing supports), AOI group 2 (those who require high-cost accommodations), and all other beneficiaries (Table VI.1). For example, beneficiaries in AOI group 2 are more likely than those in group 1 or in neither of the these two AOI groups to be receiving only DI benefits and less likely to be receiving only SSI benefits. Beneficiaries in AOI group 2 are also more likely to be males than those in AOI group 1 or in neither of these AOI groups (which are essentially equally divided between men and women). The age distributions of the three analytic groups are generally similar, although members of AOI group 1 are somewhat younger. Both AOI groups were similar to non–AOI beneficiaries in terms of the very small percentage that have requested communications from SSA to be provided in any language other than English.

Table VI.1: Characteristics of AOI and Other Beneficiaries (Percent)

English

Not only do beneficiaries included in our preliminary classification of AOI groups 1 and 2 account for a majority of eligible beneficiaries, they also account for the majority of Ticket users. In addition, it appears that they have higher TTW participation rates than do other beneficiaries. In examining results for Phase 1 states only, where beneficiaries have had the most time to use their Ticket, we find that of the approximately 19,600 beneficiaries using Tickets, 71 percent were in an AOI group; 57 percent in AOI group 1 and 14 percent in AOI group 2 (Figure VI.2).

Click here for Figure VI.2

We also observed above average participation rates for both of these AOI groups (Table VI.2). Eligible beneficiaries we classified into AOI group 1 participated at a rate just above the average for all beneficiaries ((0.78 percent compared with 0.74 percent overall), while eligible beneficiaries in AOI group 2 participated at a rate almost twice as high (1.4 percent compared with 0.74 percent). We also computed the participation rate looking just at assignments to providers using one of the two new payment systems, milestone-outcome and outcome-only. Those figures suggest that beneficiaries we classified into the AOI groups participated in the new payment systems at a rate that was very low, but still greater than the rate at which other non-AOI beneficiaries participated.

The high participation rate runs counter to expectations about AOI beneficiaries who were predicted to have a difficult time locating an EN to accept their Tickets. In part, this result reflects our preliminary AOI classification system, which uses only information on disabling conditions and so cannot capture individual differences in work history, functional status, and available work supports. The evaluation will investigate this issue more thoroughly as we obtain more detailed administrative and survey data. The result may also reflect the fact that our AOI groups contain many beneficiaries, particularly those with mental illness, mental retardation, several visual impairments, and severe hearing impairments, for whom there are relatively strong advocacy and service systems designed to foster employment. Thus, these beneficiaries may find it easier to learn about the TTW program and participate than beneficiaries for whom there are less well developed support systems (for example, those with low back impairments).

 Table VI.2: TTW Participation Rates for AOI and Other Beneficiaries (Percent)

As with the general TTW program, beneficiaries classified in the two AOI groups have overwhelmingly assigned their Tickets to an SVRA (Table VI.3). Looking just at Ticket users in the Phase 1 states, 88 percent of those in AOI group 1 used an SVRA, which is only slightly larger than the percentage for all Ticket users (86 percent). However, 94 percent of beneficiaries we classified in AOI group 2 used an SVRA. Correspondingly, the AOI beneficiaries are largely served under the traditional payment system rather than under either of the two new TTW payment systems.

Table VI.3: Ticket Assignments to Different Provider Types and Payment Systems for AOI and Other Beneficiaries (Percent)

SVRA

D. RELATIVE EFFECTS OF TTW FOR AOI GROUPS

It is too early to measure the relative effect of TTW on the employment and benefit receipt of beneficiaries in the AOI groups. To give the program time to deliver the required services, to afford beneficiaries time to obtain and hold jobs, and to gather the required data we do not expect to present results on program effects until much later in the evaluation.

E. CONCLUDING OBSERVATIONS

The preliminary findings presented above encompass somewhat different perspectives on the issue of TTW services for the AOI population. The process analysis suggested that screening out individuals who may fall in one or more AOI groups may be a common practice among non-SVRA ENs, making it difficult for the AOI population to gain access to TTW services and likely limiting their choice of providers. The process findings are consistent with the concentration of Ticket-using beneficiaries in the two AOI groups in SVRAs, but the higher participation rates observed for the two AOI groups were not foreshadowed by our discussions with providers.

The high participation rates based on the preliminary definition of the two AOI groups highlight the diversity among beneficiaries with similar impairments. While the impairments used to define the two AOI groups are correlated with use of ongoing supports or workplace accommodations, many beneficiaries with those impairments may be able to work without substantial outside assistance. Thus, the results indicate that the evaluation must go beyond simple definitions based only on impairments in order to understand fully the factors that limit participation in TTW and ultimate employment success.

Furthermore, the preliminary results showing high participation rates do not imply equivalent employment success for the AOI groups. The full analysis of AOI issues must therefore wait until more follow-up data are available on employment and benefit receipt and until the estimation of program effects can be implemented.

Notes:

¹Several efforts have been completed or are underway that relate to the AOI issue. First, the Ticket to Work and Work Incentive Advisory Panel (2002) issued a report that made several recommendations concerning how AOI issues could be studied. Second, SSA sponsored an Adequacy of Incentives Advisory Group through the Disability Research Institute, which recently released a report containing several recommendations for changing the TTW program to deal more effectively with AOI groups (AOI Advisory Group 2003). Third, SSA sponsored a technical workshop on the AOI issue that resulted in the development of a book addressing topics such as the context for assessing AOI issues and lessons from other programs (Rupp and Bell 2003). As the TTW evaluation proceeds, we will consider the work of these and other groups in addressing the AOI issue. Return to text.

²The evaluation will not have direct access to FICA tax reports because of data confidentiality rules, but SSA staff will assist the evaluation by using that information to help develop more refined definitions of the AOI groups, and those definitions will be available. Return to text.