Conclusions and Implications
By the end of 2004, the third year of TTW operations, SSA had successfully implemented the TTW program as designed, but two key limitations inherent in that design have become apparent and will need to be addressed.
One limitation is the low rate of participation in TTW. Even in the Phase 1 states, where the program has operated the longest, just 1.1 percent of beneficiaries have assigned their Ticket to a provider. This participation rate reflects, in part, the paradox inherent in TTW: Its goal is to promote work among a group of individuals judged to be incapable of substantial employment, which is the very basis for receiving disability benefits from either the DI or SSI program. In reality, the vast majority of beneficiaries will not attempt to secure a job once they are on the rolls. For instance, only about 2.5 percent of any enrollment cohort will ultimately leave the rolls due to work, and less than 0.5 percent of all beneficiaries on the rolls at a point in time eventually leave due to work (Newcomb et al. 2003; Berkowitz 2003). Nevertheless, our findings from the TTW evaluation surveys suggest that 7 percent of beneficiaries see themselves earning enough to stop receiving benefits in the next year, and 15 percent see themselves doing so in the next five years. Thus, far fewer beneficiaries have participated in TTW than express an interest in achieving the earnings that would move them off the rolls. Although our knowledge about the reasons for this gap is limited, its presence suggests that it would be worthwhile to investigate ways to increase beneficiary participation.
The second limitation is the relatively low rate at which service providers actively participate in TTW. While more than 1,100 providers had registered as ENs by June 2004, 60 percent of them have not yet accepted any Tickets, and the program manager reports that it has become increasingly difficult to recruit more ENs. Our analysis suggests that this low participation rate is at least partially a result of the structure of TTW payments, which does not appear to create a financial incentive for providers to recruit and serve beneficiaries beyond what they can fund with their other revenue streams. Furthermore, interviews with SVRA staff indicate that their agencies are becoming less aggressive in pursuing Tickets than they were early in the program rollout. Without active participation by ENs and SVRAs, TTW cannot achieve its goals of increasing beneficiaries’ use of employment-assistance services and expanding the range of service choices available to them. Further program refinements may be warranted if greater provider participation is to be encouraged.
This kind of mixed success in the early stages of the program was envisioned by Congress when it passed the Ticket Act. Specifically, the legislation provided for the commissioner to assess the program as it rolled out, making changes that would help to achieve program goals more effectively (or recommending changes when legislation would be required). As we prepared this report, SSA used its Ticket Act authority to propose new regulations that seek to strengthen the financial incentives for providers to participate actively in TTW (SSA 2005). SSA has also launched an analysis of ways to increase beneficiary awareness of and participation in TTW. Future evaluation reports will examine how these efforts may be helping TTW to reach its full potential.
Beyond these general observations about the status of the TTW program in its second two years of operation (2003-2004), eight key findings emerged from the implementation analysis documented in this report that will need to be considered in the TTW review and refinement process.
A. THE INITIAL TTW ROLLOUT HAS BEEN COMPLETED; EFFORTS NOW FOCUS ON ONGOING OPERATIONS
In September 2004, SSA completed the TTW rollout with the final mailing of Tickets to eligible DI and SSI beneficiaries who were on the rolls when TTW began in their state. Nationwide, SSA mailed more than 10 million Tickets, and approximately 9 million beneficiaries were eligible to use their Ticket in September 2004. Now that the initial rollout is complete, the program has reached the point of ongoing operations: SSA, with the assistance of the Program Manager, is mailing Tickets to new beneficiaries, facilitating Ticket assignments to ENs and SVRAs, and paying providers for those beneficiaries who are meeting milestones or working enough to reduce their cash benefits to zero. In addition, SSA continues to refine TTW operations, pursue potential revisions to Ticket regulations, and enhance its broader effort to encourage and support beneficiary return to work.1
The Program Manager has implemented all systems needed to operate TTW and, as of early 2005, had recruited more than 1,300 ENs. At this stage, one of the greatest challenges to sustaining the program is to recruit and retain more ENs. Innovative marketing approaches have become necessary as recruitment and retention of ENs has become more difficult. The Program Manager also must implement the “timely progress” requirements of TTW and must continue efforts to help ENs complete payment paperwork accurately and take other measures to make the payment process more efficient.
B. BENEFICIARY PARTICIPATION CONTINUES TO GROW BUT REMAINS VERY LOW
Since mid-2002, the beneficiary participation rate in Phase 1 states has increased each month, although the increases are quite small, bringing the overall participation rate to 1.1 percent in March 2004. The participation rate in Phase 2 states has also risen continuously but had reached only 0.6 percent by March 2004.2 Participation rates are substantially higher for some beneficiary groups than others. For example, rates are higher for young beneficiaries, those with sensory impairments or some college education, and those who live in certain states or participate in DI or SSI work incentive programs, but no group has an extraordinarily high rate. Even for those with hearing impairments, the group with the highest identified participation rate, the rate is under 6 percent in the Phase 1 states.
It is important to note that the observed participation rates reflect not only a beneficiary’s decision to enter the program but also the beneficiary’s ability to find a provider willing to accept their Ticket. In other words, the rates reflect beneficiary demand for employment services as well as the supply of providers willing to accept Tickets under the current payment system. Raising the participation rate is therefore likely to involve targeting both beneficiaries and providers.
Interestingly, there is some preliminary evidence that participation is higher than average among the four groups of beneficiaries identified by the Ticket Act as those likely to have difficulty obtaining services in the performance-based TTW environment: (1) individuals with a need for ongoing support and services, (2) individuals with a need for high-cost accommodations, (3) individuals who earn a subminimum wage, and (4) individuals who work and receive partial cash benefits. To some extent, this finding reflects the fact that we imperfectly identified these groups on the basis of the available administrative data. As a result, we have probably included some individuals who do not face substantial barriers to getting services along with those who do. In our next report, we will examine participation rates based on the survey data, which will allow us to substantially refine the definitions of these groups.
In assessing participation rates, it is important to remember that there has always been some uncertainty about how many beneficiaries will participate in the TTW program. The program designers saw TTW as a way to change expectations and opportunities, thereby improving a system under which only one-half of one percent of beneficiaries left the rolls because of employment (Mashaw and Reno 1996; Berkowitz 2003). In essence, there seemed to be a sense that the old system was not effective in helping beneficiaries reach their full potential and that the new system could only make things better.
In the Ticket Act, Congress did not set out expectations for participation rates, although it did refer to the historically low rate of exits due to employment and expressed hopes for a doubling of that rate to at least one percent of beneficiaries. Thus, one way to view participation rates is whether they are high enough to generate an increase in program exits due to work. The participation rates are higher than 0.5 percent, so the program has the potential to increase access to services and thereby increase work-related program exits. Also, as noted in Chapter IV of this report, there is some suggestive evidence that TTW may have such an effect. However, any clear conclusions about this issue cannot be drawn until the impact findings, presented in the next evaluation report, are available.
C. SVRA PARTICIPATION REMAINS STRONG BUT APPEARS TO HAVE FALLEN OFF SOMEWHAT IN MORE RECENT MONTHS
All SVRAs participate in TTW, and about 90 percent of all in-use Tickets have been assigned to SVRAs. It appears, however, that the Phase II SVRAs were initially less aggressive than Phase I SVRAs about obtaining assignments from their beneficiary clients. Our analysis of SVRA data on case closures suggests that this lower TTW participation rate is not a result of the fact that SVRAs are serving fewer beneficiaries. It appears, instead, that SVRAs are making less of an effort to obtain assignments from their beneficiary clients. Initially, SVRAs were concerned that many beneficiaries would assign their Tickets to another EN after receiving extensive services from the SVRA, leaving the agency unable to recoup its costs. As this concern has diminished over time, the SVRAs may have relaxed their efforts to obtain large numbers of Ticket assignments.
SVRA data also point to a drop in the percentage of SSA beneficiary clients whose cases are recorded as being closed, with the client in competitive employment. This decline does not seem to be directly associated with the Ticket rollout because it occurred at approximately the same time in all three groups of states (i.e., Phase 1, 2, and 3). Still, the reasons for the decline are unclear. The slow economy during the observation period might explain the trend, but there is no comparable decline for SVRA clients who are not SSA beneficiaries. It is possible, however, that the economy had a greater impact on beneficiary clients, because their disability benefits gave them an alternative source of income, or because their disabilities are, on average, more severe than those in the general population served by SVRAs. This finding, coupled with the reduction in payments to SVRAs under the traditional payment program during the same period, suggests that the concerted efforts of both SSA and the SVRAs to promote better employment outcomes among SSA beneficiary clients have not achieved their intended effect. This issue will be addressed more fully in the next evaluation report, at which point there will be more detailed data on beneficiary use of SVRA services.
D. PROVIDER INTEREST IN TTW IS ON THE WANE
While many ENs continue to take Tickets, there is evidence that enthusiasm for TTW is waning. The number of ENs has grown very slowly. Some have officially dropped out, others have stopped accepting Ticket assignments, and a large share have no assignments at all. The Program Manager reports that the task of recruiting ENs has moved from “hard sell” to “almost impossible.”
The main issues cited by ENs continue to be those raised in the initial evaluation report. For instance, ENs often see little or no net financial incentive to participate, they are unwilling to substitute risky TTW funding for more stable funding from other sources, they do not perceive much beneficiary demand for the services they offer and express concern that many beneficiaries want to work at levels that would let them retain benefits, they view the payment mechanism as cumbersome, and they have trouble finding good jobs for beneficiaries because of a poor economy.
The ENs’ views of the financial incentives provided by TTW are supported by our analysis of EN costs and revenues. It appears that ENs that relied solely on TTW payments would have lost money after two years of operation. In order to break even, ENs would have to see several substantial changes: they must reduce service costs to a bare minimum, obtain payments on a much larger share of their Ticket clients (to date, just 17 percent of DI-only beneficiaries and 11 percent of SSI-only beneficiaries who assigned their Tickets in the first year after rollout generated at least one payment within the next two years), and obtain more payments for those Tickets that generate payments. This finding does not mean they will lose money in the long-term, but it does indicate that ENs must begin to see the possibility for dramatic improvements if they are to continue to participate actively in TTW. Without their participation and willingness to take the risks inherent in a performance-based payment system like that used in TTW, it may not be possible to test the program completely. If ENs can use other sources of funding to cover most of their costs, the financial picture improves, but treating TTW as just a supplemental funding source is likely to limit its ability to achieve its goals of greatly expanding the number and variety of providers who will assist beneficiaries.
SSA tried to help ENs deal with financial issues through the capitalization initiative, but this appears to have had little positive effect. ENs we interviewed generally had either not heard of it (which suggests problems with its promotion) or saw it as impractical because they lacked the ability to pursue outside funds (which suggests it was not the kind of assistance ENs wanted). SSA also has made some progress in addressing issues connected with the payment mechanisms, but these changes have yet to have a substantial impact. As of August 2004, only 11 payments had been processed under the Certification Outcomes Payment Process, which was designed and implemented to simplify and expedite the payment process. ENs we interviewed were unable to provide much feedback on this option, in part because it was relatively new at the time of our interviews in 2004. If they had heard of it at all, few of their clients would have reached the point where it became a real choice for pursuing payments. Meanwhile, the effort required for SSA to manually process many payments remains high, in large part because of the sheer complexity of the TTW, DI, and SSI program rules and the processes required to implement all those programs.
Despite signs of waning provider interest, we found that assignments to ENs continue to grow and that assignments to ENs in Phase 2 states are just as high as they were in Phase 1 states in the comparable month of the Phase 1 rollout. Further, very few Tickets have been formally deactivated. It would therefore appear that EN activity is at least not on the decline. It is possible, however, that substantial numbers of assigned Tickets have become inactive, albeit not officially. If so, we are likely to begin seeing many more deactivations in the near future as the 24-month review is instituted for Tickets assigned early in the Phase 1 rollout.
E. THERE IS LITTLE EVIDENCE THAT TTW HAS LED TO A SUBSTANTIAL CHANGE IN THE DELIVERY OF SERVICES TO BENEFICIARIES
SVRAs continue to both dominate Ticket assignments and rely on the traditional payment system for the overwhelming majority of clients. Most SVRAs report that they are serving beneficiaries who are similar to individuals served in the past and do not appear to have changed the mix or intensity of services provided. Results from the recent round of SVRA interviews confirm our earlier finding that many agencies see the TTW program as a new payment system that imposed new administrative requirements but that has little effect on current service delivery efforts. With respect to ENs, most are providing relatively low-cost services to beneficiaries due to the low rate at which TTW generates payments and the lag time between providing services and receiving payments. Only a very small number of them offer innovative services; the bulk of providers have not substantially changed their approaches to service delivery, which may reflect the fact that most ENs are using funding external to TTW to support their basic service operations.
Nevertheless, a few noteworthy changes have occurred. Some SVRAs report that their staff now have a better appreciation for the complexity of SSA’s program rules and work incentives, and for the need to understand the difficult personal and financial decisions beneficiaries are making when they attempt to obtain employment or return to work. A number of SVRAs are routinely referring beneficiaries to local BPAOs to provide them with accurate information on the effect of employment on financial and health care benefits. In addition, a few SVRAs are expanding their own capacity to provide those services through staff development activities and by identifying individuals or organizations that can effectively provide this service to beneficiaries.
F. INITIAL SURVEY FINDINGS SUGGEST THAT TTW MAY HAVE UNREALIZED POTENTIAL
Data from the National Beneficiary Survey show that there might be substantial demand for employment services among beneficiaries. In particular, it appears that almost 15 percent of disability beneficiaries expect to leave the rolls for work in the next five years. These beneficiaries represent a key target group for TTW, and the program is, in fact, reaching some in this group, as indicated by the finding that about half of the beneficiaries who have assigned their Tickets say that they expect to exit the rolls due to earnings in the next five years.
Another way to see the potential demand for TTW services is to focus on the 13 percent of beneficiaries who reported that they worked during 2003 (the reference period for the National Beneficiary Survey). While many are probably earning less than the substantial gainful activity level, they show both an interest in employment and an ability to obtain jobs. TTW (and the BPAO program) could probably help many of these individuals increase their earnings and economic self-sufficiency by helping them to understand their options and by providing them with additional supports.
Yet, because only about one percent of beneficiaries have enrolled in TTW, it appears that the program is only reaching a small fraction of the target population. The existence of untapped potential is also indicated by the estimated two-thirds of beneficiaries who report not knowing about TTW at all.
While the presence of untapped demand is encouraging, there are also reasons to be cautious. For instance, TTW cannot meet beneficiary demand if providers do not participate, and the survey data offer a sobering picture of potential EN payments. Specifically, if only half of TTW participants see themselves exiting the rolls because of work in the next five years, it seems likely that ENs will be able to generate outcome payments for fewer than half of the people they enroll. On top of that, history suggests that beneficiary expectations are much too optimistic, since less than 1 percent of beneficiaries have left the rolls for work in the past.
The challenge facing SSA as it tries to improve TTW, therefore, is to find the mix of outreach, services, and incentives that will encourage more of the beneficiaries who see themselves exiting because of work to assign their Tickets and then succeed in earning their way to financial self-sufficiency. The evaluation will help in this process by linking the survey and administrative data to determine whether there is information in the SSA administrative records that would help SSA or the ENs identify those beneficiaries who see themselves working their way off the rolls. SSA’s current efforts to refine outreach strategies may also help in this regard, as will its ongoing efforts to address implementation issues. In particular, we expect improvements to flow not only from the planned new TTW regulations but also from other initiatives designed to encourage a return to work. There could also be improvements in service delivery if ENs and SVRAs identify and implement more effective strategies, building on the experience of the few providers who have achieved at least some measure of success.
G. OBSERVED TRENDS IN BENEFICIARY OUTCOMES ARE CONSISTENT WITH TTW HAVING A SMALL EFFECT ON BENEFICIARY BEHAVIOR
The trends in three key outcomes for Ticket-eligible beneficiaries in Phase 1 states are consistent with, but do not necessarily imply that, TTW had an effect on beneficiary behavior. The findings are based on data for TTW participants and nonparticipants in the 12 months before and up to 15 months after Tickets were mailed to these individuals. For SSI beneficiaries, the trends in outcome measures for participants before and after Ticket mailings differ from those observed for nonparticipants in ways that are consistent with the new program having, to some extent, promoted employment and program exit. They suggest that TTW may have slightly increased the proportion of SSI beneficiaries who receive zero benefits and who combine zero benefits with substantial employment, and that it slightly decreased the average benefit amount. For DI beneficiaries, relative rates of growth in the outcome measures during the postmailing period for participants compared with nonparticipants provide weaker evidence that is consistent with the TTW’s intended effects.
These findings are tenuous for several reasons. Most important, we know from the survey results that many TTW participants are motivated to find work and, as a group, would probably be more successful in the labor market than nonparticipants even if Tickets had never been mailed. However, the analysis of outcome trends is based on simple analytic techniques that do not fully control for such motivational differences. Consequently, although this simple preliminary analysis may suggest that TTW had an effect, later, more rigorous analyses may fail to substantiate such a finding. In addition, the findings presented here are tenuous because they are based on data that reflect a short postmailing followup period, during which time relatively few beneficiaries assigned their Tickets and our ability to observe changes in their employment and benefits was confined to just 15 months or less. This problem will not be as severe in future analyses because they will be based on data for a longer follow-up period.
H. THE NEW FINDINGS REINFORCE THE PRESSING NEED FOR CHANGE IF THE PROGRAM IS TO REALIZE ITS POTENTIAL
Despite the reasons to be optimistic about TTW’s future, albeit cautiously so, our recent findings reinforce an important conclusion from our initial evaluation report: substantial changes to TTW will be required in order to sustain its momentum.
One place to start would be to give providers a stronger financial incentive to participate. Our analysis suggests that ENs are likely to lose money if they rely only on TTW payments to support their programs. In order to ensure that ENs continue to participate actively in the TTW program, it will be necessary to make substantial changes to several aspects of the program. In particular, it is likely to require that Ticket payment amounts be increased, payment-processing costs be reduced, and employment outcomes for participating beneficiaries be improved from their current levels. Without changes to all these factors, it seems likely that TTW will be consigned to a small, supplementary role in financing services that are for the most part paid for and shaped by other programs.
SSA might substantially boost provider interest in TTW by changing the way the new payment systems work when a beneficiary is also participating in one of SSA’s other work incentive programs. For example, the trial work period allows beneficiaries to receive benefit payments for nine months even after they are earning above the amount that would otherwise result in zero benefits, so providers cannot receive outcome payments in those months. Similarly, the Section 1619a program allows SSI recipients to earn well above the substantial gainful activity level indefinitely without losing all of their benefits—again making their providers ineligible for outcome payments, even while SSA is achieving savings through partially reduced benefits. Thus, these DI/SSI program features intended to give beneficiaries an incentive to work also make it harder for ENs to get paid. Changes that would provide partial payments to ENs when their clients are participating in these work incentive programs would benefit the ENs, but would require Congressional action to modify the TTW law. Whether they would ultimately lead to more use of TTW and increase savings to SSA will depend on the behavior impact on beneficiaries.3
Overall, it appears that TTW has yet to create a robust market that will provide employment support services to disability beneficiaries. There seems to be substantial latent demand for these services, as evidenced by the 15 percent of all beneficiaries who express intent to obtain sufficient employment to leave the rolls. But that demand has yet to be translated into actual Ticket assignments; overall beneficiary awareness of and participation in TTW remains low. With regard to the supply of services, all SVRAs participate in TTW, but other providers have not embraced the program. Recruiting ENs has become more difficult over time, and most of those that have registered are not accepting Tickets. Furthermore, it appears that those ENs that have accepted Tickets must subsidize their TTW efforts, because the flow of Ticket payments to ENs falls far short of the estimated cost of delivering services. Thus, the market envisioned by TTW remains frail and will need a substantial infusion of energy, along with changes to its basic structure, in order to maintain beneficiary and provider interest. Such an infusion has significant risks—we cannot tell in advance if benefit savings will pay for the added TTW costs—but we see no other way to interest entrepreneurs, large and small, in building this market.
If the market can be sustained long enough, it might be possible that exits for work will increase in the future for several reasons. First, if the economy improves, beneficiaries may have more to gain from working than from receiving benefits, and providers may find it easier to find them good jobs. Also, it appears that it may take a substantial period of time before major effects will be seen. Emerging research on the SVRA system suggests that it often takes two or more years for beneficiaries who are receiving services to leave the rolls. We are just starting to reach that point for the TTW participants, so there is some hope that more participants will start to leave the rolls and generate payments for providers. In addition, SSA plans to test benefit offsets that will reduce work disincentives for DI beneficiaries. That might provide a significant boost to demand for EN services under TTW, especially if the payment system is revised to reward partial benefit reductions. Last, TTW may benefit as non-SSA efforts to improve employment outcomes for people with disabilities (e.g., the Medicaid Buy-In and Disability Navigator programs) are expanded and refined.
In the long term, the greatest effect of the TTW program may be to have helped change expectations about the potential for many people with disabilities to work if they have appropriate supports. A change in the expectations of all stakeholders—from beneficiaries, families, and friends, to SSA staff and staff in other federal and state agencies that serve beneficiaries, to the many professionals who work with beneficiaries—may ultimately promote more work and greater economic self-sufficiency among people with disabilities, even if the specific features embodied in the current TTW program do not reach their full potential.
1 After this report was drafted, SSA published proposed revisions to TTW payment rules that would substantially increase payments to ENs. We will discuss these new rules more completely in our next report. Return to Text.
2 We do not analyze Phase 3 participation rates in this report, because at the time we conducted the analysis we only had data from the first 5 months of the Phase 3 rollout. Return to Text.
3 While the proposed new rules SSA published after this report was drafted would substantially increase payments to ENs, they do not currently try to streamline the payment process further. Assessing providers’ reactions to this new mix of incentives will be a major issue in future evaluation reports. Return to Text.