III. Beneficiary Participation in Ticket to Work

Ticket to Work participation rates (the number of Tickets in use divided by the number of Ticket-eligible beneficiaries) have continued to rise each month since the early months of program rollout. However, participation rates in the Phase 2 and 3 states lag behind those observed at comparable points of TTW rollout in the Phase 1 states. Most of the difference is attributable to differences in assignment rates to SVRAs; participation rates at ENs are similar in all phases.

As of December 2004 (the last month for which we have complete data), the participation rate in Phase 1 states had risen to 1.4 percent, up from the 1.1 percent for March 2004 ( Thornton et al. 2006). This overall level of participation is well below the 5 percent rate used as a standard in the preliminary evaluation design report (Stapleton et al. 2003). However, introduction of TTW did appear to have a positive impact on beneficiary receipt of employment services (see Chapter XIII). It is also important to recognize that the Ticket participation rate is much lower than the percentage of beneficiaries who obtain services from SVRAs, because SVRAs are not obtaining Ticket assignments from all the beneficiaries they serve. In fact, based on RSA data that have been matched to SSA administrative data, 4.6 percent of all Ticket-eligible beneficiaries in Phase 1 states received services from SVRAs or ENs in the first year of Ticket rollout. ¹ This is partly because many beneficiaries served by SVRAs started to receive services before Ticket rollout, and partly because SVRAs did not obtain assignments from a large share of those beneficiaries starting to receive services after rollout.

It is also worth noting that TTW participation does not have to be as high as five percent to achieve the modest goal expressed in the Ticket Act: doubling the number of beneficiaries who leave the rolls because they find work. As only about one-half of one percent of beneficiaries were exiting due to work at the time of the Ticket Act’s passage, the goal could be attained by inducing as little as one-half of one percent of beneficiaries to assign their Ticket and exit the rolls. Thus, the current participation rate does not necessarily imply that TTW is falling far short of the act’s goal. Of course, it also does not imply achievement of that goal. Chapter XIII presents more direct evidence concerning the early impact of TTW on program exits.

For the first time we are able to report statistics on participation in Phase 3 states; December 2004 is the 13th month after the first Phase 3 rollout month and the 3rd month after completion of the initial Phase 3 mailing. At this stage, the overall participation rate in the Phase 3 states is between the participation rates in the Phase 1 and 2 states at the comparable stage of the latter’s rollouts. The Phase 3 EN participation rate at this stage is virtually identical to that in both the Phase 1 and Phase 2 states at the comparable stage. The only substantial source of variation in participation rates across the three phase groups at the comparable stage is variation in SVRA participation rates.

We now know that the relatively high SVRA participation rate in Phase 1 states resulted entirely from the relatively large number of assignments SVRAs obtained from “pipeline cases”—clients already receiving SVRA services before the start of their Ticket rollout; Phase 1 SVRA assignment rates for new clients early in the rollout were comparable to those observed more recently in Phase 2 states. This finding is based on an analysis of SVRA case closure data from the RSA that we linked to SSA data. Another interesting finding from the analysis of the linked data is that, since the start of the rollout, SVRAs have not obtained assignments from a large share of their new clients who are, in fact, Ticket-eligible—perhaps over half, based on the most recent data available.

The overwhelming majority of Tickets continues to be assigned to SVRAs (91.7 percent as of December 2004), and most are assigned under the traditional payment system, which is available to SVRAs only (85.6 percent as of December 2004). We also found that the percentage of Tickets assigned to SVRAs has gradually increased after the end of each phase’s rollout, as has the percentage assigned under the traditional payment system.

We previously concluded that the high percentage of Tickets assigned to SVRAs and the high percentage assigned under the traditional payment system appear to limit the extent to which TTW represents a dramatic break from the past. If current trends continue in the Phase 1 and Phase 2 states and are replicated in the Phase 3 states, the program will become even less of a departure in the sense that TTW will be dominated by providers and a payment system that were available before the program was initiated. This point is reinforced by our interviews with several SVRA staff members (discussed in Chapter XI) who reported that the agencies have not made major changes in their service offerings or targeting.

Participation rates vary with the characteristics of eligible beneficiaries. Earlier analysis, based on administrative data only, determined that no combination of observed characteristics leads us to predict a probability of participation higher than 10 percent or so; that is, even among those beneficiaries most likely to participate, based on observed characteristics, many will not do so.

In the last report, we identified numerous other characteristics from administrative data that are at least somewhat predictive of participation. Among these characteristics, age stands out as a particularly important predictor, with participation rates declining sharply with age, holding many other variables constant. Beneficiaries with sensory impairments, and especially hearing impairments, are much more likely to participate than are other beneficiaries. The small percentage of Ticket-eligible beneficiaries participating in other DI or SSI work incentive programs also participate at relatively high rates. In addition, participation rates vary markedly across states and increase with education.

For this report, we extended the earlier analysis to include characteristics that we are able to observe in the NBS. The new analysis reinforces the findings from the administrative data analysis and shows that the following are positive predictors of participation: disability onset before age 18; no spouse or other relatives in the household; no children under age six; receipt of little or no other public or private assistance; and, for DI beneficiaries only, below-average benefits.

Additional analysis of the survey data provides some evidence that beneficiaries facing challenges to return-to-work beyond their disabilities are somewhat more likely than others to assign their Ticket to non-SVRA ENs. They include beneficiaries with limited or no work experience (i.e., SSI-only beneficiaries), older beneficiaries, those with limited education, Hispanic beneficiaries, single parents, and those living with children under age six. Somewhat counter to this finding, however, we found that those in households with incomes above 300 percent of the poverty line are more likely than others to assign their Ticket to non-SVRA ENs. Findings on the use of one of the new payment systems, versus the traditional payment system, are similar.

This chapter extends the findings presented in our initial and second evaluation reports through December 2004 (the initial report included data through August 2003, and the second report included data through March 2004). We summarize the findings from the most recent analysis, focusing on how previous findings have changed and presenting completely new findings. The major sections present updated rollout and participation statistics and discuss how participation rates, provider type, and payment type vary with beneficiary characteristics. Appendix C provides detailed statistics.

A. Rollout Statistics

1. Ticket Mailings and Eligible Beneficiaries with Tickets

The Ticket rollout was formally completed in September 2004, when SSA finished sending Tickets to existing Ticket-eligible beneficiaries in the Phase 3 states (see Exhibit III.1); all mailings after that month went to beneficiaries who became eligible for TTW after the rollout ended—mostly new SSDI and SSI beneficiaries. As of December 2004, SSA had mailed over 10 million Tickets to beneficiaries. As previously reported, SSA used a slower and more uniform schedule for the Phase 2 and 3 mailings than for the Phase 1 mailings because of difficulties experienced by providers and others in Phase 1 states in handling the large number of beneficiary inquiries generated by the mailings.²

Exhibit III.1. Ticket Mailings, by Month and Phase (in thousands)

[D]

Source: Based on July 2005 extract from SSA’s Disability Control File through September 2004. Note: Values for October to December 2004 are estimates based on preliminary analysis of a March 2006 extract. Because some beneficiaries move, the data show that a small number of Tickets sent to those we have classified as living in Phase 2 or 3 states were sent before their phase’s rollout began.

As of December 2004, there were 9.23 million Ticket-eligible beneficiaries.³ About 30 percent of them are in the Phase 1 states, another 30 percent are in the Phase 2 states, and the remaining 40 percent are in the Phase 3 states.

2. Participation Rate

The TTW participation rate is defined as the number of “in-use” Tickets (i.e., Tickets currently assigned to providers) as a percentage of current Ticket-eligible beneficiaries. At the beginning of each rollout, rates vary substantially from month to month and across phases depending on how quickly Tickets are mailed out (and hence how quickly the participation rate denominator grows) and on how providers treat pipeline cases (which affects the participation rate numerator). As time passes, however, the rate stabilizes and then changes slowly because the vast majority of those who participate in any one month were also participating in the previous month. The number of in-use Tickets can decline only if Tickets are formally deactivated, and it is likely that some beneficiaries whose Tickets are in use are not actively receiving services, are not employed, or are not otherwise seeking employment.

By December 2004, 83,568 Tickets were in use. Reflecting the rollout schedule, 45 percent of the Tickets were held by beneficiaries residing in Phase 1 states, 30 percent in Phase 2 states, and 25 percent in Phase 3 states. Participation rates continued to rise steadily. The Phase 1 participation rate reached 1.38 percent in December 2004, the Phase 2 rate reached 0.90 percent, and the Phase 3 rate reached 0.56 percent.⁴

We reported previously that the Phase 2 participation rate appears to be on a track that is lower than that for the Phase 1 participation rate, and that continues to be the case. Exhibit III.2 permits comparisons of rates across phases by showing the number of months since each phase’s first rollout month (the zero month) on the horizontal axis. In this way, we can see the participation rate for each set of states at the same point in the rollout (the comparison is complicated slightly because the Phase 2 and Phase 3 rollouts were stretched out over longer periods than the Phase 1 rollout, and some differences might be associated with seasonal factors; for example, the rollouts did not all start in the same season of the year). For instance, 25 months after the rollout started, the participation rate in the Phase 2 states was 21 percent less than the rate observed in the corresponding month for Phase 1 states (0.90 in Phase 2 states compared with 1.14 in Phase 1 states). The Phase 3 rate appears to be on a track that is between those for the Phase 1 and 2 rates.⁵ For example, 13 months after the start of the rollout, the Phase 1 participation rate was 0.63 percent, 12.5 percent higher than the Phase 3 rate, and the Phase 2 rate was 0.50 percent, 10.7 percent lower than the Phase 3 rate. States were assigned to phases in a way that tried to make those in Phases 1 and 3 generally similar to each other (Stapleton and Livermore 2003; Thornton et al. 2005). Thus, the fact that participation rates in Phases 1 and 3 exceed those for Phase 2 (at comparable points in the rollout) is not surprising. The fact that Phase 3 rates seem to lag behind those observed for Phase 1 suggests that formal beneficiary enrollment in TTW is declining, but as the evidence from the next section shows, the decline appears solely caused by changes in the extent to which SVRAs obtain assignments from their beneficiary clients.

Exhibit III.2. Participation Rate, by Months Since Rollout Start and Phase

[D]

Source: Based on July 2005 extract from SSA’s Disability Control File.

3. Participation by Provider Type

In addition to the overall participation rate, the participation rates at each of the two provider types (ENs and SVRAs) are an important indicator of program success. As in each of our two previous reports, we again note that the vast majority of in-use Tickets was assigned to SVRAs but point to important differences across the Phase 1, 2, and 3 states. The Phase 2 SVRA participation rate is substantially lower than the Phase 1 SVRA participation rate during the comparable month in the rollout, and the Phase 3 SVRA participation rate falls between the corresponding Phase 1 and Phase 2 rates in their respective comparable months (see Exhibit III.3). In fact, holding constant the number of months since the rollout began, variation in SVRA participation rates across phases essentially accounts for all of the variation in overall participation rates across phases; variation in EN participation rates across phases is remarkably small.

The difference across phases does not merely reflect the fact that the rollouts in Phases 2 and 3 were slower than in Phase 1. If that were the only reason for lower participation rates in Phase 2 and 3 states, then—holding months constant since the start of rollout—the differences would likely be observed for ENs, not just for SVRAs. In addition, we would observe no differences across Phases 2 and 3, and differences between rates for those phases and Phase 1 would narrow as time passed. Instead, differences in the EN rates are tiny, with a substantial difference between the overall Phase 2 and 3 rates, and the difference between the overall Phase 1 and 2 rates appears to be increasing rather than decreasing.

Exhibit III.3. Participation Rates, by Months Since Rollout Start, Phase, and Provider Type

[D]

Source: Based on July 2005 extract from SSA’s Disability Control File.

In each of the Phase 1 and Phase 2 state groups, the percentage of assigned Tickets in use at SVRAs has been gradually increasing since the end of the respective rollouts and appears to be leveling off at above 90 percent (93.9 percent for Phase 1 states and 92.1 percent in Phase 1 states as of December 2004). Although the SVRA and EN participation rates have been increasing in both phase groups, SVRA participation rates have been increasing more rapidly. It is too early to observe post-rollout trends in Phase 3 states, but at the end of the rollout in Phase 3 states, the percentage of assigned Tickets in use at SVRAs was just slightly below the values observed in the Phase 1 and 2 states in the comparable months in their rollouts.

Now that matched RSA and SSA data are available, we have been able to investigate variation in SVRA participation rates across phases. With the matched data, we are able to determine the extent to which SVRAs obtained Ticket assignments from entrants into their systems. Although we observe entrants only for cases closed by the SVRAs before September 30, 2004, we can note for these cases the percentage that assigned their Ticket to an SVRA in each month since the start of a phase’s rollout. If Ticket assignment is unrelated to duration from entrance to closure, the estimates will be unbiased estimates of the percentage of beneficiary entrants assigning their Ticket in each month, including cases not yet closed.

The evidence from the matched data shows that the relatively high assignment rates for Phase 1 SVRAs reflect SVRAs’ success in obtaining assignments from a larger share of pipeline cases (i.e., cases that entered SVRA service before Ticket rollout began). In fact, it appears that shortly after the rollout start, Phase 2 SVRAs were obtaining just as large a share of assignments from “new” clients—those determined eligible for services after the start of rollout—as did the Phase 1 SVRAs. The data also show that SVRAs are obtaining assignments from only a minority of the beneficiary clients they serve. The findings for new clients might change as data for later closures become available, but we think the findings for pipeline cases will not change because most such cases will have closed before September 30, 2004.

We base our conclusions on Exhibit III.4, which plots the percentage of SVRA entrants who assigned their Ticket to an SVRA, by phase and month of entry relative to the start of Ticket rollout in their phase (month zero). The analysis includes only cases closed after receipt of services and after the client’s Ticket mailing date, but before September 30, 2004. We used six-month moving averages to smooth the plots. Thus, the plotted values are the percentage of entrants who assign their Ticket and entered the SVRA system during the six-month period ending on the month plotted. The last point plotted is for those entering between January and June 2004 and closing after services by the end of December 2004. We reiterate that these series will change as new data become available because the data exclude those who entered service during the sample period and did not close by the end of September 2004. The greatest number of updates will be for those entering in the most recent months, but it is hard to predict the direction of change.

It is apparent that the Phase 1 SVRAs were much more likely to obtain assignments from pipeline cases than either the Phase 2 or 3 SVRAs. The assignment rate statistic for Phase 1 SVRAs is higher than the statistic for Phase 2 SVRAs in every pre-rollout month from -132 to -1, with the exception of months -49 to -40. The Phase 2 and Phase 3 statistics are similar for most pre-rollout months, with the exception of months -57 to -30, when the Phase 2 rate is notably higher. We do not know what accounts for the relatively high Phase 2 assignment rate for those entering service during this period.

In our second evaluation report (Thornton et al. 2006), we tentatively concluded, on the basis of more limited evidence, that the Phase 2 and 3 SVRAs were simply less aggressive than Phase 1 SVRAs about obtaining Ticket assignments from their beneficiary clients, perhaps because of lower expectations about the value of assignments relative to the administrative cost of obtaining and processing them. The new evidence suggests that this conclusion might apply to pipeline cases, but there is no clear evidence yet that it applies to new cases. We also cannot rule out the possibility that idiosyncratic differences in how SVRAs manage their long-term cases are the underlying cause of the variation in assignment rates for those cases across the three phase groups.

Exhibit III.4. Percentage of Beneficiary Entrants to SVRA Services Who Assigned a Ticket, by Phase and SVRA Entry Month

[D]

Source: Based on RSA data for closures through September 2004 matched to the Ticket Research File. Note: Statistics plotted are six-month moving averages. The sample excludes cases that closed before the individual’s Ticket mailing date, cases that closed after September 2004, and cases that closed with receipt of services. “Entrants” are those determined eligible for services, and “entry month” is the month of eligibility determination.

The above analysis does not consider the possibility that the introduction of TTW had an impact on the number of beneficiaries obtaining services from SVRAs and that the impact may have differed by phase. We examine such a possibility in Chapter XIII.

From a few months after rollout start to the end of the Phase 2 series, the assignment rate statistic for Phase 2 SVRAs is higher than for Phase 1 SVRAs. Given the incomplete data, we think it is too early to conclude that the assignment rate for new entrants in Phase 2 SVRAs is higher than in Phase 1 SVRAs, but there is certainly no indication that it is lower.

Based on the RSA data, since TTW started, the Phase 1 and Phase 2 SVRAs have typically obtained assignments from only 30 to 40 percent of the new beneficiary clients they serve. This does not count cases that close without receipt of services; not surprisingly, SVRAs rarely obtain assignments from these cases. Although the numbers might change as we obtain data about more recent case closures, it is apparent that many beneficiaries receive services from SVRAs without assigning their Ticket to an SVRA. In 2003 4.6 percent of all Ticket-eligible beneficiaries in Phase 1 states received SVRA services in 2003. Yet the SVRA participation rate in those states ranged from 0.5 in January to 0.9 in December. Clearly the SVRA Ticket participation rates in Phase 1 states over this period would have been several times larger had the SVRAs obtained assignments from all beneficiary clients served in 2003. Presumably the SVRAs do not obtain assignments for many cases because they think reimbursement from SSA is highly unlikely. For many beneficiary clients, earnings at levels sufficient to trigger payments to the SVRA under any payment system are not a reasonable goal; for some SVRA clients, the goal is something other than paid employment (e.g., the ability to function as a homemaker).

4. In-Use Tickets by Payment Type

As in previous reports, we found that assignments to the three payment systems (traditional, milestone-outcome, and outcome-only) largely mirror assignments to provider types because only SVRAs can use the traditional system. Thus, most in-use Tickets are assigned under the traditional payment system, necessarily to SVRAs. In December 2004, 86.4 percent of Tickets in the Phase 1 states were assigned under the traditional payment system, 81.2 percent of Tickets in Phase 2 states, and 81.0 percent of Tickets in Phase 3 states.⁶

Exhibit III.5. Percentage of In-Use Tickets Assigned Under the Traditional Payment System, by Months Since Rollout Start and Phase

[D]

Source: Based on July 2005 extract from SSA’s Disability Control File.

The most recent data show that the percentage of Tickets assigned under the traditional payment system has been rising very slowly in Phase 1 and 2 states since the end of the phases’ respective rollout periods (see Exhibit III.5). In the Phase 1 states, 81.4 percent of in-use Tickets were under the traditional payment system in month 12 (5.0 percentage points lower than the December 2004 value); in the Phase 2 states, the comparable figure was 75.3 percent (5.9 percentage points lower than the December 2004 value). It is too early to establish a trend for the Phase 3 states.

The pattern of participation under each of the two new payment systems varies little across the three phase groups (see Exhibit III.6). In December 2004, 0.16 percent of eligible beneficiaries in Phase 1 states had assigned their Ticket under the milestone-outcome payment system, and 0.04 percent had assigned their Ticket under the outcome-only system. Thus, 78.6 percent of Tickets assigned under the new payment systems are assigned under the milestone-outcome system. Experience to date in the Phase 2 and 3 states is similar to the Phase 1 experience at comparable points in the rollout.

Exhibit III.6. Participation Rates for New Payment Systems, by Months Since Rollout Start and Phase

[D]

Source: Based on July 2005 extract from SSA’s Disability Control File.

We have observed no appreciable change in SVRAs’ use of the new payment systems. The percentage of SVRA assignments under one of the new payment systems in Phase 1 states was essentially constant from September 2002 through December 2004, at 5 percent. The percentage is twice as high in Phase 2 states, at 10 percent, but has not changed since March 2003. In Phase 3 states, the figure has been steady at 6 percent since March 2004. Most SVRA assignments under the new payment systems are to a small number of SVRAs (see the state participation statistics later in this chapter).

5. Deactivations and Reassignments

As in earlier reports, we have examined administrative data on deactivations and reassignments to determine whether substantial numbers of beneficiaries who have assigned their Ticket are changing providers, formally withdrawing from participation, or being withdrawn. The number of deactivations has been small relative to the number of in-use Tickets since the beginning of the program and continues to be small (fewer than 3 per 1,000 Tickets in use as of December 2004), and reassignments are extremely rare (just 4 per 10,000 in-use Tickets in December 2004). However, we noted some interesting patterns as depicted in Exhibit III.7, which shows deactivations as a percentage of in-use Tickets by phase and months since rollout start.⁷

Exhibit III.7. Net Deactivations, by Months Since Rollout Start, Provider Type, and Phase

[D]

Source: Based on July 2005 extract from SSA’s Disability Control File. Note: Net deactivations are defined as total deactivations minus reassignments. Statistics before month six of each rollout are not meaningful because of the small number of assignments.

First, net deactivations are much less frequent for Tickets assigned to SVRAs than for Tickets assigned to ENs (0.13 percent versus 1.22 percent in December 2004). Second, net deactivations from ENs for beneficiaries in Phase 1 states were relatively high during the period 8 to 15 months after the start of the Phase 1 rollout (from October 2002 through May 2003), peaking at 6.0 percent in March 2003. As discussed in our first report, a number of large ENs consolidated or terminated their operations during this period.⁸

After that period, net deactivations in Phase 1 states hovered around 1.5 percent until month 26 and then increased to around 2.0 percent for four consecutive months before falling to the 1.0 percent level or lower after month 30. The small increase for months 26 through 30 may reflect deactivations initiated by providers as the first cohort of Ticket participants reached the 24-month point in cohort assignments, after which providers are required to deactivate the Tickets of participants who are not making timely progress. The administrative data show no evidence of an increase in deactivations by Phase 1 SVRAs after month 24, although our interviews with the Program Manager indicate that some Phase I SVRAs did begin deactivating Tickets in response to letters that the Program Manager sent out in November 2004 (month 34). The Program Manager also reports that, after the initial flurry, further requests for timely progress reports went ignored, particularly by Phase 2 and 3 SVRAs, as there was little consequence for failure to comply. No response to the letter is interpreted as affirmation that the person is making timely progress. SSA’s proposed changes to the regulations drop the timely progress requirement.

As pointed out in our previous report, we do not know how many Tickets classified as in-use are actually inactive. Ticket users who halt their return-to-work effort have little motivation to withdraw their Tickets, and, as indicated above, providers have little incentive to take the initiative to do so themselves. Hence, we have to conclude that some—and possibly many—in-use tickets are inactive. It appears, however, that the vast majority of participants are engaged in some form of employment or employment preparation activity. Chapter VII presents tabulations on the employment and employment plans for Ticket participants from a survey conducted in 2004. As of that time, all but 5 percent of participants were employed, seeking work, or planning to seek work in the not too distant future (see Exhibit VII.1).

6. Participation Rates by State

Participation rates continue to vary by state. Part of the variation results from the phased rollout, but variation is high even within phases. Vermont, a Phase 1 state, continues to exhibit the highest participation rate; since March 2004, the state’s participation rate has more than doubled, to 5.6 percent (Exhibit III.8). In contrast, Massachusetts, a neighbor of Vermont, has the lowest participation rate among Phase 1 states, at 0.6 percent. Only Delaware has a participation rate that is even half the size of Vermont’s, at 2.8 percent.

Exhibit III.8. Ticket Participation Rates by State, Provider Type, and Payment Type, December 2004

[D]

Note: Based on July 2005 DCF data. Source: Based on July 2005 extract from SSA’s Disability Control File.

South Dakota’s participation rate, 2.7 percent, is also remarkable. Even though, as a Phase 2 state, its rollout started almost a year later than the rollout for the Phase 1 states, its participation rate is tied for third highest with Wisconsin, a Phase 1 state. At 1.5 percent, Idaho has the highest participation rate among Phase 3 states and is ranked eighth among all states (tied with Michigan).

It is apparent from the left side of Exhibit III.8 that variation in state participation rates is largely driven by SVRA participation rates; with few exceptions, a large majority of assignments in each state are assignments to SVRAs. The Phase 1 states with the highest EN participation rates are Arizona (1.6 percent) and Wisconsin (1.2 percent); for Phase 2, the highest rates are for the District of Columbia (0.4 percent) and Nevada and Tennessee (each at 0.2 percent). In Phase 3, the EN participation rate for the Virgin Islands, at 0.6 percent, stands out. (It is worth noting that the Virgin Islands do not have an SVRA).

With few exceptions, cross-state variation in use of the three payment systems is closely related to variation in SVRA participation, as is evident in the right side of Exhibit III.8, and is not surprising given the preponderant use of the traditional payment system by SVRAs. Vermont again stands out, with a participation rate of 1.7 percent under the Outcome-only payment system (29.7 percent of assignments in the state)—all of which are at the state’s SVRA. Thus, given the number of eligible beneficiaries in the state, Vermont’s SVRA is not only obtaining a particularly large number of assignments, but it is also using one of the new payment systems relatively frequently. Participation under the milestone-outcome system was exceptionally high in Oklahoma, where the SVRA accepts a relatively large number of beneficiary clients under that system (the Oklahoma SVRA has more experience with the TTW payment systems because it ran a demonstration program that was similar to TTW as part of the earlier State Partnership Initiative). In the Phase 2 group, use of the new payment systems is highest in Indiana, Connecticut, Louisiana, and the District of Columbia. The Virgin Islands is the only Phase 2 jurisdiction with an exceptionally high rate of utilization under the new payment system.

We also found that the SVRA participation rate for each Phase 1 and 2 state increased from March through December 2004. In addition, EN participation rates increased in most states, but Nevada experienced a substantial decline (0.5 percentage points), and four others experienced very small declines.

B. Predictors of Participation

In earlier reports (Thornton et al. 2004, 2006), we used administrative data to analyze how the participation rate varies with beneficiary characteristics. For this report, we used the first round of the NBS data, linked to administrative data, to enhance the participation analysis. To provide context for the presentation of the new findings, we begin with a summary of the earlier findings before turning to the new findings.

1. Findings from Analysis of Administrative Data

The most important of the earlier findings come from multivariate analysis of participation. For each factor, we estimate the relationship between the probability of participation and the factor, holding other factors constant. We have also conducted bivariate analyses of how the likelihood of participation varies with each factor, but without holding other factors constant. In the bivariate analysis, the relationship between a single factor (e.g., age) and participation might reflect the relationship between other predictors that change with age (e.g., impairment type) and participation. In multivariate analysis, we are able to estimate the effect of age on the likelihood of participation after holding impairment type (and other observed factors) constant.

Using data for March 2004 (Thornton et al. 2005, Appendix B), we conducted the multivariate analysis for Phase 1 states. The first important finding from that analysis is that it is particularly difficult to predict which beneficiaries participate based only on characteristics observed in administrative data. That is, we cannot define even a small group of beneficiaries based only on characteristics observed in administrative data in which the participation rate is very high in an absolute sense. In fact, when we use all of the characteristics as predictors together, the highest estimated probability of participation is less than 10 percent, and only 1 percent of beneficiaries have a predicted participation probability of 4.4 percent or higher. Although 4.4 is over four times the overall rate at the time of the analysis, it is still noticeably small.

Thus, it is not possible to rely on administrative data to identify beneficiaries who are “highly likely” to participate in TTW. The low predicted probabilities suggest that other unmeasured factors, such as the nature/severity of the individual’s impairment, other sources of support, and personal motivation, play an important role in beneficiary decisions. The survey data capture some such factors and are included in the analysis presented in the next section. Although we cannot use administrative data to identify beneficiaries highly likely to participate, we can identify several factors predictive of participation, holding other factors constant. The multivariate analysis also shows that, holding other factors constant, the likelihood of participation was higher if, at the time the Ticket was mailed, the beneficiary:

• Was relatively young

• Had attained a relatively high level of education

• Had a sensory impairment (especially hearing)

• Was classified as African American or Asian/Pacific Islander

• Had been on the rolls between 6 and 60 months

• Had been awarded benefits at the initial determination stage

• Was already using one of the work incentive programs (Section 1619 for SSI and participation in the extended period of eligibility for DI)

In addition, holding other things constant, participation rates were especially low for those:

• On SSI only

• With mental retardation, a musculoskeletal impairment, a circulatory condition, or a respiratory condition

• Requesting written communications in Spanish

The relationship between age and participation is particularly important. Participation rates are much higher for younger beneficiaries than for older beneficiaries. For instance, in March 2004, the probability that a beneficiary age 18 to 24 had assigned his or her Ticket was 2.7 percentage points higher than for a beneficiary age 60 to 64, holding other things constant. This relationship, along with the fact that a large majority of beneficiaries are over age 50, makes the average rate for all beneficiaries much lower than for relatively young beneficiaries.

We also included a variety of county characteristics in the multivariate analysis. Although bivariate analyses show some of the characteristics to be predictive of participation, they are not substantially predictive after controlling for other factors.⁹

2. Findings from National Beneficiary Survey

The data from the first round of the NBS provide additional information about characteristics related to TTW participation. We used NBS data for the 2,932 respondents from Phase 1 states to conduct a multivariate analysis of Ticket participation in June 2003 (when the survey sample was drawn, 15 months after the start of the Phase 1 rollout).¹⁰ Participation itself is based on administrative data. The sample participation rate (weighted to reflect the beneficiary population in Phase 1 states) is 0.82 percent as compared with 0.83 percent based on administrative data for the same month.

We included the following characteristics in the analysis (an asterisk indicates that the characteristic is from the survey data): title; primary insurance amount for DI beneficiaries; DI and SSI benefits if countable earnings are zero; cash benefits received from other programs*; months on the disability program rolls; Medicare eligibility; age; sex; race (Caucasian, African American, or other)*; Hispanic ethnicity*; education*; parental education*; living arrangements (living alone or with unrelated others, living with spouse or another adult as if married, living with minor children)*; age of disability onset*; main condition that restricts activities*; scores on physical and mental wellness scales*; functional limitations (e.g., concerning ability to walk, climb steps, lift 10 pounds, grasp objects, reach, stand, or crouch; ability to perform ADLs for bathing and dressing, getting around the house, getting into or out of bed, and eating; ability to perform instrumental activities of daily living IADLs for getting around outside the home, shopping, and preparing meals*; obesity (body mass index greater than 30)*; evidence of substance abuse*; and family income relative to the official poverty standard.* The following discussion of the findings focuses on the characteristics that are the strongest predictors of Ticket participation after holding constant all other characteristics on the list.

At least qualitatively, many of the findings reinforce the findings based on the administrative data only; quantitative differences may reflect differences in specifications, timing of measurement, and random sampling error. The survey analysis pertains to June 2003, whereas the administrative data analysis pertains to March 2004. We did not include all of the administrative variables and categories in this analysis because sample sizes for some groups were extremely small (e.g., those with hearing impairments and those in DI or SSI work incentive programs). It is also important to recognize that the estimates presented here are based on a survey sample of a few thousand beneficiaries, whereas the administrative data estimates summarized above are based on data for all of the over 2 million beneficiaries in Phase 1 states. Hence, differences due to sampling error can be substantial. All estimates reported here reflect statistically significant effects (5.0 percent level using a two-tailed test). Detailed results for participation appear in Appendix Table B.22, and detailed results for provider type and payment type appear in Appendix Tables B.23a and B.23b.

a. Participation

Overall, the variables included in the survey data enhance the ability to predict which beneficiaries will participate. Yet, even with the inclusion of these variables, it is not possible to single out individuals who are “highly likely” to participate. The highest predicted participation rate is 22.4 percent (more than twice the highest rate with use of administrative data alone), and 1 percent has predicted probabilities of 11.1 percent or greater (versus 4.4 percent or greater with use of administrative data alone).

Age is a strong predictor of participation. Holding other characteristics constant, those age 18 to 24 are 5.7 times more likely to participate than those 55 or older (Appendix Table B.22). African Americans are 80 percent more likely to participate than Caucasian Americans, holding other characteristics constant. We did not find significant differences for other races or Hispanic ethnicity. Participation increases with education, with the participation rate for those with more than a high school education 4.1 times higher than for those with less than a high school education, holding other things constant.¹¹ Beneficiaries with sensory impairments are 80 percent more likely than others to participate.¹² DI beneficiaries (both concurrent and DI-only) are 50 percent more likely to participate than SSI-only recipients. Participation is relatively low for those who first entered the DI or SSI program less than 6 months earlier, increases gradually through 24 months, is highest for those on the rolls between 24 and 60 months, and is significantly lower for those on the rolls for more than 60 months. Those on the rolls for more than 60 months are 40 percent less likely to participate than those on the rolls for 24 to 60 months.

Other findings pertain to variables not captured in administrative data. The first of these is that beneficiaries unable to perform one or more ADLs or IADLs without assistance are only about half as likely to participate as those with less severe or no ADL or IADL limitations. The evidence also indicates that inability to perform one or more physical functions reduces participation, but after controlling for ADLs and IADLs (as well as for other variables), the estimated effect is relatively small and not statistically significant.¹³

The probability of participation declines with age of disability onset after controlling for other factors (including current age). Those experiencing disability onset before the age of 18 are the most likely to participate. Those experiencing onset at age 55 or older are 60 percent less likely to participate than the former group. As expected, other groups fall between these two.

We expected to find that mental and physical health status would be predictive of participation, but the estimated effects are small and not statistically significant. It could be that other independent variables, such as the functional and activity limitation variables, have captured the effects of health status on participation, but we have not explored this possibility further.¹⁴ We also did not find significant effects for obesity or substance abuse after controlling for other variables.

Beneficiaries who live with a spouse, significant other, or other family members are less likely to participate than those who live on their own or with unrelated adults. Holding other variables constant, beneficiaries who live with relatives but do not have children under age six are 20 percent less likely to participate, while those with at least one child under age six are 70 percent less likely to participate. One possible explanation for lower participation rates when an individual lives with other adults is that the income of the other adults reduces the need for the beneficiary to generate income via work. In addition, the presence of others in the household may create better opportunities for the beneficiary to engage in productive household activities—particularly for those with children under age six. These factors appear to more than offset any positive effect that the availability of personal support within the household might have on participation.

Holding other variables constant, we expected to find that high benefit levels in the absence of countable earnings would reduce participation because of strong incentives to stay on the rolls, but we found no significant effect. One variable closely associated with benefits did, however, have a large negative effect: DI primary insurance amount (PIA). Holding other things constant, those with PIAs above $1,200 are only half as likely to participate as those with lower PIAs. This difficult-to-interpret finding might be termed a benefit effect because the DI benefit of a beneficiary with no dependents and with countable earnings below the SGA level is equal to the beneficiary’s PIA. The analysis, however, directly controls for benefits. Similarly, PIA is highly correlated with age, but we have controlled for age in the analysis. Hence, it seems that the negative effect of a high PIA on participation is attributable to some other factor associated with PIA that is not captured in the control variables.

A beneficiary’s PIA can be viewed as a composite measure of the beneficiary’s earnings experience; high PIAs are achieved only by those beneficiaries who received high levels of earnings subject to Social Security payroll taxes during a long period of their work career. High levels of past earnings may be predictive of high levels of potential earnings if the beneficiary returns to work, and we would expect high predicted earnings to increase participation, which is the opposite of what we found. One possible explanation is that beneficiaries with high past earnings are more likely than others to have accumulated substantial wealth, which would reduce the incentive to return to work. More broadly, the fact that such individuals have entered DI despite the low DI replacement rate for workers with high earnings suggests that they are poor candidates for return-to-work for some other reason—extremely severe disability, substantial income from other sources (pension, a spouse, private disability benefits, etc.), and perhaps others. Perhaps high earners have better opportunities and stronger incentives than other workers to continue work after disability, so they are more likely than others to delay their entry into SSDI until they are ready to retire permanently. They might enter SSDI only when changes in their circumstances make permanent retirement attractive: significant deterioration in their health, availability of private pension or disability benefits, changes in their spouse’s circumstances, etc. Although other control variables might serve as proxies for some such differences between high PIA beneficiaries and others, they likely do so only in a limited fashion.

We considered the possible effect of other (non-SSA) cash (e.g., private disability insurance) or near-cash (e.g., food stamps) benefits (public and private) that would likely be jeopardized by return-to-work. Our hypothesis posited that beneficiaries with such benefits would be less likely to participate. We found, however, that the results depend on the value of the other benefits. Those reporting low levels of such benefits (estimated to be worth less than $200 per month) were 60 percent more likely to participate than those reporting no such benefits. Those reporting high levels of such benefits ($500 or more per month) were—consistent with our hypothesis—40 percent less likely to participate, but the result is only marginally significant.¹⁵ It may be that low levels of other benefits are indicative of material hardship (e.g., not enough food or fuel) and that such hardships might motivate the beneficiary to seek work.

We also included an indicator for relatively high household income—that is, at least 300 percent FPL. We view this variable as a crude measure of a household’s total resources and expect that beneficiaries in high-resource households will have less of an incentive to participate in TTW. We found no evidence of any effect, however.

In many respects, the findings from the analysis of participation predictors are qualitatively similar to those from the analysis of predictors of employment (see Chapter II). That is, factors that predict employment also predict Ticket participation, after controlling for other factors, and in the same direction. Some exceptions, however, apply. Other things constant, SSA benefits are a negative predictor of employment, but not of participation; men are more likely than women to be employed but not more likely to participate; African Americans are more likely to participate than either Caucasians or those of other races, but not more likely to be employed; those of other races are more likely to work than either Caucasians or African Americans, but not more likely to participate; living with a child under age six reduces the likelihood of participation, but does not reduce the likelihood of employment; those with sensory disorders are more likely to participate, but not more likely to be employed; those in poor mental or physical health are much less likely to be employed, but only marginally less likely to participate.

b. Provider and Payment Type

We also consider characteristics predictive of type of provider and type of payment system for those who do participate. The number of (unweighted) participants in this sample is 1,105. By design, the respondents are approximately uniformly distributed across the three payment types. After weighting to reflect the population from which they were drawn, 13.0 percent had assigned their Ticket to an EN, 84.2 percent had assigned their Ticket under the traditional payment system, 13.5 percent under the milestone-outcome system, and 2.3 percent under the outcome-only system. These statistics are comparable to what is observed in administrative data for Phase 1 in June 2003.¹⁶

Several characteristics are associated with an increased likelihood of assignment to an EN, holding other characteristics constant (Appendix Table B.23a). SSI-only recipients are 70 percent more likely than DI recipients to assign their Ticket to an EN. The likelihood of assignment to an EN increases with age; those in the oldest age group (55 and above) are 4.7 times more likely than those in the youngest age group (18 to 24) to assign their Ticket to an EN. Hispanics are 80 percent more likely than non-Hispanics to assign their Ticket to an EN. Those with less than a high school education are 90 percent more likely than those who completed high school to assign their Ticket to an EN; unmarried parents with children are 70 percent more likely than others to assign their Ticket to an EN; and all parents with children under age six are 2.9 times more likely than others to assign their Ticket to an EN.

These findings suggest that participants facing return-to-work challenges other than disability—i.e., limited or no work experience, age, limited education, Hispanic ethnicity, parenting alone, and presence of preschool children—are more likely than others to be served by ENs. However, one finding appears to be contradictory to this conclusion: participants in households with incomes of at least 300 percent of the federal poverty line are 80 percent more likely to assign their Ticket to an EN. Not surprisingly, these same characteristics are associated with an increased likelihood of assignment under one of the new payment systems.

Given assignment under one of the two systems, fewer findings concern factors affecting the likelihood of assignment under the outcome-only payment system, probably reflecting both the relatively small sample for this analysis (722) and the small share that assigned their Ticket (after weighting) under the outcome-only system (14.8 percent) (Appendix Table B.23b). One clear finding is that participants under one of the new payment systems who experience disability onset relatively late in their lives are substantially more likely to assign their Ticket under the outcome-only system. For instance, those who experienced disability onset at age 55 or later are 2.6 times more likely to assign their Ticket under the outcome-only system than those who experience disability onset before age 18. In the earlier analysis of administrative data, we found that older participants under the new payment systems were more likely to assign their Ticket under the outcome-only system. In the current analysis, the relationship between age and payment system is relatively weak, although still positive, after controlling for age of onset and other characteristics. It appears that age of onset, which is highly correlated with age, largely explains the earlier finding.

¹ See the discussion of aggregate impacts on service enrollment (Chapter XIII) for more information. (back)

² See Appendix A for the rollout schedule and a list of states by phase. (back)

³ The number of beneficiaries eligible at the end of the period is lower than the cumulative number of Tickets mailed because of exits from the beneficiary rolls among the working-age population, which happened primarily because they either reached retirement age or died. (back)

⁴ This estimate is based on reporting through July 2005. Because of the lags in recording all assigned Tickets, we consider the July 2005 data as providing an accurate measure of actual Ticket assignments only through December 2004. (back)

⁵ The very high participation rate early in the Phase 3 rollout reflects the fact that some beneficiaries classified as residing in Phase 3 states had obtained and assigned their Tickets during the Phase 1 or 2 rollouts, most likely because of a later change in residence. Early in the rollout, the number of individuals in this group is substantial relative to the number of Ticket-eligible beneficiaries in the Phase 3 states, but their influence on the participation rate quickly disappears as Tickets are sent to all other beneficiaries in the Phase 3 states. A similar but smaller phenomenon is observed early in the Phase 2 states. (back)

⁶ Our earlier estimate was 75.9 percent. Most assignments reported after completion of the extraction of data for the earlier report were to SVRAs. (back)

⁷ We excluded the small number of reassignments from the count of deactivations, but the exhibit would change little if they were included. We omitted the first six months of the rollout period because the small numbers of in-use Tickets early in each phase’s rollouts leads to large but meaningless variation in net deactivations as a percentage of in-use Tickets. (back)

⁸ Based on Program Manager data from December 2004, the most extreme case is a provider that had accepted 361 assignments from the start of the Phase 1 rollout through December 2004 but had only 12 assignments in the last month. Two other ENs had at least 100 assignments fewer in December 2004 than the number they had accepted since rollout, and 116 ENs had withdrawn from the program entirely. (back)

⁹ The variables are population density, population loss between 1990 and 2000, percent African American, percent nonwhite, percent Hispanic, percent of populations living in households with income below the poverty line, percent of employment in manufacturing, unemployment rate, another low-employment index, an urban/rural index, the percentage of workers using public transit, a housing stress index, and a low-education index. (back)

¹⁰ All survey respondents were Ticket-eligible in that month. (back)

¹¹The estimates for education are based on the survey measure of education rather than on the administrative measure because the latter is missing for many observations. (back)

¹²Small samples prevent separate estimation of rates for hearing, vision, and communication impairments. (back)

¹³For analysis purposes, we used responses to questions about functional limitations and activity limitations to categorize respondents into four groups: (1) no functional, ADL, or IADL limitation; (2) moderate functional, ADL, or IADL limitations only; (3) inability to perform an ADL or IADL without assistance; and (4) inability to perform at least one physical function. “Moderate” means that the individual can perform the function, ADL, or IADL on his or her own, but with difficulty. Based on the survey, most beneficiaries are in at least one of the two “inability” categories, and many are in both; only 2 percent are in the “no limitation” category. Based on the estimates and after controlling for other variables, those reporting no or moderate conditions only are equally likely to participate, those reporting severe ADL/IADL limitations only but not severe functional limitations are 52 percent as likely to participate, those reporting severe functional limitations but not severe ADL/IADL limitations are 85 percent as likely to participate, and those reporting both severe ADL/IADL and functional limitations are 44 percent as likely to participate. (back)

¹⁴ This statement alludes to the statistical problem of collinearity between the independent variables. The analysis relies on the independent variation in each variable to estimate the variable’s coefficient. The independent variation of a particular variable is the variation left after removing the variation that can be accounted for by the other variables. Even if a variable’s total variation is high, the independent variation might be too low to estimate an effect with enough precision to be statistically different from zero, especially if the true effect is small. The variance inflation factor (VIF) is sometimes used as a measure of the extent to which collinearity affects the precision with which the coefficient of a variable can be estimated. The square root of the VIF is the amount by which the coefficient’s standard error is inflated by the inclusion of all of the other independent variables. Thus, a VIF of 1.0 implies no effect at all, a VIF of 4 corresponds to a doubling of the standard error, and a VIF of 9 corresponds to a tripling of the standard error. Three of the health category variables had VIF values ranging between 2.3 and 3.1—high but not excessive. Only five of the independent variables had VIF values in excess of 4.0—the indicator for those age 25 to 39 and each of the four indicators for age of onset. The effects of these variables on participation are so strong, however, that we found significant effects despite the level of collinearity. (back)

¹⁵ The result is significant at the 6.5 percent level using a one-tailed test. (back)

¹⁶ The weighted percentage assigned to ENs, 13.0 percent, is higher than the comparable number from the administrative data for June 2003, when 8.1 percent of in-use Tickets in Phase 1 states were assigned to ENs. The difference reflects sampling error. The weighted percentages assigned under each payment system are very close to the administrative data figures from June 2003 (83.9 percent under the traditional payment system; 13.4 percent under the milestone-outcome system; and 2.8 percent under the outcome-only system), reflecting the fact that the sample was stratified by payment type and the weights designed to reflect the variation in sampling probabilities across strata. (back)