Executive Summary.........................................i
Background....................................................1
Service Ratings...............................................2
Customer Segment Information........................4
Access to Service...........................................6
Complete Call Handling..................................7
Automated Services.......................................8
Caller Spoke to a Representative.....................9
Preference for Future Contact........................10
Appendices...............................................Tabs
Question Summary.........................................A
Technical Appendix........................................B

As part of an ongoing evaluation of SSA's 800 number service, the Office of Quality Assurance and Performance Assessment conducts a semiannual customer satisfaction survey. The questionnaire was recently redesigned to reflect the survey's purpose as one element of SSA's new Market Measurement Program (MMP), with a focus on customers' ratings of service using a six-point, "world-class," rating scale. Although modified, the survey still provides data comparable to earlier 800 number caller recontact surveys, particularly with respect to the caller's actual experience. Where possible, comparisons with the results from the previous three surveys are presented in this report. The results for this survey reflect the perceptions and opinions of 1,220 customers who called the 800 number service in February 1999.

• Service ratings: Eighty-four percent of the survey respondents rated the 800 number service on the day of their call as "excellent," "very good" or "good" overall. This overall satisfaction rating is up slightly from the August 1998 and February 1998 ratings, both 83 percent, and is just 1-percentage point short of SSA's fiscal year (FY) 1999 goal of 85 percent. The 33 percent "excellent" rating is the same as for August 1998, and meets the FY 1999 goal. The satisfaction ratings for specific aspects of service ranged from 87 to 90 percent for employee attributes to 78 percent for access to service.

• Customer segment: The MMP calls for analyzing survey data by segments reflecting the type of business customers have with SSA. Customers are divided into six primary segments: Enumeration (Social Security number (SSN) or card-SSN); Earnings/Personal Earnings and Benefit Estimate Statement; Claims; Appeals; Postentitlement (PE); and General Inquiry. Claims and PE are further broken out by program: Retirement and survivors insurance, disability insurance (DI) and supplemental security income. Overall satisfaction ratings by customer segment and program range from 93 percent for DI PE to 74 percent for Appeals.

• Access: When asked about their experience getting through to SSA, 41 percent of all callers said they had tried to call earlier in the day when the lines were busy. About a third of these callers had tried to call three or more times. (These results cannot be compared to the results for similar questions on previous surveys, since the earlier surveys asked separate questions to distinguish between callers who heard a busy signal and those who heard a recorded message that all lines were busy.) Over half (59 percent) of the callers who chose to speak with a representative said they were placed on hold.

  As seen in other customer satisfaction surveys, quick access to service correlates with higher ratings for all other aspects of service.

• Complete call handling: Complete call handling has a profound effect on satisfaction. Virtually all callers (98 percent) were satisfied with service overall if they did not have to wait on hold or call back because the lines were busy, and their business was handled completely by the automated service or the representative. Ratings fall drastically if the caller's issue is not handled completely. Of the callers who used an automated service, 59 percent said it handled their issue completely. While this is a significant decrease from 73 percent in August 1998, the results for the last four surveys suggest a seasonal pattern (i.e., significantly higher completion rates in August than in February). About 77 percent of the callers who spoke to a representative said the employee handled the matter completely. This is the same rate as in the February 1998 survey and slightly less than the August 1998 rate of 79 percent.

• Preference for future SSA contact: Results from recent surveys show several trends: A small but increasing preference for calling the 800 number (from 70 percent to 73 percent), and a declining preference for calling a local office (from 21 percent to 14 percent). However, preference for visiting a local office has doubled over the same time period, from 5 percent to 11 percent. Not surprisingly, the higher the access rating, the more likely callers are to prefer calling the 800 number again.

Since 1989, the Office of Quality Assurance and Performance Assessment (OQA) has conducted periodic surveys of customers who call SSA's 800 number, to assess their satisfaction with SSA's service. The survey is conducted semiannually and was recently redesigned to reflect its purpose as one element of SSA's new Market Measurement Program (MMP). This report presents the results of the February 1999 survey, which is the first to use the redesigned questionnaire.

The MMP is an integrated plan for obtaining data on SSA's customers, employees and stakeholders to support Agency planning and decisionmaking. One aspect of MMP, interaction tracking, entails brief surveys of customers shortly after an SSA contact, focused on the aspects of service known to be important to them to obtain current impressions of service. Interaction tracking surveys with 800 number callers, field office callers, and office visitors serve as one of the primary mechanisms for monitoring and reporting on SSA's customer service performance. The results for the 800 Number Customer Survey, along with those for other interaction tracking surveys, will be used to measure progress toward the Agency's performance goals.

The redesigned questionnaire focuses on customers' ratings of service using a six-point, "world-class," rating scale. While the changes in the questionnaire have resulted in some loss of comparability to previous surveys, comparable data exist for many questions, especially those concerning the caller's actual service experience. Where possible, comparisons with the results from the previous three surveys are presented in this report. Tab A, which contains a question-by-question breakout of responses to the current survey, displays the previous results for all questions for which comparable data exist.

Participants for the 800 Number Customer Survey are randomly selected from electronic files containing records of incoming calls to SSA's national 800 number during the survey period. These callers may have talked with a teleservice representative, listened to an automated message or hung up while in queue. Since the caller's name is not available, interviewers call the originating telephone number and attempt to identify, contact and interview the person who placed the sample call. The survey attempts to contact individuals within 3 days to a week of their call to the 800 number. Since August 1997, the interviews have been conducted by employees of Management Research and Planning (MRP), Inc., a private company under contract with OQA.

The findings in this report are based on responses obtained through interviews with individuals who reached the 800 number service during February 1999. By completing 1,220 interviews, MRP achieved a 65 percent response rate, after adjusting the sample size to include only calls originating from household telephones. Calls were excluded when they originated from a public or business telephone since, generally, the individual who called the 800 number could not be identified. Calls were also excluded when they came from a telephone number that was disconnected at the time the survey was attempted. (The response rate for previous surveys was calculated without excluding these callers - see tab B for more details.) Nonrespondents either declined to participate in the survey or could not be interviewed for some other reason. Nonrespondents, had they been successfully contacted, might have answered the questions differently from the individuals who participated in the survey.

Survey data reflect callers' perceptions and opinions of SSA's service as well as their preferences for future contacts with the Agency. The following discussion highlights the most notable findings from the February 1999 survey. Differences in survey results are statistically significant, at the 95-percent confidence level, only when specifically indicated. Tab B describes technical aspects of the survey including sampling methodology, statistical computations and reasons for nonresponse. Since not all questions applied to all callers, the percentages shown are based on the number of respondents for each question unless otherwise indicated.

Back to Table of Contents

Service Ratings

The Agency's "world-class" six-point scale now used for customers' ratings of service was introduced in the February 1998 survey for the overall service rating and in February 1999 for specific ratings of service (e.g., employee courtesy). The six-point scale offers ratings of "excellent," "very good," "good," "fair," "poor" and "very poor." Previous surveys were based on a five-point rating scale. As a result, overall satisfaction ratings for surveys conducted prior to February 1998 cannot be compared to those for February 1998 and continuing. Likewise, this survey establishes a new baseline for ratings of specific aspects of service. Throughout the report, the "satisfaction rating" refers to the combined percentage of respondents rating service as "excellent," "very good" or "good."

Ratings and performance goal--An 84 percent overall satisfaction rating for SSA's service on the day of the sample call shows a slight increase compared to the February and August 1998 ratings, both 83 percent. This rating is 1-percentage point short of the Agency's goal of 85 percent for fiscal year (FY) 1999.¹ The rating of "excellent" given by 33 percent of the February 1999 callers meets the FY 1999 goal. This high "excellent" rating has been sustained since August 1998 and reflects a significant increase compared to 23 percent in February 1998. Ratings for specific aspects of service attributable to employees were high at 88 or 90 percent. The lowest rating, at 78 percent, was for access to service.

Comments on overall ratings--Callers who rated service as excellent were asked what they liked best. The number one answer related to receiving fast and efficient service (cited by over one-fourth of these respondents), followed by the fact that the matter was handled completely and the high level of employee courtesy (both about 20 percent). (Some respondents mentioned multiple features.)

Callers who rated service as very good or good were asked what SSA would have to change before they would rate the 800 number service as excellent. Interestingly, well over one-third of these respondents either had no comment or indicated that they did not know. About one-fourth of these respondents suggested that SSA provide faster service and/or or hire more representatives, and a little more than 10 percent indicated SSA should change or get rid of the automated service. Three respondents said they just never give ratings of "excellent."

Back to Table of Contents

Customer Segment Information

The MMP calls for segmenting and analyzing survey data according to the type of business that customers have with SSA, in order to identify and address a customer segment's specific service needs. The MMP categorizes customers according to program and by the following six types of business: Enumeration (Social Security number or card-SSN Applicants), Earnings/Personal Earnings and Benefit Estimate Statement (PEBES), Claims, Appeals,

Postentitlement (PE) Transations and General Inquiries. While detailed information about each segment will be obtained through another key element of the MMP--customer segment analysis--the results for this survey may be analyzed by customer segment.

Claims and PE are further broken out by program: Retirement and survivors insurance (RSI), disability insurance (DI) and supplemental security income (SSI), for a total of 10 customer groups. The following chart shows the overall satisfaction rating for each group (determined by asking callers their reasons for contacting SSA). The satisfaction ratings should be interpreted with caution because of the small sample size involved. Although the ratings range from 93 percent to 74 percent, only the differences at the highest and lowest ends of the ratings are potentially significant, due to the small size of the groups. (Note: OQA is not able to stratify the 800 number universe prior to sample selection, since the only information we have for the universe is the incoming telephone number and not the reason for the call.)

*Statistically significantly lower at the 95-percent confidence level than satisfaction of the top rated customer segment.

Back to Table of Contents

Access to Service

Experience getting through--Several survey questions relate to the caller's experience in reaching the 800 number service. Forty-one percent of all callers said they had tried to call earlier in the day when the lines were busy. Over a third of those who had called earlier (35 percent) had tried to call three or more times. (Previous surveys solicited a greater level of detail about whether callers heard a busy signal and/or heard a message that all lines were busy after asking to speak to a representative. The new wording for the redesigned questionnaire does not permit comparisons with previous surveys for these questions.)

Fifty-nine percent of the callers who chose to speak with a representative said they were placed on hold, a significant increase compared to the August 1998 rate of 46 percent, but consistent with February 1998 and August 1997 (58 percent and 55 percent, respectively). Nine percent of these callers hung up before speaking to a representative. While about half of those who hung up while on hold chose to call the 800 number again later to complete their business with SSA, 38 percent decided to call or visit a local office.

Access and satisfaction--We analyzed callers' satisfaction ratings for access and overall service according to the various experiences that may have contributed to perceptions of access. Some callers had tried to call earlier in the day when the lines were busy, some were placed on hold, and some experienced both, or neither, of these events. The following chart shows that satisfaction is high at 92 percent for service overall and 94 percent for access for callers who get through right away. It falls to 73 percent and 58 percent, respectively, when callers experience both a busy signal/message and time waiting on hold. The satisfaction of callers who waited on hold falls between those who got through right away and those who had to call back because the lines were busy.

The chart also illustrates the effect of access on overall satisfaction: As the access rating goes down, the overall satisfaction rating declines as well. In analyzing the survey results, we also found that quick access to service correlates with higher ratings for all other aspects of service. As the access rating declines from "excellent" to "very poor," the other service ratings decline, even for aspects of service seemingly unrelated to access, like courtesy and job knowledge. This effect of access on other ratings has been observed in other customer surveys.

Back to Table of Contents

Complete Call Handling

Complete call handling has a profound effect on the overall satisfaction rating. Earlier in this report, we discussed the relationship between the caller's ability to access the 800 number service and satisfaction with the service. As shown by the following chart, 98 percent of callers who got through right away and whose call issue was handled completely, either by an automated service or a representative, are satisfied with service overall. When callers have to call more than once and are unable to complete their business during the call, the satisfaction rating drops to less than 60 percent. The difference between complete and incomplete call handling is between 25- and 37-percentage points in the overall satisfaction rating.

[It should be noted that some call issues require multiple contacts with SSA and cannot be handled by a single call to the 800 number. Also, some issues requiring multiple contacts are adverse in nature, which we know from previous surveys may negatively affect the satisfaction rating, independent of the level of service provided.]

Back to Table of Contents

Caller Used Automated Services

Stable usage--The percentage of callers using automated services has remained stable over the last three surveys at 25 to 26 percent. Also consistent with prior surveys, the most popular options are obtaining or replacing a Social Security card (37 percent) and obtaining information about earnings and future Social Security benefits (30 percent).

Complete handling--Overall, 59 percent of callers who used an automated service said it handled their matter completely. This was a significant decrease from 73 percent in August 1998. However, the results for the two previous surveys suggest a seasonal pattern, since 61 percent of respondents in February 1998 and 75 percent in August 1997 said their matter was handled completely. However, no seasonal pattern is evident in the reported usage of the various automated services. Callers who said their issues were handled completely by the automated service had an overall satisfaction rating of 93 percent.

Of the callers whose issues were not handled completely by the automated services, half (51 percent) chose to call the 800 number again, but over a third (36 percent) decided to call or visit a local Social Security office. (This is the same pattern seen for callers who hung up while waiting on hold.)

As shown by the following table, reports of complete call handling ranged from 68 to 35 percent, depending on which automated service was used.

Reasons for not using automated services--As in past surveys, the overwhelming majority (91 percent) of callers who preferred to speak to a representative provided the same two reasons: Either they thought a recording would not answer their questions or they simply wanted to speak to a person rather than listen to a recording.

OQA will conduct a more detailed special study of 800 number automated service later this year to determine why the automated messages did not handle calls to completion.

Back to Table of Contents

Caller Spoke to Representative

Complete service--Of callers who spoke with a representative, 76 percent indicated the representative handled the matter completely so that no later contact or action was needed. This is consistent with the results for the three previous surveys. Callers who said their issues were completely handled by the representative gave a satisfaction rating of 94 percent for service overall, one percentage-point more than callers who said their issues were handled completely by the automated service.

Back to Table of Contents

Preference for Future Contact

Trends--Exhibiting a rising trend, 73 percent of respondents said they would call the 800 number the next time they contact Social Security. This was a slight increase over the last three surveys: 72 percent in August 1998 and 70 percent both in February 1998 and August 1997. Fourteen percent of the callers said they would prefer to call the local Social Security office, a significant decrease and a declining trend compared to 19 percent in August 1998, 20 percent in February 1998 and 21 percent in August 1997. Conversely, the last four surveys show a significant increase and rising trend in callers who prefer to visit a local office, from 5 percent in August 1997 to 11 percent in February 1999.

Not surprisingly, the likelihood of calling the 800 number again declines as the access rating declines. The percentage of callers who say they prefer the 800 number for the next SSA contact falls steadily from 85 percent to 44 percent as the access rating declines from "excellent" to "very poor." Still, the primary reason (35 percent) respondents gave for not calling the 800 number again was that they simply prefer to deal with the local office. Difficulty getting through accounted for a much smaller proportion (16 percent) of the reasons.

Back to Table of Contents

Tab A

800 NUMBER CUSTOMER SURVEY-QUESTION SUMMARY

The following table shows the results for each question of the February 1999 survey, which was revised to align with SSA's new Market Measurement Program (MMP). The new questionnaire has been simplified for ease of administration and uses SSA's "world-class" 6-point rating scale, with a top rating of "excellent," for all service ratings. Another change reflects the MMP requirement to analyze responses by customer segment and program. For all comparable questions, results for the previous three surveys also are displayed. Some of the results for the August 1998 survey may differ slightly from those previously published due to removing responses from business callers.

In addition to the questions shown, respondents for the February 1999 survey were asked a series of questions about how they would define "good service." The results for these questions will be provided in a future Office of Quality Assurance and Performance Assessment report to be used in updating SSA's customer service standards.

OQA 800 Number Customer Survey - Question Summary

Tab B

Footnote:
¹ Customer satisfaction goals apply to all types of SSA service, not just that provided via the 800 number. These results will be combined with the results for other surveys to report on SSA's performance.

Back to Text