Hostname: page-component-78c5997874-s2hrs Total loading time: 0 Render date: 2024-11-14T23:04:25.706Z Has data issue: false hasContentIssue false
  • English
  • Français

Problems in Examining the Validity of the ACTFL Oral Proficiency Interview

Published online by Cambridge University Press:  07 November 2008

Lyle F. Bachman
Lyle F. Bachman
Affiliation:
University of Illinois, Urbana-Champaign
Get access Rights & Permissions [Opens in a new window]

Abstract

The primary problems in measuring speaking ability through an oral interview procedure are not those related to efficiency or reliability, but rather those associated with examining the validity of the interview ratings as measures of ability in speaking and of the uses that are made of such ratings. In order to examine all aspects of validity, the abilities measured must be clearly distinguished from the elicitation procedures, in both the design of the interview and in the interpretation of ratings.

Research from applied linguistics and language testing is consistent with the position that language proficiency consists of several distinct but related abilities. Research from language testing also indicates that the methods used to measure language ability have an important effect on test performance. Two frameworks—one of communicative language ability and the other of test method facets—are proposed as a basis for distinguishing abilities from elicitation procedures and for informing a program of empirical research and development.

The validity of the ACTFL Oral Proficiency Interview (OPI) as it is currently designed and used cannot be adequately examined, much less demonstrated, because it confounds abilities with elicitation procedures in its design, and it provides only a single rating, which has no basis in either theory or research. As test developer, ACTFL has yet to fully discharge its responsibility for providing sufficient evidence of validity to support uses that are made of OPI ratings.

Type
Articles
Information
Studies in Second Language Acquisition , Volume 10 , Issue 2 , June 1988 , pp. 149 - 164
Copyright
Copyright © Cambridge University Press 1988

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Adams, M. L. (1978). Measuring foreign language speaking proficiency: A study of agreement among raters. In Clark, J. L. D. (Ed.), Direct testing of speaking proficiency: Theory and application (pp. 129–149). Princeton, NJ: Educational Testing Service.Google Scholar
American Council on the Teaching of Foreign Languages. (1986). ACTFL proficiency guidelines. Hastingson-Hudson, NY: Author.Google Scholar
American Psychological Association. (1985). Standards for educational and psychological testing. Washington, DC: Author.Google Scholar
Bachman, L. F. (in press). Fundamental considerations in language testing. Reading, MA: Addison Wesley.Google Scholar
Bachman, L. F., & Clark, J. L. D. (1987). The measurement of foreign/second language proficiency. Annals of the American Association of Political and Social Sciences, 490, 20–33.CrossRefGoogle Scholar
Bachman, L. F., & Palmer, A. S. (1981). The construct validation of the FSI oral interview. Language Learning, 31, 67–86.CrossRefGoogle Scholar
Bachman, L. F., & Palmer, A. S. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16 449–465.CrossRefGoogle Scholar
Bachman, L. F., & Palmer, A. S. (1983). Oral interview test of communicative proficiency in English. (Available from authors.)Google Scholar
Bachman, L. F., & Savignon, S. J. (1986). The evaluation of communicative language proficiency: A critique of the ACTFL oral interview. Modern Language Journal, 70, 380–390.CrossRefGoogle Scholar
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.CrossRefGoogle ScholarPubMed
Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1–47.CrossRefGoogle Scholar
Carroll, J. B. (1941). A factor analysis of verbal abilities. Psychometrika, 6, 279–307.CrossRefGoogle Scholar
Carroll, J. B. (1968). The psychology of language testing. In Davies, A. (Ed.), Language testing symposium: A psycholinguistic approach (pp. 46–69). Oxford: Oxford University Press.Google Scholar
Carroll, J. B. (1983). Psychometric theory and language testing. In Oller, J. A. (Ed.), Issues in language testing research (pp. 80–107). Rowley, MA: Newbury House.Google Scholar
Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: M.I.T. Press.Google Scholar
Clark, J. L. D. (Ed.). (1978). Direct testing of speaking proficiency: Theory and application. Princeton, NJ: Educational Testing Service.Google Scholar
Clifford, R. T. (1978). Reliability and validity of language aspects contributing to oral proficiency of prospective teachers of German. In Clark, J. L. D. (Ed.), Direct testing of speaking proficiency: Theory and application (pp. 191–209). Princeton, NJ: Educational Testing Service.Google Scholar
Cronbach, L. J. (1971). Validity. In Thorndike, R. L. (Ed.), Educational measurement (2nd ed.) (pp. 443–597). Washington, DC: American Council on Education.Google Scholar
Cronbach, L. J. (1980). Validity on parole: How can we go straight? New Directions for Testing and Measurement, 5, 99–108.Google Scholar
Cronbach, L. J. (1984). Essentials of psychological testing (4th ed.). New York: Harper & Row.Google Scholar
Cronbach, L. J. (1985). Construct validation after thirty years. Paper presented at the Symposium on Intelligence in Honor of Lloyd Humpreys,University of Illinois, Urbana.Google Scholar
Dandonoli, P. (1987). ACTFL's current research in proficiency testing. In Byrnes, H. & Canale, M. J. (Eds.), Defining and developing proficiency: Guidelines, implementations, and concepts (pp. 75–96). Lincolnwood, IL: National Textbook.Google Scholar
Faerch, C., & Kasper, G. (1983). Plans and strategies in foreign language communication. In Faerch, C. & Kasper, G. (Eds.), Strategies in interlanguage communication (pp. 20–60). London: Longman.Google Scholar
Fitzpatrick, A. R. (1983). The meaning of content validity. Applied Psychological Measurement, 7, 3–13.CrossRefGoogle Scholar
Glaser, R., & Nitko, A. J. (1971). Measurement in learning and instruction. In Thorndike, R. L. (Ed.), Educational measurement (pp. 625–670). Washington, DC: American Council on Education.Google Scholar
Guion, R. M. (1977). Content validity: The source of my discontent. Applied Psychological Measurement, 1, 1–10.CrossRefGoogle Scholar
Hambleton, R. K. (1984). Validating the test scores. In Berk, R. A. (Ed.), A guide to criterion-referenced test construction (pp. 199–230). Baltimore, MD: Johns Hopkins University Press.Google Scholar
Higgs, T. V., & Clifford, R. T. (1982). The push toward communication. In Higgs, T. V. (Ed.), Curriculum, competence and the foreign language teacher (pp. 57–79). Skokie, IL: National Textbook.Google Scholar
Hymes, D. H. (1972). On communicative competence. In Pride, J. B. & Holmes, J. (Eds.), Sociolinguistics (pp. 269–293). Harmondsworth: Penguin.Google Scholar
Hymes, D. H. (1982). Toward linguistic competence. Philadelphia: University of Pennsylvania, Graduate School of Education.Google Scholar
Johnson, K. (1982). Some communicative processes. In Johnson, K. (Ed.), Communicative syllabus design and methodology (pp. 147–155). Oxford: Pergamon.Google Scholar
Kane, M. T. (1982). A sampling model for validity. Applied Psychological Measurement, 6, 125–160.CrossRefGoogle Scholar
Linn, R. L. (1980). Issues of validity for criterion-referenced measures. Applied Psychological Measurement, 4, 547–561.CrossRefGoogle Scholar
Liskin-Gasparro, J. (1984). The ACTFL proficiency guidelines: A historical perspective. In Higgs, T. V. (Ed.), Teaching for proficiency, the organizing principle (pp. 11–42). Lincolnwood, IL: National Textbook.Google Scholar
Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635–694.Google Scholar
Lowe, P. (1978). Third rating of FSI interviews. In Clark, J. L. D. (Ed.), Direct testing of speaking proficiency: Theory and application (pp. 159–169). Princeton, NJ: Educational Testing Service.Google Scholar
Lowe, P. (1985). The ILR proficiency scale as a synthesizing research principle: The view from the mountain. In James, C. J. (Ed.), Foreign language proficiency in the classroom and beyond (pp. 16–17). Lincolnwood, IL: National Textbook.Google Scholar
Lowe, P. (1986). Proficiency: Panacea, framework, or process? A reply to Kramsch, Schulz and, particularly, to Bachman and Savignon. Modem Language Journal, 70, 391–397.CrossRefGoogle Scholar
Lowe, P., & Liskin-Gasparro, J. (no date). Testing speaking proficiency: The oral interview. ERIC Clearinghouse on Languages and Linguistics, Q & A. Washington, DC: Center for Applied Linguistics.Google Scholar
Madaus, G. F. (Ed.). (1983). The Courts, validity, and minimum competency testing. Boston, MA: Kluwer-Nijhoff.CrossRefGoogle Scholar
Magnan, S. (1986). Assessing speaking proficiency in the undergraduate curriculum: Data from French. Foreign Language Annals, 19, 429–438.CrossRefGoogle Scholar
Messick, S. A. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 30, 955–966.CrossRefGoogle Scholar
Messick, S. A. (1981a). Constructs and their vicissitudes in educational and psychological measurement. Psychological Bulletin, 89, 575–588.CrossRefGoogle Scholar
Messick, S. A. (1981b). Evidence and ethics in the evaluation of tests (Research Report 81–9). Princeton, NJ: Educational Testing Service.Google Scholar
Mullen, K. A. (1978). Determining the effect of uncontrolled sources of error in a direct test of oral proficiency and the capability of the procedure to detect improvement following classroom instruction. In Clark, J. L. D. (Ed.), Direct testing of speaking proficiency: Theory and application (pp. 171–189). Princeton, NJ: Educational Testing Service.Google Scholar
Mullen, K. A. (1980). Rater reliability and oral proficiency evaluations. In Oller, J. W. & Perkins, K. (Eds.), Research in language testing (pp. 91–101). Rowley, MA: Newbury House.Google Scholar
Oller, J. W. (1976). Evidence of a general language proficiency factor: An expectancy grammar. Die Neuren Sprachen, 76, 165–174.Google Scholar
Oller, J. W. (1979). Language tests at school: A pragmatic approach. London: Longman.Google Scholar
Oller, J. W. (1983). A consensus for the eighties? In Oller, J. W. (Ed.), Issues in language testing research (pp. 351–356). Rowley, MA: Newbury House.Google Scholar
Popham, W. J. (1978). Criterion-referenced measurement. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Savignon, S. J. (1972). Communicative competence: An experiment in foreign-language teaching. Philadelphia: Center for Curriculum Development.Google Scholar
Savignon, S. J. (1983). Communicative competence: Theory and classroom practice. Reading, MA: Addison-Wesley.Google Scholar
Stevenson, D. K. (1981). Beyond faith and face validity: The multitrait-multimethod matrix and the convergent and discriminant validity of oral proficiency tests. In Palmer, A. S., Groot, P. J. M., & Trosper, G. A. (Eds.), The construct validity of tests of communicative competence (pp. 37–61). Washington, DC: TESOL.Google Scholar
Upshur, J. A. (1979). Functional proficiency theory and a research role for language tests. In Brière, E. J. & Hinofotis, F. B. (Eds.), Concepts in language testing: Some recent studies (pp. 75–100). Washington, DC: TESOL.Google Scholar
Upshur, J. A., & Homburg, T. J. (1983). Some relations among language tests at successive ability levels. In Oller, J. W. (Ed.), Issues in language testing research (pp. 188–201). Rowley, MA: Newbury House.Google Scholar
Valdman, A. (Ed.). (1987). Proceedings of the Symposium on the Evaluation of Foreign Language Proficiency (pp. 29–44). Bloommgton: Indiana University, CREDLI.Google Scholar
Vollmer, H. J., & Sang, F. (1983). Competing hypotheses about second language ability: A plea for caution. In Oller, J. W. (Ed.), Issues in language testing research (pp. 29–75). Rowley, MA: Newbury House.Google Scholar
Widdowson, H. G. (1978). Teaching language as communication. Oxford: Oxford University Press.Google Scholar