The seminar was held on 4–5 February 2010 at Hughes Hall, Cambridge, UK, preceded by a one-day workshop on 3 February 2010. The seminar was attended by 80 participants, representing 14 countries. I attended as Steering Committee Observer for the Council of Europe.
The workshop, attended by 30 international delegates to the seminar, opened with a 30-minute talk, which I gave on the Modern Languages Projects of the Council of Europe up to the publication of the Common European Framework of Reference for Languages: learning, teaching, assessment (CEFR) in 2001 and the recent Recommendation of the Committee of Ministers to Member Governments regarding its use. The workshop was then conducted by Dr. Anthony Green of the Centre for Research in English Language Learning and Assessment, University of Bedfordshire, and was devoted to ‘CEFR Linking and Standard Setting for Writing’. Using well-chosen concrete examples, the participants worked their way through the successive stages of linkage: familiarisation, specification and validation. The hands-on experience was much enjoyed and a number of valuable lessons were learnt, namely:
-
1. That despite the wide diversity of experience and institutional background, the level of consensus in attaching a level to given descriptors was high.
-
2. That some raters were consistently rating descriptors higher (or lower) than the general consensus and became more self-aware as a result of the exercise.
-
3. That where, in two carefully chosen cases, the general consensus placed a descriptor at a level different from that ascribed to it in CEFR, it was possible to identify the feature of the descriptor responsible for the discrepancy.
In all, the workshop was felt by all participants to have been highly successful and participants were encouraged to consider holding similar events in their home environments, bearing in mind the need for equally careful advance planning and preparation. There are plans to run similar workshops across Europe over the next three years as part of the EU-funded strand of English Profile activity.
The seminar proper opened with an account by Nick Saville (Cambridge ESOL), tracing the development of the English Profile Programme (EPP) from the formation of the core group in 2005. The first year saw the formulation of core objectives:
-
1. to set learning goals for English at levels A1-C2
-
2. to identify criterial features for each level
-
3. to provide illustrative examples.
In 2006, a project structure was set up, a draft research agenda agreed, and the Cambridge Learner Corpus (CLC) was made available to English Profile researchers. Developed jointly by Cambridge ESOL and Cambridge University Press, the CLC consists of the writings of candidates in the Cambridge examinations in English as a foreign language and has been collected from over 135,000 learners around the world. The Project was registered with the Council of Europe as a project for the description of English in relation to the Common European Framework of Reference for Languages. In November 2009, 3-year EU funding was obtained for networking and corpus collection. Other important developments in 2009–2010 included:
-
1. Start made on collection and analysis of the new Cambridge English Profile Corpus, which will complement the CLC by incorporating a wide range of non-exam data, including both spoken and written learner output.
-
2. Setting up of two research fellowships.
-
3. Setting up of a website to serve as the main English Profile communication tool.
-
4. Creation and availability of digital versions of the Threshold level series (Breakthrough, Waystage, Threshold and Vantage).
-
5. Two volumes reporting work carried out in the project were in active preparation.
-
6. Global networking had been further developed and extended, to be formalised in the foundation of an English Profile Association.
Overall, it could be said that the past year marked a transition from the stage of preparation, planning and piloting to full working, producing concrete results, and from a limited project to a many-sided programme.
Ron Ragsdale (ELT Publishing Director, Cambridge University Press) then gave an update on the involvement of Cambridge University Press in English Profile. With the overall aim of producing global standards for English based on CEFR, the role of the Press is:
-
1. To co-finance the programme.
-
2. To assist in the production of the Cambridge English Profile Corpus.
-
3. To set up and maintain the website and the English Profile Journal (which is in development).
-
4. To publish resulting studies and improved learning materials.
-
5. To promote English Profile globally.
Paul Heacock is Publishing Manager, with Julia Harrison responsible for networking and corpus management, and Melissa Good for publications. In discussion, questions were raised concerning access to corpus materials, and distinctions were explained between those open to the public, those open to those in the network (later, members of the association) and those restricted to the core research teams.
After lunch, John Hawkins (Cambridge University Research Centre for English and Applied Linguistics (RCEAL)) presented the work that he and Luna Filipovic (Cambridge University Department of Linguistics/Cambridge ESOL) have undertaken on Theoretical Foundations for EPP Criterial Features. Since CEFR is non-language specific, it does not seek to identify the actual grammatical structures, categories and lexis for reception and production at successive levels of proficiency. This has to be done separately for each specific language. The RCEAL group have approached the task for English empirically, searching the CLC for evidence of what successful candidates for successive examinations in the Cambridge ESOL suite, corresponding to Levels A2 to C2 are, or are not, able to use at successive levels. This gives four basic types of feature: grammatical/usage distribution, in both cases positive/negative. Positive criterial features are those found regularly in scripts at level n but not, except sporadically, at level n−1. Negative criterial features are those which are strongly associated with errors at level n and below, but are used error-freely at level n+1. Examples were given of all four types. From their observations and on theoretical grounds, they arrived at four general principles of second language acquisition (SLA):
-
1. minimise learning effort
-
2. minimise processing effort
-
3. maximise expressive power
-
4. maximise communicative efficiency
and seven specific principles:
-
1. maximise positive transfer
-
2. maximise frequently occurring properties
-
3. maximise structurally and semantically simple properties
-
4. minimise quantitative load
-
5. permit negative transfer (which results in formal error but does not impair communication)
-
6. block negative transfer which impedes communicative efficiency
-
7. order SLA in accordance with these principles, which serve to define possible vs. impossible and likely vs. unlikely interlanguage stages from a given L1 to a given L2.
Thus the order of acquisition of grammar and lexis will differ not only from L1 to L2 but also from one L1 to another. This finding, backed by examples from CLC, has important implications for level definitions and indeed for the organisation of language learning, teaching, assessment and qualification on an international basis. It remains to be seen whether and how a viable scheme for English Profile can be constructed in accordance with these findings.
Annette Capel (Cambridge University Press) then presented her work on English Profile Wordlists: validation and future developments. The work on vocabulary now went well beyond word listing, distinguishing word senses. Topic coding on an experimental basis enabled words to be grouped into word families or by parts of speech. The material for A1 to B2 was now available online to English Profile partners only. The compilation of lists for C1 and C2 was now due to start, with computing support and analysis from February 2010.
Ron Martinez (University of Nottingham) then presented work under way on phrasal verbs and expressions, identifying 200 phrases not in English Profile lists for A1 to B2 as well as many mismatches. Much of the material collected was at levels C1 and C2 and was now being consolidated.
Tony Green (Centre for Research in English Language Learning and Assessment (CRELLA), University of Bedfordshire) then gave an account of his work on functional descriptors at Levels C1 and C2. CEFR descriptors at the C Levels had been compared with other attempts to define advanced functional competences, with a view to their reorganisation, supplementation, and so on. In general it had become clear that functional progression was essentially qualitative rather than quantitative. Broadly, the same functions had to be covered at every level, but so as to meet increasingly demanding criteria (this has been, of course, a basic principle of the T-level series, which has kept the same superordinate functional classification from Breakthrough to Vantage).
In the final session of the first day, Anne O'Keeffe and Margaret Healy (Mary Immaculate College, University of Limerick) presented the Cambridge, Limerick and Shannon (CLAS) corpus project. A specialised spoken language corpus has been collected at the Shannon College of Hotel Management, consisting of transcribed lectures, practicals, EFL classes, presentations and meetings held in connection with their Bachelor of Business Studies degree, attended by 176 Irish and 182 international students taught by a staff of 27.
On the second day, Dora Alexopoulou (RCEAL), Angeliki Salamoura (Cambridge ESOL) and Helen Yannakoudakis, PhD student at the Computer Laboratory of the University of Cambridge (funded by Cambridge ESOL within the framework of the English Profile 139 Programme), gave an update on English Profile data collection, now extended to spoken sources and moving from paper-based to online collection, with plans for the automatic classification of texts. Learner data was now more specific through questionnaires to learners and teachers together with their CEFR ratings of students. An ‘interest club’ and teacher training arrangements were also planned. By June 2010 there should be 40 setups with 100 learners each contributing 300 words, thus 1,200,000 words in all, plus additional data from Education First, now partners in the programme. This action would be repeated from June–November 2010, producing a combined total of 2,700,000 words. In discussion, Cyril Weir (CRELLA) asserted that the problem of calibration must be tackled early. What he said was being calibrated: texts, performances, items? In his view, lexico-grammatical tests were the best level predictors.
Ted Briscoe (Computer Laboratory (CL) Cambridge University) presented the progress made following a different line of enquiry, relying on automatic data analysis. An automatic marking system is under development, along with a semi-automatic error coding system which can be used for automatic data-driven level assignment. No attempt was made to identify ‘criterial’ features. Instead all features found at one examination level rather than another were regarded as ‘discriminatory’ features. They were found in considerable numbers and their statistical treatment produced the best level assignment on a necessarily probabilistic basis. It was hoped that automatic speech recognition would soon be sufficiently reliable for the method to be extended to spoken language. In later discussion Professor Briscoe expressed the view that too much had been made of functional criteria. What was of central importance was to master a new grammatical and lexical coding. Given that, the understanding and use of language was largely a matter of inferencing – a universal human ability.
Mike McCarthy (Nottingham University) gave an update on English Profile spoken language research, looking at the turn-openers and turn-closers characteristic of successive CEFR levels. It is also intended to extend the transcription of spoken texts to the intonation patterns used.
Luna Filipovic led a discussion session on emerging patterns in the search for criterial features for each CEFR level. An example was ‘subject-to-subject raising verbs and adjectives’. These were first found at B1 level (seem to & supposed to). They were then generally introduced at level B2 (appear to, likely to, turn out to, certain to, etc.), but chance to was first noted at C1. Similarly, subject-to-object raising verbs (unpassivised) were first found as follows: A2 none; B1 want, like, expect (e.g. I didn't expect her to come so early); B2 prefer, imagine; C1 believe, find, take, suppose; C2 presume, declare, remember. A number of grammatical features were discussed, as were patterns of error typical of successive levels. A lexical grid was presented, showing that while highly frequent words were first introduced at a low level, new senses appeared much later (e.g. she cut the cake at A2, but to cut a long story short only at C1. This line of enquiry is clearly producing interesting, concrete results, but it is still far from clear how a viable level system is to be constructed on this basis and what the consequences for learning, teaching and assessment of doing so might be.
Stephen Spencer, holder of a PhD scholarship awarded by Cambridge ESOL to take part in the English Profile Programme, then presented on lexical and grammatical errors in spoken discourse, and on noun phrase constructions in CLC.
The final presentation, from Melissa Good (Cambridge University Press) and Roger Hawkey (Cambridge ESOL), explored the collaborative work now in progress towards a glossary for English Profile. This would be a ‘jargon-busting’ resource, available on the website, rather than a full in-depth treatment.
I was then asked for a final comment on the seminar. I said that I found the volume of work presented very encouraging, showing clearly the transition from planning and preparation to a productive phase in a complex, many-sided programme. Participants now understood their place in relation to that of the other partners and were hard at work, producing concrete, practical results. Coordination of the very different contributions was of course necessary to ensure the coherence of the programme. There were still some fundamental concepts and issues to be fully clarified. Careful thought would also be required to assess the full significance of detailed investigations for the overall scheme. I was sure that the Council of Europe would welcome the developing international outreach of the programme and the value attached by the project partners to the link with the Council. One consequence of the special position of English in global communication was that a programme like English Profile could command considerable resources. The Council of Europe is firmly committed to plurilingualism and it is to be greatly hoped that the experience gained in English Profile could then be made available to teams working to improve provision for the learning, teaching and assessment of other European languages, as had been the case with the T-level series and CEFR. I looked forward to next year's report on what would certainly be a year of continuing productivity.
