A smorgasboard of DDL journal activity

Last month, in addition to the release of new corpora, two journals released special issues dedicated to DDL/CL in language learning.

One is the open-access Language Learning & Technology. I haven’t read it yet, but the table of contents looks very interesting. The other one is Language Testing. It’s interesting to see how CL and questions of assessment interact.

Finally, though not a whole dedicated issue, ReCALL has an online first article titled ‘Unlearning overgenerated be through data-driven learning in the secondary EFL classroom’. This will be the first article I get to, as overgenerated be is a recurring issue for many of my students and I’m curious to see what the authors found.

What bounty 🙂


If the ReCALL link above isn’t working for you, here is the doi: https://doi.org/10.1017/S0958344017000246




Alive (and a meta-analysis)

Well, that was a longer break from blogging than I expected. I’ve had a tremendously busy winter, but hopefully I can get back to updating more frequently.

The first item on my list is to mention that Boulton & Cobb’s meta-analysis of DDL has been published. It adds context, detail, and discussion to their earlier slides.

It’s not a quick read, but the short version is that DDL works quite well in general; there are very encouraging results and several medium-to-large effect sizes were found. Going forward there needs to be more fine-grained research on for whom, for what, under what conditions, and for how long does DDL work well. They also make some important points about what information needs to be included in the future by researchers doing quantitative work on DDL.

To not mention*, it just feels right

@anthonyteacher has a great post at his site discussing the patterns “not to VERB” and “to not VERB”. He writes about his students’ reactions to the constructions, his own view, and some findings from Google N-grams and COCA. You should read his post in full.

I basically agree with everything he says, with one point he makes that I would like to extend a little bit. So I’d like to highlight this paragraph from his post, and especially the statement I put in red:

All of this data tells me several things. First, “to not” is on the rise, most likely due to the fact that the ability to separate an infinitive has become more accepted and “to not” has probably rolled in through a snowball effect. Second, the placement of “not” does not necessarily imply emphasis, as can be seen in the sentences above. Third, while my speech may make some of the older generations shake their first with anger, possibly telling me I am killing English, I can now reply confidently that my speech is the vanguard of an English where “not” is as placement-fluid as “they” is gender-fluid. My speech may be a speech that is likely to boldly go where few have gone before. Or to not boldly go, because language change is really unpredictable, and this is just a tiny thing.

I chose to highlight this section because I felt that sometimes my own choices regarding placement of “not” are definitely, if not necessarily, done for emphasis, but after thinking about it I don’t think it is a matter of placing emphasis per se. Rather, it is about restricting possible meanings/uses.

Let me explain. Here are two partial lines from COCA (query terms: not to mention):

1) … He would talk only if I promised not to mention he lived in …

2) … But tours and marketing materials, not to mention data on the average student, won’t tell you if that college will …

In the first line, I, personally, would probably phrase that as “to not mention”, though not necessarily. The point is that both constructions feel natural to me. However, I can’t imagine myself saying that about the second line. To me, and the way I’m processing these constructions, the first line’s meaning is straightforward, but the second line’s meaning is based on my understanding of “not to mention” as a fixed or partially fixed expression in this instance.

In this case, the construction is not simply negating the mentioning of something (in fact, the thing in question is explicitly and necessarily mentioned/understood). Indeed, the online Cambridge Dictionary, for example, defines “not to mention” as a phrase used when you want to emphasize something that you are adding to a list.

So, generally speaking, I process “to not VERB” as basically interchangeable with “not to VERB” (with a personal preference for “to not VERB”) when the meaning is straightforward (i.e. negating the verb). But “not to VERB”, perhaps because of it’s associations with certain fixed expressions, seems to me to have a broader range of usage. Something like this:

“not to VERB”: can negate the verb or have idiomatic/figurative meaning and usage

“to not VERB”: restricted to negating the verb

Here is a sample of COCA lines from @anthonyteacher ‘s post:


All the “to not VERB” uses here have meanings that can be understood as simply negating the verb. I suspect it would be this way throughout all the lines.

At least, until the language changes some more 😉

If I have said something glaringly, obviously wrong please tell me. Or if you have evidence of “to not VERB” used in an idiomatic/figurative way, please share it. Or if you have better choices for terminology etc. etc. etc. …

Parallel corpora and concordances

This might be tl;dr … If you are just looking for a list or some links to parallel corpora, please go to the end of this post.

In response to my presentation at this years ETJ Tokyo conference, where I talked about the parallel corpus and DDL tool called SCoRE, I was asked whether there were parallel corpora available in other languages. Short answer: Yes! Caveat: They are not always straightforward to use.

First of all, a quick explanation of what is a parallel corpus. It is a kind of bi- or multilingual corpus. A parallel corpus is a corpus that contains text from one language that is aligned with translations from one or more other languages; so, for example, if I query talk * in the Japanese interface of SCoRE I will get concordance lines in English that contain talk+ any word and concordance lines in Japanese that are translations of those English lines. These are parallel concordances.

English-Japanese parallel concordancing in SCoRE

Here is another illustration  showing a sample of a concordance from the Portugese-English parallel corpus COMPARA. The query terms were “talk” “.*” (this is the syntax for the talk + any word search in COMPARA, quote marks included).

Parallel concordancing in COMPARA

Parallel corpora are often used in translation and contrastive studies [see McEnery, A., & Xiao, R. (2007). Parallel and comparable corpora: What is happening. Incorporating Corpora. The Linguist and the Translator, 18-31]. Although they are not used as much in language learning, there has been promising work recently, particularly (as far as I’m aware) here in Japan [see Anthony, L., Chujo, K., & Oghigian, K. (2011). A novel, web-based, parallel concordancer for use in the ESL/EFL classroom. In Corpus-based studies in language use, language learning, and language documentation (pp. 123-138). Brill.; see also Chujo, K., Kobayashi, Y., Mizumoto, A., & Oghigian, K. (2016). Exploring the Effectiveness of Combined Web-based Corpus Tools for Beginner EFL DDL. Linguistics and Literature Studies, 4(4).]

Parallel concordancing can used for activities like translation tasks, of course, but they are also useful for DDL, at least in certain situations. In my experience, having translations of English concordance lines available in students’ L1 is very helpful for both lower-proficiency students and novice DDL students. Both the content and format of concordance lines can be difficult for such students, but in both cases the L1 support offered by parallel corpora allows students to quickly grasp the meaning of the English lines, letting them focus on the context or patterns in the lines. Even if they don’t always need the L1 support to really understand the English lines, they often feel more comfortable and are more receptive to doing activities and work that they are generally unaccustomed to doing. Perhaps as they become more familiar with concordance lines they can switch to monolingual lines.

Another benefit is that they can get a sense of how differently (or similarly) concepts, ideas, or notions may be expressed in the L2 as compared to their L1. Students can pick up on shades of meaning, nuance, and usage. I’ve seen this lead to lexical development where students have commented that they found a phrase or new (and natural-sounding) way to express something they had previously expressed inaccurately due to L1 interference, or had been completely unaware of because it wasn’t covered in any traditional way (i.e. it really is something they discovered for themselves). It’s only anecdotal, but I have spoken with my students about these mini ‘light-bulb’ moments and they react very positively to them.

There can be issues, though. There needs to be some understanding of, say, the directionality and relationship of the source material to the translations, or where the translations have come from and their quality, and of course that the translation seen in a concordance line is almost certainly not the only potential/accurate way to translate the source text. And another thing to keep in mind is that students’ need to share a single L1 unless the corpus is multilingual with translations available for all of the students’ L1s (which would overcome one issue but possibly raise others).

But still, parallel concordances can be quite useful and make it easier for students to get involved in doing DDL work. For more info about uses and issues with parallel corpora/concordances I recommend reading ‘Frankenberg-Garcia, A. (2005). Pedagogical uses of monolingual and parallel concordances. ELT Journal, 59(3), 189-198.’

Finally, where are these parallel corpora? A simple google search will turn up numerous parallel corpora available for download, such as the Open Parallel Corpus (OPUS), but that means you need to run your own parallel concordancing software. Something like AntPConc might be a relatively easy-to-use piece of software for this. However, even if you are comfortable running an application like AntPConc, the parallel corpora you find might not be appropriate for your students unless you are in an ESP environment with students learning language for, say, international legal or technical contexts (like the EuroParl corpus).

Alternatively, I’ve compiled a very brief list of some parallel corpora and projects that have web-based interfaces. A caution, though, I am familiar only with the English-Japanese corpora on this list; although some of the others have been used for language learning, or designed with language learning as a goal, I cannot vouch for the pedagogic applicability or accuracy of the other language combinations here (I’ll leave that to folks who understand the languages in these corpora).

Note: All of these are combined with English


SCoRE (and WebSCoRE); WebParaNews


E-C Concord (more info can be found here); BFSU CQPweb has several parallel corpora (for guests the user ID and password are both test)








COMPARA (further information about COMPARA is available here)


Tatoeba; Linguee; Reverso Context

Learner Language

ENEJE (this parallel corpus aligns essays by Japanese EFL students with edits made by native English speakers)

I’m sure there are many more. Feel free to list others in the comments 🙂


Not just for reference: dictionaries and corpora

These are the slides for a presentation/workshop I am giving at the ETJ Expo (details) in Tokyo this weekend. I am talking about some of the uses of learner dictionaries and English/Japanese parallel corpora like SCoRE and WebParaNews.

Handout: etjhandout-2

Note: Slide 4 erroneously links to ‘oven’, not ‘stove’, from the monolingual dictionary link.

DDL meta analysis

Last month some PPT slides on a meta analysis of DDL by Thomas Cobb and Alex Boulton were uploaded to the Lextutor FB group. (h/t MuraNava)

You can peruse the slides yourself, but one of the findings I found interesting was that while the best results seemed to be concentrated in learners of intermediate to advanced proficiency, lower proficiency learners also had generally good results, suggesting educators don’t need to be afraid of introducing DDL techniques with those lower proficiency learners.

For comparison with some older work, here are links to a preliminary meta analysis investigation by Cobb and Boulton; and a meta analysis limited to DDL in Japan by Mizumoto and Chujo.

SCoRE Corpus Update

The Sentence Corpus of Remedial English was updated and moved to a new website earlier this month: http://www.score-corpus.org. After following the link, there is a button in the top right corner of the page to switch to English.

The tools (Pattern Browser, Concordancer, Quiz Generator) seem to be the same, but it appears the corpus itself has expanded. The new website is much more comprehensive in the information it provides, including pages for things like SCoRE-related publications and user guides in both Japanese and English (there used to only be a Japanese language user guide). There is even a page where users can submit example sentences that may be included in future updates of the corpus.

Very good to see this project developing and continuing.