What else works at scale?


Are there other theories or types of learning that can succeed at scale?


I read in an article about Stanford Moocs (Cooper & Sahami, 2013) that peer assessments (Sadler & Good, 2006) are proposed as a means for providing human assessments at scale, so if you consider formative assessment a type of learning…


Mmm, probably not the most popular example to propose… but, admittedly lecturing does work at scale. As most of our present professionals did get started that way. And although at some point, Socratic method was included at some point, but - to my knowledge - this also happened more between peers, than teacher/student. So … in a way, I have the feeling that any learning will scale if it fits human traits (eg. making conversation, showing/sharing knowledge, taking in snippets of information to build a (personal) bigger knowledge framework). The fact that many lecture based MOOCs do attract an audience, can either be a result of people wanting to follow something that seems familiar, and not necessarily has the best pedagogy (cynical view), or because it simply works. Cfr the griot’s (https://en.wikipedia.org/wiki/Griot), who tells his story for others to learn.


Anything that can be automated works at scale. Anything that cannot be automated must be distributed, and if so, works at scale. Only centralized non-automated things do not work at scale.


Why wouldn’t you? Every feedback is a possible learning activity. The one giving feedback learns be relating to something and the one receiving feedback can use this as an opportunity to further reflexion.
Or was your question rhetoric? :smiley: (Discussing in a foreign language has its pitt falls :wink:)


@downes this does feel like an absolute truth when I read it. But it triggered another thought, which admittedly goes beyond education… I am wondering where the boundaries of automation are located. Or whether automation is a human technological form that is on its way to become integrated in evolution? Humans scale, and have scaled before any automation existed, so what made us scale? Localized transmission of knowledge (oral tradition), the automated cell/gene replication in combination with adaptations? If this is the case, anything that can be replicated or found in evolution is scale-able, and at the same time, this will also be the boundary of any scaleability.
MOOCs in a way are replicating what already happened, but across location. And those who are successful are those that feel ‘natural’. Assessment must be traceable to earlier forms (trial and error). I wonder, can we scale up peace by education, if education never seems to have been able to remove war and passion for a long period of time. Like MOOCs (at the start) were assumed to be able to do.


Agree @Zerove, feedback is a great learning activity - the appropriate feedback at the appropriate time optimizes learning whether it is an agent, a teacher or a peer who gives it. Which make feedback design for learning at scale a cardinal point to develop and improve on.


@downes And perhaps what works best at scale are those things that can be automated in part and distributed in part (e.g. peer review, structured conversation, game-based learning, stealth assessment, formative assessment, open learner models). Automated without distributed becomes mechanical. Distributed without automated becomes chaotic.


This is a good question:

"Humans scale, and have scaled before any automation existed, so what made us scale? Localized transmission of knowledge (oral tradition), the automated cell/gene replication in combination with adaptations? "

And I think that the answer is inherent in the nature of life: that we are autonomous, and interactive, so we create a distributed network of diverse activities, adapting to local conditions, and scaling naturally.

Life seeks conditions of success. Humans, crickets, birds, plants - we migrate to the places where we flourish and avoid the places we don’t, each making our decisions one by one.

Too dense a network and society fails. Too sparse a network and society fails. Autonomy is productive; eliminate it and society fails. But where autonomy is extended to point where it disrupts the network, society fails. (These aren’t truisms; they are empirical observations, and subject to verification.)


I thing that is good points @sharplem and I think that they are finely supplemented by @Downes points about density of network and the civilisation (society) is possible in between Autonomy and lack of autonomy.
The hard part in both cases is to find a viable balance. And that balance will presumably work differently in different situation/contexts/instances.



Hi Zerove, Yes I guess to most people here this is obvious. Still, it is a form of assessment and if students learn from that or not depends on the student. It can also be a waste of time if students don’t see the feedback as an opportunity to learn. This may also have to do with the level of authority they assign to whomever provides the feedback and their own orientation on learning in the context.


I have been very influenced recently by reading Mejias’ Off the Network and I would like to put in a good word for what can’t be automated and may very well be beyond the gaze of the network, some of which is related to education and learning. Of course I realise that this may be a bit anti-MOOC :smiling_imp:


Thanks @francesbell for focussing on the not automatable. Lots of things are not automatable and this is highly relevant for MOOCs as well as for Learning Analytics. I find your comment very relevant in this discussion :smile:


@zerove @francesbell Supporting, or at least acknowledging, that learning that isn’t automatable and can be off network is a major challenge for MOOC providers, and a major benefit for learners. It’s a challenge for MOOC providers in that it can’t easily be analysed and may be disruptive. And it can be a benefit for learners in growing communities and affinity groups around shared learning interests, as well as offering a counter-narrative to that provided by the MOOCs. I wouldn’t regard this off-MOOC learning as anti-MOOC at all. One of the most fascinating parts of working with FutureLearn is understanding learners’ off-platform activities (through e.g. interviews with learners and diary studies).


@sharplem @zerove @francesbell Your discussion about off-MOOC learning has reminded me of a paper written by Veletsianos and colleagues this year in which they discuss ‘the limitations of clickstream-based research methods’.

Veletsianos, G., Collier, a, & Schneider, E. (2015). Digging Deeper into Learners’ Experiences in MOOCs: Participation in social networks outside of MOOCs, Notetaking, and contexts surrounding content consumption. British Journal of Educational Technology, 46(3), 570–587. doi:10.1111/bjet.12297/abstract


Thanks, I read the paper some time ago. Its really very good. :sunglasses:


This very much a response to Fleur’s contribution…
The fully online (but not a mooc, at least not using conventional definitions) program that I’ve directed over the past 3 years was premised on the concept of creating an online learning community. I’m involved in investigating the nature of an online learning community and what I’ve found so far is that members of the community need to be emotionally supported, or to use E. Wenger’s concepts, peripherally engaged and drawn into heart of the discussions and learnings that are occurring. However, this nurturing doesn’t seem to be enough to create a viable, ongoing structure that supports life long learning. Criticism, both the offering and the acceptance of, appears to be what is missing. We’ve used Helen Longina’s (Longina, H. (1994). The fate of knowledge in the social theories of science. In Schmitt, F.T. (Ed.), Socializing Epistemology: The Social Dimensions of Knowledge. Lanham, MD: Rowman & Littlefield.) conception of consensual knowledge building. In order for the developed knowledge to count, criticism must be offered and acted upon. We’ve implemented these ideas in the online courses by having the students challenge each other’s ideas and their own through peer and self-assessment processes in authentic problem-based contexts.


A few more ideas of what works at scale – kind of surprised no one mentioned texts/documents (writing, books, webpages). Outside educational settings watching YouTube how to’s is becoming a learning practice that works at scale.

Can we add to that searching – there is new work on ‘search as learning’
http://www.diigubc.ca/IIIXSAL/ – and an upcoming issue in the Journal of Information Science – Special Issue on Recent Advances on Searching as Learning - 2015 (Not sure if it is out yet).


Thanks for this pointer to the article in BJET. I haven’t read this and it is now going on my reading list :smile:


This is a great question and an interesting reply. I wondered whether we also need to think about the kinds of organisations that scale and how they achieve that. Civilisation arises with states and forms of centralised government. Knowledge begins to be stored and circulated via libraries and through religious organisations.

There is something about scale that involves organisational forms, governance and government. Autonomy is enabled and constrained by the organisations through which we cohere as societies. When life first develops it is dependent - in mammals on the mother and as life forms develop self-consciousness they are increasingly dependent on collective cultures. Knowing and learning is in some ways a restriction on autonomy which involves authority.