00:11
Hello Len. Hi Asami. So today we are doing our monthly installation of 科学系ポッドキャスト
and for May 2025 we are the host of this monthly event. No way!
勝手に私が半年前ぐらいにサイエントークのレンさんに立候補してあったのね。
Whenever you need the next host to come up with a new theme, I'm kind of curious about this note thing.
And he was like, yeah, sure, I'll cue you up. And then he dropped the bomb like, you know, two months before like, oh, by the way, you're next.
I mean, you know, two months is... It's long enough in advance. And honestly, this is not such a high effort event.
I just had to set a theme and ask people to participate based on that theme. And it seems like there are plenty. There are like 20 something people interested in publishing the episode.
So as usual, this will be available on a Spotify playlist curated by Len-san and me. So if you're interested in listening to what everybody else has to say about notebook and note taking in general,
that's where you hear it. And so far there has already been like, you know, eight or nine or so people publishing.
Things like 手書きノート派かタイピング派かとか結構個人的なpersonal experienceから話してくれてる人が多いかなっていう印象ともちろんそれが聞きたかったことなんだけど
他にもねなんかnotebook LMとかなんかいろんなツールの話をされてる人もいるし文献整理what sort of citation management you useとかも結構話してくれてるので
It's kind of fun to hear because everyone has a different way of doing this. And it's it's not something you like hear a lot about unless you directly, you know, write papers with them or something.
So it's kind of fun. But for us, I wanted to set this theme personally because I wanted to have an excuse to read about this debate of handwriting versus typing for note taking activity, which is better kind of debate.
03:03
And I mean, I know that ultimately it's up to you, right? It's it's up to you, your preference.
Well, don't give away the answer up front. I mean, we're supposed to talk about this for like an hour or something.
Hey, kids these days don't have patience like that.
That's right. Yeah, we're supposed to say it up front. Okay. All right.
Anyway, but if you would entertain us to listen to the rest of the episode, because I did find like a train of studies that was interesting. And I don't think this is the only train of studies that studies this handwriting effect versus typing effect.
Because it seems like both people from developmental psychology field, sort of interested in like more toddlers learning process type people, as well as sort of neuroscience type camp people who are interested in the similar thing, looking at, you know, different age group, different sort of lens.
So the neuroscience camp is looking more at the brain activity using fMRI studies and stuff. Those are interesting. But today, I wanted to focus specifically on this train of studies that started in 2014.
That was done by Mueller and Oppenheimer. Yeah, in 2014, titled, the pen is mightier than the keyboard. Advantages of longhand over laptop note taking.
What a great title.
I know.
I know you shared this with me before, but I just want everyone to know like, or at least a great creative title, right? Maybe there are some people that would argue that maybe it's too creative, but I think this creativity draws you in and tells you what it's about to argue.
So, yeah, yeah, I think I am always for whimsical titling.
Yeah, bring me more whimsy, academia.
Yeah, yeah, you know, when you have to go through so many papers, a little whimsy here and there definitely helps.
Yep, yep.
But anyway, so these two study the effect of longhand, which is seems to be the word that people use to refer to handwriting over laptop note taking of kids in college age.
Prior to this studies, of course, they were not the first ones to start comparing these kind of correlational studies.
But in the past, like at least prior to 2014, most of the experimental tests seem to focus on the participants capacity to perform essentially multitasking rather than like laptop note taking versus handwriting.
06:10
So it's like looking at more of which method gives you higher tolerance for distraction, which is slightly different questions than notes should be taken by handwriting versus typing.
So they wanted to really zoom in on this question of handwriting or keyboard typing.
They've designed three experimental conditions whereby they show them TED talks of some kind, like 15 minutes-ish that covers a topic that is supposed to be like mildly interesting.
Like maybe not everyone's cup of tea, but it should be interesting enough.
So it's like some level of, oh, this is a new information engagement.
So in order to sort of remove the distraction factor, they've disconnected these laptops from the Internet.
So kids are only using laptop as a way to take notes, like really just as a tool.
So the first study of this showed that.
And then the second study, I'm just going to go through different types of studies here.
The second study, they tell some of the students to give them information saying that laptop note takers tend to write down words in verbatim.
And so that is transcribing what they hear or see into notebooks typing way rather than actually digesting, understanding what the content of the video is saying.
So they give that information upfront to some of the students and then compare the laptop non-intervention.
So just laptop user as usual, as study one, and then laptop users with intervention.
So the ones who were alerted to the concept of the dangers of writing verbatim notes using laptop.
Okay. Yep.
So that's the study two.
Study three is looking at more and focusing further into this encoding functions of the notes taking activities.
Well, typically when you're taking notes of something, you're trying to learn new things or retain that information that you took the notes of.
And they're really looking at in this third variation of the experiments, how well can note taking method improve the encoding process?
And also, how much would it improve if they can have a chance to look at the notes before they take the quiz?
09:01
So in the third one, they're not just writing notes, but they write notebooks, take a little break, review the note, and then take a quiz that has something to do with the learning material.
So there's a revision opportunity here.
You following me so far?
I think so. I am also I'm looking at the paper.
Which I now have.
And I think I'm following the three types or I'm following why they tried to do the three types of study.
Right. So study one is kind of the baseline.
We're looking at topics and we're giving you either the option to use notebooks or laptops or we're setting them up for that.
Okay. So laptop, notebook, topic.
Second one, they're adding an intervention in the form of instruction.
So they're telling the laptop intervention class, quote, actually, because I'm looking at it.
Basically, take notes in your own words and don't just write down word for word what the speaker is saying.
Yeah. End quote.
And then three seem to get at a little bit more of a nuance, I guess, was my understanding.
Yeah.
It's a little bit like, is it just that having a laptop and you just suck in more information and that's where the additional sort of gain is coming from?
Yeah, I think what number three, the study three tried to do is to understand, like, one, does either method make a better note, better notes to review?
And two, does reviewing the material help them recall the information better or worse, depending on your note taking method?
Yes. Okay. I think we're on the same page with that then.
Yeah.
I have one question, but maybe you should finish it first and I'll try to also find the answer.
Oh, no, no, no, it's fine.
Do you want me to ask it now?
Yeah, yeah, yeah.
Okay. I'm curious what I'm, so I'm looking at study two and I'm looking at that intervention where they say, please don't just write down word for word.
Yeah.
And I'm trying to see.
Yes, participants in laptop non-intervention and longhand conditions were given a laptop or pen and paper and were instructed, were doing a study about information in the classroom.
Right.
I, okay, I see how they've decided this, right?
They chose only to tell the intervention, laptop intervention participants don't take the notes word for word.
Yes.
Because they assume that only the laptop, right, intervention participants can take notes word for word.
12:00
Yes.
Yes.
I think that's an important distinction that I skipped, but yes, they only intervene with the laptop users, not the handwriting people, because the assumption is that because most people can type much faster than writing, the risk of, and that's been proven somewhere else.
The tendency is for laptop users to go into transcription mode rather than actually engaging with the material.
We can talk more about that later, but so the handwriting I think is just, it's kind of nearly impossible at least at the speed that they're showing the new information at whichever TED talk video that they're choosing.
It's not possible to do the verbatim.
They will be forced to sort of, you know, cherry pick what to take notes of.
Right.
They'll have to cherry pick the notes.
And I think I agree with their sort of assumption that you can't take them word for word.
I think there's more at least maybe in an educational pedagogy approach to talk about where also telling students to sort of not necessarily, you know, don't take them verbatim because we know that you can't, but to take notes in your own words and attempt not to write down what the speaker is saying is probably going to impact a longhand version too.
I am saying this from an educator's perspective.
Yeah, yeah, yeah.
And I think it's fair.
Exactly how they intervene with regards to the laptop users, that can also sort of have a different effect on how these kids will be intervening.
Right. And they've done a lot here.
So I don't think that should have also been in this paper.
I'm just noting, perhaps for consideration, either for us in our sort of just discussion way, or for consideration for the listener, there's an interesting thing here about telling people how to take the notes that may play a role in some of the way that notes take effect on you.
So anyway, but with that, maybe we can continue to what you wanted to share either from this results or the connections to other papers.
I'm not sure which way you want to take this.
Yeah, so I want to quickly go through these three experiments results because they're fairly straightforward.
So the first study where they just compared handwriting people versus laptop users, there is very little difference in recalling of the factual information, like how many years did X and Y take type of questions that are in the learning material.
15:01
Okay.
But slight overperformance on the handwriters than the laptop users, very slight.
And much more significant difference comes in conceptual recall.
So understanding more of what is this new topic that was introduced in the new learning material, et cetera, et cetera.
Those information were recalled much better according to this specific experiment by handwriters than the laptop users.
And the difference, at least relatively speaking to the factual information, is much more significant in the conceptual information recalls.
That's all you really need to know for this experiment.
Just know they looked at factual and conceptual information from the handwriters and laptop user perspective.
Handwriters perform better than the laptop users in both cases, but a lot better for conceptual learning than the factual learning.
The factual learning difference is pretty small.
Yep.
Information one.
So that's the conclusion of study one.
The results of study two.
So now we're comparing people with laptops users and the handwriting users and the laptop intervened users who are, like we've just talked about, being told to not copy word to word what the learning material is saying.
So now we have three groups to compare.
And we also, in this case, look at the factual information recall and conceptual information recall.
And so for the factual information recall, the order of performance was laptop intervened people performed better than the handwriting people, than the laptop non-intervened people.
So that's the order that they performed.
And that's figure four, if you're following, by the way, Len.
Yes, I am following.
And for the conceptual studies people, the three performance was quite different.
So for conceptual information recall, handwriting outperformed by a lot than a laptop intervened users, which was slightly better than the laptop non-intervened users.
So in both cases, having no intervention as a laptop users resulted in worse performance than any of the other two methods.
18:00
Right.
Yes.
Which one are we looking at for figures again?
Figure four.
This is figure four.
Yeah.
So, yeah, at least, yes, they both look to have worse, barring that the error bars indicating standard errors of mean based on their thing definitely overlap heavily.
Definitely overlap heavily.
And I think this is kind of inevitable when your sample pool is so small.
Yeah, that's true.
Their sample is relatively small, right?
Like in the orders of like dozens of people, not like thousands.
Yes, they had 67 students and they had to knock, I think, one or two out of there.
Right.
It is surprising that like in figure four, the conceptual doesn't have any overlap between the longhand scores and the laptops in general, though.
I guess that's fair.
Yeah.
I am checking because I noticed while I was on the page here that there is a correction page for this article.
Oh, interesting.
And it's not huge.
I did a quick scan.
But that same figure we're looking at right now, and I've shared that link with you, is still showing the difference.
But it's interesting that I think some of the bars themselves are actually a bit shifted and the error bars get a little closer between the longhand and the laptop intervention stage.
They still don't look visually very close.
So I don't think that changes their takeaway conclusions.
But just something to chew on, I think.
That's an interesting thing.
Okay.
Thanks for flagging this correction.
I didn't even notice that.
This is why we talk about studies, because I didn't notice until we started looking very close at the figures and I started like scanning around.
Yeah.
No, this is very cool.
And I do appreciate when authors publish corrections to their statistical analysis because it shows that they actually give a shit.
Yep.
And in fact, Figure 5, when you get to it, looks like there's actually more significant difference, at least with less overlapping error bars.
Yeah.
I just wanted to check.
The overall story changes very little, I think.
Yes, I agree.
But still, it's small but meaningful changes, I think, to report.
As you said, credit to the authors, I think, in this.
Yay.
Yay.
21:00
More authors should do this.
Yes.
Yes.
So sorry to cut you off there.
I wanted to sort of add that that was there, but also that your story is intact.
Or their story is intact.
Yeah.
So with that, Study 3.
So they wanted to see the effect of reviewing notes before this encoding task.
So this time they did the same thing as Study 1, but the students had the opportunity to look at their notes after some distraction tasks and participate in a quiz.
Yes.
And they looked at the quiz performance.
Now this is a two-by-two design studies where there is laptop users and handwriting people on one axis.
And then there's people who had the chance to study and the people who didn't have a chance to study the notes before they take the quiz.
Yes.
So now we're looking at four groups, like four conditions.
So with that in mind, they also looked at the factual and the conceptual information.
And interestingly enough, the factual information was best retained by the handwriter with notes.
So handwriting people with notes reviewing.
Followed by laptop people with notes, but by a lot lower success rate.
And then followed by laptop people without the study opportunity and the handwriting people without the study opportunity.
So surprise, surprise, people who had a chance to review their notes performed better.
But it is interesting that for at least for the factual performance, factual information recall, the handwriting people outperformed everybody else by a lot.
Yeah, that's the big standout of this.
So they're measuring performance with a z-score, so from all the scores that they get.
But it's huge in terms of, at least visually, that this is notably a win, hence the title, for the longhand and of course study version of this.
Which is also, just to kind of remind everyone, it's also interesting that in all of the other two studies that they've done, the study one and study two,
the significance between different conditions were much bigger in a conceptual information recall, as opposed to factual information recall.
But here, we're seeing much bigger difference, or at least more similar scale of difference in factual information recall versus conceptual information recall.
24:08
So that's interesting.
And for the conceptual recall performance, we have the best performer, again, with the handwriting with notes reviewing opportunity.
But the second one in the running is laptop users without the opportunity to review the notes.
Yeah, which is interesting.
Interesting, followed by handwriters with no opportunity to study their notes.
And the worst performers were people who used laptop and also had the chance to study their notes.
Yeah, this one is taking me time to digest.
And I haven't read how the authors explain this, but it's certainly a surprise, I would say.
Yeah, again, I was not particularly interested in what the authors had to say or justify, because I don't think that's their point.
Yep, that's totally fine.
And I think sometimes it's best for us to read and look at the actual data and their analysis first anyway, and to get to the story after.
But I think this is interesting because it goes to show, I think, that differentiating factual information recall from conceptual information recall is important as far as note-taking comparative study goes.
I think that's definitely true here.
And I think you can always come up with the reasons why laptop users with the ability to review notes perform the worst of all groups, you know, in some way, shape or form.
But I'm not really interested in hunting down the reason.
I am, however, interested in how many other studies were inspired by this paper.
Yes.
Because they, by they I mean Mueller and Oppenheimer, basically concluded the studies by saying a resounding yes to the handwriting people.
Take all their notes with handwriting, they perform way better than everyone else is their conclusion based on this data.
Yep, that is their conclusion.
And, you know, this paper got a lot of attention.
So far, it's been cited about 2,000 times.
Pretty notable, I think.
Yep, that's substantial.
Right.
And when your paper gets attention like this, the interesting next step that could happen is a lot of people trying to understand what you did.
27:03
And especially in the field of psychology, this seems to be a thing that one does.
And it's called direct replication studies to really, you know, do exactly what the original authors did and, you know, see if that applied to your sample pool and whatnot.
And this is one of them conducted five years later, entitled, How much mightier is the pen than the keyboard for note taking?
A replication and extension of Mueller and Oppenheimer studies.
Can we can we talk about momentarily just a shout out to the idea of having something in a field that is a direct replication study and has high level of, like, educational merit?
Or at least that's what I'm gathering, right?
Is the idea this is like, we went and did it.
And like, it's important to do this.
That's great.
Yes.
Yeah, yeah, no, it's really great.
And I'm so glad that they have done it.
And so that's that.
Right.
So with that said, let's now dig in.
What did I want to do?
Yeah.
So, I mean, I, again, didn't read the entire paper of this because what they've done is already being told by us in the title.
They did the direct replications.
And their conclusion basically was like, OK, we see the similar trend as Mueller and Oppenheimer for sort of recalls and note taking performances.
However, for encoding function, this follow up studies in 2019 said we didn't see the similar statistical strength in terms of significance.
So sure, the trend is there, but not as strongly as the original study seems to be confident about.
Right.
Is their conclusion.
Yeah.
Nuanced.
I like it.
It seems like many other people did, because only two years later, in 2021, there was a sort of small meta analysis across other direct replication studies of this Mueller-Oppenheimer studies reported by Urey et al.
In 2021, they have looked at eight other studies that did the same thing and looked at the metadata across these studies.
And they do note that this is still an exploratory meta analysis.
So their conclusion in 2021 is saying that both the original as well as the follow up replication studies associated higher word count in the notes with better quiz performance.
30:05
However, also a higher verbatim overlap with worse quiz performance.
Right.
And now I can only see the abstract on this one.
Yeah.
Yes.
And they were saying worse quiz performance with a caveat.
And then they're saying that the second point, yes, second point about the verbatim overlap resulting into worse quiz performance is not as robust as the original studies claims to be according to the replicated studies.
Right.
So what it suggests, I think, at least qualitatively, is that how one exactly takes the notes has less to do with the quiz performance.
So whether it has a lot of words or if it's a lot of words of your own or a lot of words of the transcription doesn't really make a difference in your quiz performance.
However, it does help if you have a lot of words in your notes.
Yeah.
Yeah.
Based on this between the amount of words in your notes and how that helps you.
Right.
Which implies something about either and we'll get more into maybe this the engagement of you on the material and or simply the amount that you are able to pay attention to and perhaps write down.
Right.
Yeah.
Or maybe you didn't have a breakfast that day.
Yeah.
Okay.
Yeah.
The magic of statistics is also supposed to work out some of those kinks.
Right.
But I am not a statistician.
But we only have eight studies.
At least here.
Yeah.
We're only looking at eight other direct replication studies of, I'm guessing, similar number scale.
Yeah.
So instead of dozens, we're now looking at hundreds of participants in total.
Yeah.
But we're also not looking at like tens of thousands of people.
So the magic of statistics a little weak here.
Yeah.
However, I think this newest meta analysis studies suggesting that, and they call this in a title, don't ditch the laptop just yet.
Right.
Yeah.
So what I think is basically saying it's not about whether you take the notes by hands or by keyboard.
What matters is the content of this and not the modality, essentially.
Right.
It's.
Which is like, I feel like I knew that already going in.
But, you know, it's good.
It's good to have somebody's work prove with evidence.
33:06
Yeah.
My, my.
They made sense.
Yeah.
Instinct was confirmed.
Yep.
Yep.
Yep.
So it doesn't matter if people call you old school for using handwriting or people claim that you are clickety clackety, whatever, with your keyboard.
Just, just use whichever one you prefer.
Use whichever one that is perhaps available at the time, because I don't imagine that you have access to a nice keyboard in like the jungle of Amazon.
Right.
So.
No, not unless you're carrying around one of those tanks of a laptop that exist out there.
Exactly.
But.
So, so, you know, use whichever one that's available.
Use whichever ones that you like.
As far as note taking goes, doesn't matter which one you use.
Matters more that you do something else with it, like reviewing the notes or perhaps, you know, none of these studies looked at.
But I think we both know that engagement of the material is important.
Yeah.
And how it's been delivered to you, to the students is also important.
Way bigger factor than the note taking modality, we think.
Yes.
And I, yeah, I would argue this probably loudly and with great energy from the top of any hill that you put me on.
Just because I feel that it's important.
Even there's a question that once came up in conversation with me and others that somebody mentioned, which was you may be able to always do the research on a subject or a context or a construct or something.
Right.
And but the question, I think, should also be, why is it that we're trying to uncover this?
Right.
Like say the answer is that laptops are the, you know, are at least notably worse in terms of like cognitive retaining of information.
Sure.
The problem, not that this is not a good example of this, right?
It's fine, right?
Like if laptops turned out to be the absolute worst, like actually made you lose information by using them, right?
Sure.
This is false, right?
But then in that case, the outcome would be us addressing it in a way that we would have to then improve something about the laptop usage because it isn't going away.
Right.
Like there are studies in other fields that we're not getting into right now, right?
They go somewhere else in a much dangerous way where it's like, you know, if we arrive at this answer, right, we can remove this other thing.
36:04
And that's not what you can do.
Right.
Even if that's the case, there are people and systems involved that we have to care for.
People use computers.
Like, OK, so how do we make it so that they are better?
Right.
That would still be the question there.
Yeah.
But what we get a lot of from like this nice sort of story that you can follow in those three papers is that it's kind of complicated and it probably has to do with the person's general time, not just time, it's not a measure of time, their engagement with the subject matter and how they're doing that through the note-taking process.
Yeah, yeah.
I think this is sort of like a classic case of, you know, you realize that the answer is more nuanced than you initially thought.
Yeah.
And that it may be less satisfying or definitive, but as scientific study goes, it is important to, you know, factoring these, you know, perhaps you did ask the question and you went for it.
But perhaps the real culprit was elsewhere.
Right.
Or the real bigger factor is elsewhere.
Noticing that experimentally is still meaningful.
And I'm glad that this information is out there in public.
Right.
Yeah.
So perhaps some of us had the instinct that the modalities doesn't matter from the get go.
But would we have had verified tests that prove this otherwise?
Not yet without these studies.
Right.
So...
Yeah, we needed people investigating it in the way that they did and to the scale that they did and to the intended replication that they did.
Yes.
Yes.
So perhaps the catchy titles, the whimsical titles were, you know, a little bit misleading in terms of the level of confidence that the initial authors might have had.
But I still take the whimsy, though.
Thanks for that.
And...
Yeah, I'll take it.
So...
Yeah, I think it's always good to know whenever you see find something that looks a little too definitive as science goes.
Yeah.
Yeah.
Think a little harder and poke around a little.
You'll probably find a more nuanced answer somewhere else.
Yeah.
A lot of times the answer is a bit more nuanced.
And that can be really hard to sort of sit with.
I...
That is great.
Also, just a note.
Great work sort of gathering all three of those papers' information because those first three studies, that was a ton of, like, generally, like, how are we organizing all of these people and ways in which we test things.
39:12
So thank you for doing all that summary, Asami.
That was great.
Oh, thank you.
Yeah, that was...
No, no, I'm glad to do that.
So I'll link the playlist on a show note.
Go ahead and listen to everybody else, what they have to say about Notebook.
I think we have, what, four more hours we can talk about Notebook and note-taking in general.
Probably.
Can I add one thing at the end, I think?
There might be something interesting here.
Sure.
I have not been able to access this paper yet.
However, through my investigations, there is an interesting paper titled Elucidating the Cognitive Processes Involved in the Note-Taking Effect, which came out in 2022 by the Ames Research Center grant, I guess.
These two individuals, I can't see where they're from, but Lakshmi A. Lalkandani, perhaps, I might have butchered that, and Alice F. Healy, hopefully I did not butcher that one.
The abstract is demonstrating their attempt to focus on what they're calling three unique cognitive mechanisms during note-taking.
This is now a quote.
Generative processing, summarization, and sustained attention, end quote.
To connect, the ability to create and put things together is generative processing.
It's taking novel information, prior knowledge experience, mashing them together, semi-paraphrased from their abstract.
Summarization is what we expect summarization to be.
It's not doing a copy and paste.
It's taking the information and attempting to pull out all the bits and pieces that are most important in an organized fashion.
Sustained attention is perhaps what you would expect, paying attention and not being distracted.
So these three things are, I think, not even I think, are definitely related to what we're talking about in response to the story of those three papers.
And in particular, I'm going to simply read out their last sentence.
They ran two experiments, but some of the details are not in the abstract.
However, what they're suggesting is that the generative processing, the part where you sort of create things from novel and previous experience, cannot account for what they're calling this note-taking effect.
The advantage of note-taking when there's no opportunity to review the notes.
They're investigating that idea.
That was also a quote from the abstract.
And what they instead note is that the summary and attention bits are more sort of important.
42:03
And so if we look back at those, like, no study opportunity ones, right?
The more that summarization and attention were at perhaps a higher level may also have been associated with the higher output of a score, right?
Or the higher ability of a student in there.
Something for maybe the listeners to investigate.
Maybe they can check out this article if they have access.
Or to at least remember that a lot goes into note-taking.
And it involves all types of processes.
And probably the more that you use all of them, the sort of generally more positive the outcomes are going to be if you want to remember that material.
Or use it.
Cool.
Yeah.
All right.
Well, that's it then.
Maybe we can talk more about our personal note-taking habits on our next episode.
Sure.
That sounds good to me.
Oh, no.
My habits.
All right.
Continue.
To be continued.
Bye.
Bye.
That's it for the show today.
Thanks for listening.
And find us on X at Ego de Science.
That is E-I-G-O-D-E-S-C-I-E-N-C-E.
See you next time.
