A continuing challenge – How to assess online reading comprehension?


One of the most pressing challenges we’ve faced in planning and teaching online reading comprehension has been how to assess and track students’ knowledge of online reading strategies. Originally we had hoped to use the Online Reading Comprehension Assessment tool (ORCA), developed by Professor Don Leu and the New Literacies Team from the University of Connecticut. This sophisticated assessment tool simulates an online environment where students are challenged to complete an online research problem by locating information, evaluating their search results, synthesizing information from several sources and then communicating their final solution by email. As students complete the ORCA, their choices and actions are automatically saved and coded. After visiting the research team, and testing the ORCA ourselves, we learned that it is currently being trialed with students in Connecticut and Maine. In future, the ORCA will be a valuable tool for assessing students’ online literacy skills.

Our assessment purpose
Although it would have been interesting to compare a student’s skills when reading offline and online texts using an instrument like the ORCA, our assessment purpose was primarily formative:
1. to provide a simple measure of the internet strategies students are aware of using to complete school tasks.
2. to determine changes in students’ awareness of internet strategy use over the period of instruction.
3 to guide and inform our teaching in response to the ‘gaps’ in students’ identified online strategy knowledge.

We decided to use a simple pen and paper reflection task, also developed by the New Literacies research team , “Formative Assessment of Students’ Emerging Knowledge of Internet Strategies” – FASEKIT.
(Click here for a link to the FACEKIT internet assessment proforma that we used.)

What we did.
We administered the FASEKIT proforma at the end of the first two terms this year and we will continue to do so at the end of Terms 3 and 4. The students are encouraged to write for 10 minutes listing and describing the online reading strategies they find most useful when completing homework, or school research tasks. They also explain when, and why, they use each strategy.

The information collected from the Term 1 FASEKIT was prior to any online reading instruction. At the end of Term 2, we returned the students’ original FASEKIT proformas and invited several students to share strategies that they had listed previously. The students then wrote for 10 minutes continuing their personal list of online reading strategies. We will continue to administer the FASEKIT at the end of Term 3 & 4.

What we’ve learned.
The strategies nominated by the students were tallied using a taxonomy of online reading comprehension skills (TICA_Phase_II_Checklist). This checklist organises online reading comprehension strategies under the following categories –
Locate information – using a search engine and search results
Locate information within a website
Critically evaluate information
Synthesise information
Communicate information
Using these categories we compared the strategies the students listed at the end of Term 1 with those listed at the end of Term 2.  The graph below shows that in Term 1, prior to explicit online reading comprehension instruction, most students nominated simple strategies for Locating information using a search engine  such as, “Write words from the assignment sheet into Google to find out information”. The students listed a few simple strategies for Communicating information e.g.”Use print screen to save a picture of the screen when I want to back up my work or show someone”.

Online reading strategies students identified using the FASEKIT protocol

This graph shows that after a term of instruction the students listed a greater variety of internet reading strategies.  They now identified many strategies for Locating information within a website.  Strategies such as, “Scroll down to the bottom of a webpage and skim read before I say it’s a bad website”, and “Use Control F to highlight words through a webpage, so I can keep on track when I’m reading” revealed that many students were reading online texts in a more considered and thoughtful way.  The students also listed more sophisticated strategies for Locating information using a search engine. We noticed that the metalanguage for discussing webpages, that we had been explicitly teaching, was reflected in their Term2 FASEKIT comments e.g. “Read the URL and the domain extension, because it gives you a brief insight about what the website is about and who it is written by.”

This simple assessment tool is providing us with useful information which is informing our planning and teaching.  In Term 2, we deliberately focused on teaching students strategies to help them locate and evaluate relevant information when reading online. Leu’s (2008) research identified that locating is a crucial step for effective online reading comprehension.  If students are unable to navigate the internet to identify relevant information from search results, or from within a webpage, then their comprehension becomes blocked.  According to Leu’s research, even compentent “offline text readers” can become defeated by reading “bottlenecks” online.  During Term 3 & 4 we plan to use broader, online inquiry questions to teach different strategies for critically evaluating and synthesising online information.


About onlinereadingcomprehension

We are teachers at an independent school in Sydney. As we learn more about teaching online reading comprehension we intend to share our work through this blog.
This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s