The 21st century challenges individuals to deal with demands that they previously faced either not at all or to a much lesser extent. The skills needed to successfully deal with these challenges are often collated under the term 21st century skills. They include broad concepts such as digital reading, information computer technology (ICT), and complex problem solving. Even though these skills have recently experienced a lot of interest and have been included in international large-scale assessments such as PISA or PIAAC, many questions on the conceptual and the empirical role of 21st century skills remain. For instance, the question how these skills relate to other conceptions of cognition such as the Cattell-Horn-Carroll theory or whether 21st century skills (incrementally) predict important life outcomes still need more rigorous empirical research. It is the goal of this symposium to present concurrent and state-of-the-art empirical research that aims at providing a comprehensive picture on 21st century skills and their assessment. In this, the symposium is composed of three contributions on three different 21st century skills: ICT literacy (Frank Goldhammer), digital reading (Johannes Naumann), and complex problem solving (Matthias Stadler). These contributions are followed by a discussion from a cognitive science and computer-technology perspective (Art Graesser).

Given the ubiquity of information and communication technology (ICT) in daily life, ICT skills have become a key competence enabling successful participation in educational, professional, social, cultural, and civic live. Thus, there is ample need for valid measures of these skills for purposes in educational policy, research, intervention, and instruction. This presentation will address the major developmental steps of a new computer-based ICT skills measure. First, a multidimensional theoretical framework is presented defining the targeted construct of ICT skills. Second, the development of interactive ICT tasks is described. We used simulations to design authentic task environments including several simulated software applications that need to be operated to solve the given task. Third, the psychometric properties of the scale are presented. The scale proved to be one-dimensional with a reliability of .72. To establish validity we show that systematically varied item properties in items’ instructions and stimuli affect item difficulty and tap into individual differences as expected. Finally, we show that relations to reading and problem solving skills, general cognitive ability, and computer knowledge match expectations derived from the theoretical framework. Overall, our findings demonstrate how computer-based simulations can be used to develop a sound measure of ICT skills.

With the Internet having grown to be a major resource for the dissemination of knowledge, opinion, and debate, a person lacking digital reading literacy cannot fully participate in online discourse and is, thus, cut off from major information resources and channels of debate. Besides traditional literacy skills such as decoding and coherence processes, digital text frequently requires the readers to select, and order textual materials (“navigation”), a process that draws on cognitive resources in addition to text processing. Using PISA data, we first show that digital reading performance is predicted by two indicators of navigation quality, “precision”, and “task-adaptive processing”, which also mediate effects of print reading skill on digital reading performance. Second, we show that time-on-task in digital reading is more positively predictive of task performance in hard digital reading tasks and in tasks requiring complex navigation. Likewise, we show that time-on-task is more positively predictive of digital reading performance in weak readers. Our results confirm the assumption that navigation is a multifaceted process that is consumptive of cognitive resources  impacts performance in different ways. Finally, we discuss prospects and limitations of using large-scale data to explore a latent variable’s cognitive structure.

The university years represent a critical phase in the life of many students that comes along with various complex opportunities and challenges. Based on this premise, the aim of this study was to investigate the role of complex problem solving (CPS) skills in predicting university success. 150 German students worked on a measure of reasoning as well as a set of complex problem solving tasks. In addition, the students were asked for their current grade point average at University (GPA) and their subjective evaluation of their university success. CPS was significantly related to university GPA (R2 = .18) even after controlling for reasoning (ΔR2 = .09). In addition, CPS was related to the students’ subjective evaluation of their university success (R2 = .10) with incremental value over and above reasoning (ΔR2 = .09). The results suggested that complex problem solving skills helped students successfully navigating a university program even beyond reasoning skills.