Original Photo adapted from Hansueli Kramer / CC BY
Original Photo adapted from Hansueli Kramer / CC BY
|
|
Session Overview |
|
| |
| 9:00am - 10:00am | K5: Do Countries and Organizations Have Personalities? Session Chair: Fons van de Vijver Dave Bartram (CEB’s SHL Talent Management Solutions, UK, and University of Pretoria, Department of Human Resource Management, South Africa) |
| KO2-F-180 (Ⅵ) | |
| 10:15am - 11:45am | IS6: The Assessment of 21st Century Skills Session Chair: Samuel Greiff Discussant: Arthur C. Graesser |
| KO2-F-180 (Ⅵ) | |
|
|
The assessment of 21st century skills
The 21st century challenges individuals to deal with demands that they previously faced either not at all or to a much lesser extent. The skills needed to successfully deal with these challenges are often collated under the term 21st century skills. They include broad concepts such as digital reading, information computer technology (ICT), and complex problem solving. Even though these skills have recently experienced a lot of interest and have been included in international large-scale assessments such as PISA or PIAAC, many questions on the conceptual and the empirical role of 21st century skills remain. For instance, the question how these skills relate to other conceptions of cognition such as the Cattell-Horn-Carroll theory or whether 21st century skills (incrementally) predict important life outcomes still need more rigorous empirical research. It is the goal of this symposium to present concurrent and state-of-the-art empirical research that aims at providing a comprehensive picture on 21st century skills and their assessment. In this, the symposium is composed of three contributions on three different 21st century skills: ICT literacy (Frank Goldhammer), digital reading (Johannes Naumann), and complex problem solving (Matthias Stadler). These contributions are followed by a discussion from a cognitive science and computer-technology perspective (Art Graesser).
Presentations of the Symposium Simulation-based assessment of ICT skills ; goldhammer@dipf.degoldhammer@dipf.de
Given the ubiquity of information and communication technology (ICT) in daily life, ICT skills have become a key competence enabling successful participation in educational, professional, social, cultural, and civic live. Thus, there is ample need for valid measures of these skills for purposes in educational policy, research, intervention, and instruction. This presentation will address the major developmental steps of a new computer-based ICT skills measure. First, a multidimensional theoretical framework is presented defining the targeted construct of ICT skills. Second, the development of interactive ICT tasks is described. We used simulations to design authentic task environments including several simulated software applications that need to be operated to solve the given task. Third, the psychometric properties of the scale are presented. The scale proved to be one-dimensional with a reliability of .72. To establish validity we show that systematically varied item properties in items’ instructions and stimuli affect item difficulty and tap into individual differences as expected. Finally, we show that relations to reading and problem solving skills, general cognitive ability, and computer knowledge match expectations derived from the theoretical framework. Overall, our findings demonstrate how computer-based simulations can be used to develop a sound measure of ICT skills.
Processes and predictors of digital reading literacy: What we can and cannot learn from large-scale assessments ; j.naumann@em.uni-frankfurt.dej.naumann@em.uni-frankfurt.de
With the Internet having grown to be a major resource for the dissemination of knowledge, opinion, and debate, a person lacking digital reading literacy cannot fully participate in online discourse and is, thus, cut off from major information resources and channels of debate. Besides traditional literacy skills such as decoding and coherence processes, digital text frequently requires the readers to select, and order textual materials (“navigation”), a process that draws on cognitive resources in addition to text processing. Using PISA data, we first show that digital reading performance is predicted by two indicators of navigation quality, “precision”, and “task-adaptive processing”, which also mediate effects of print reading skill on digital reading performance. Second, we show that time-on-task in digital reading is more positively predictive of task performance in hard digital reading tasks and in tasks requiring complex navigation. Likewise, we show that time-on-task is more positively predictive of digital reading performance in weak readers. Our results confirm the assumption that navigation is a multifaceted process that is consumptive of cognitive resources impacts performance in different ways. Finally, we discuss prospects and limitations of using large-scale data to explore a latent variable’s cognitive structure.
The role of complex problem solving in university success ; matthias.stadler@uni.lumatthias.stadler@uni.lu
The university years represent a critical phase in the life of many students that comes along with various complex opportunities and challenges. Based on this premise, the aim of this study was to investigate the role of complex problem solving (CPS) skills in predicting university success. 150 German students worked on a measure of reasoning as well as a set of complex problem solving tasks. In addition, the students were asked for their current grade point average at University (GPA) and their subjective evaluation of their university success. CPS was significantly related to university GPA (R2 = .18) even after controlling for reasoning (ΔR2 = .09). In addition, CPS was related to the students’ subjective evaluation of their university success (R2 = .10) with incremental value over and above reasoning (ΔR2 = .09). The results suggested that complex problem solving skills helped students successfully navigating a university program even beyond reasoning skills.
|
| 10:15am - 11:45am | PA23: Creativity and Emotional Intelligence Session Chair: Johnny Fontaine |
| KOL-G-217 (Ⅳ) | |
|
|
Assessment of emotional intelligence: A plea for unscored ratings Ghent University, Belgium; Johnny.Fontaine@UGent.beJohnny.Fontaine@UGent.be
How emotional intelligence ability items should be scored has been vigorously debated. In the present study we investigate the possibility to directly derive emotional intelligence from the raw item ratings as given by the participant without any post hoc scoring of the items. We do this by investigating the internal structure of emotional intelligence subscales at item level. We hypothesized that rating-based emotional intelligence scales would be structured by two factors: a bipolar ability factor with right items loading positively and wrong items loading negatively, and a unipolar acquiescent response style factor with all items loading positively on it. This hypothesis was investigated on the Mayer-Salovey-Caruso Emotional Intelligence Test - Youth Version in a first sample of 630 Flemish pupils and a second sample of 664 Flemish pupils. In the first sample the original instrument, three rating subscales were applied; In the second sample an adapted version was applied with all four scales presented in rating format. In both samples the hypothesized structure was confirmed for all subtests using rating scales. Across the factors of all subtests a general emotional intelligence and a general acquiescence factor emerged. The nomological network further confirmed the interpretation of the factors.
Measuring emotional intelligence in early adolescents: An application of the latent change variable models 1University of Zagreb, Croatia; 2J. J. Strossmayer University of Osijek, Croatia; vbusko@ffzg.hrvbusko@ffzg.hr
This study is focused on the analysis of intraindividual changes in emotional intelligence (EI) conceptualized within the ability-based model. Particular classes of structural equation models were applied to the study of correlates of inter- and intraindividual variations in the proposed performance based measures of the three EI dimensions. The data to be presented is derived from the longitudinal study of EI development conducted in three time points on the sample of 517 primary school students aged 10 to 15 years. Following the assumptions of the latent state-trait theory (e.g. Steyer, et al., 1992), the degree to which variations in EI measures are due to individual dispositions and/or to occasion-related factors will be presented. Several single- and multi-construct latent state-trait models were tested against the EI data. According to the parameter estimates obtained, the portions of variance attributable to situational and/or interactional effects varied with the point of measurement and the EI operationalization used. Further, the true change modeling procedures (e.g., Steyer, Eid, and Schwenkmezer, 1997) employed confirmed the significant role of gender and cognitive ability measures as moderator and antecedents of interindividual differences in changes on EI measures, respectively.
Validation through inhouse-meta-analysis exemplified on an inventory of creative activities and achievements 1University of Graz, Austria; 2Federal Academy of Lower Austria (Niederösterreichische Landesakademie), Austria; jennifer.diedrich@noe-lak.atjennifer.diedrich@noe-lak.at
Creative activities and achievements can be reliably assessed using self-report inventories (Silvia, Wigert, Reiter-Palmon & Kaufman, 2011). These measures differ in their focus on personal as opposed to public achievements. The inventory to be presented – the inventory of creative activities and achievements (ICAA; Jauk et al, 2013a & Jauk et al, 2013b) – is constructed to assess both levels of achievement in eight different domains. The ICAA has been employed in eight studies along with tests of creative potential, personality, and intelligence. This inventory’s reliability and validity was estimated in two different ways: First internal consistency and measurement models are performed in a compound dataset comprising the ICAA variables of all eight studies. Second, convergent and divergent validity with measures of personality, intelligence, and creative potential were performed via meta-analyses of these eight datasets. This two-tier approach was chosen due to the reasonable homogeneity of ICAA variables but not of the validity variables. The advantages of this two-tier approach shall be presented at the conference.
Assessing creativity by meaning Tel-Aviv University, Israel; krit@netvision.net.ilkrit@netvision.net.il
The purpose was to develop a procedure of assessing creativity in terms of the meaning system (Kreitler) which is a psychosemantically-grounded methodology for assessing meaning. Three studies will be presented, describing the meaning variables found to differentiate significantly between more and less creative participants, in different samples of children, and different measures of creativity. In study 1, 158 children (ages7.2-9.4) were administered the meaning test, the Wechsler IQ test, and the Torrance test of creativity. In study 2, 71 children (mean age 10.9) were administered the meaning test and their drawings and paintings were evaluated for creativity. In study 3, 238 Beduin children (mean age 13.7) were administered the meaning test and the questionnaire “The Things Done on your Own” (Torrance). In each study the meaning variables differentiating between the more and less creative were identified. The set as a whole indicates the following tendencies characterizing the more creative children: focusing on dynamic, objective and experiential aspects, using nonverbal and verbal forms of expression, considering present inputs and distant ones, and emphasizing both the personal-subjective and the interpersonally-shared meanings. The meaning variables characterizing the more creative children could be used for developing an assessment instrument for creativity.
|
| 10:15am - 11:45am | PA24: Educational Assessment 3 Session Chair: Fariha Asif |
| KOL-G-204 (Ⅱ) | |
|
|
Identifying the profiles of social and emotional development among first grade pupils in Russia National Research University Higher School of Economics, Russia; aponomareva@hse.ruaponomareva@hse.ru
Personal, social, and emotional development (PSED) of young children is particularly important at the beginning of primary education. This paper is aimed at constructing PSED profiles (in the framework of iPIPS study of schooling progress) of Russian school children during their first year at school and determining relationship between PSED and cognitive abilities. Childrens' PSED was assessed by teachers using 11 scales. The scales themselves were arranged in three sections: adjustment to the school environment, personal development, and social and emotional development. Data were collected in one big region of Russian Federation, the Republic of Tatarstan. The sample representative of Tatarstan consisted of 1,218 (409 boys and 447 girls, mean age - 7.3 years old; gender data for 362 students is missing because of the low parents response rate) primary school students assessed by 68 teachers. 4 PSED profiles were identified using cluster analysis (k-means method). In group one children scored high on all scales and in other three groups children had certain characteristic strengths and weaknesses. A relationship between profiles and cognitive abilities was found. This information can help teachers find the most effective ways to work with children during their first year at school.
An ecological assessment framework for tracking learner growths in self-regulation, emotional engagement, and feedback responses in a virtual diagnosis learning program University of Toronto, Canada; eun.jang@utoronto.caeun.jang@utoronto.ca
The paper discusses assessment principles for learning in technology-rich learning environments, in this case BioWorld (Lajoie, 2009). BioWorld is a patient simulation program designed to support medical students’ clinical reasoning. Students are invited to diagnose virtual patients by gathering and evaluating medical evidence. The framework highlights a shift away from discrete content domain knowledge to cognitively, metacognitively, and emotionally competent applications of target knowledge. In this framework, learners’ current state of knowledge and skill mastery are constantly changing as a result of interactions with elements of learning contexts, which contradicts a static view of learning. The present study gathered multiple types of behavioral, affective, cognitive, and contextual data from students’ self-reports, computer logs, and think-aloud verbal protocols. Latent class analyses (Vermunt & Magidson, 2005) and cluster analyses were used to identify distinct latent classes among students in terms of their self-regulation, emotional engagement (Pekrun, Goetz, & Perry, 2005), and responses to expert feedback. Logistic multiple regression analyses were used to examine the predictive probabilities of successful diagnosis of virtual patients using different learner trait profiles. I discuss the viability of the ecological assessment framework for tracking learning progressions and providing intervention support tailored to individual learners’ profiles.
The anxiety factors in the Saudi EFL learners: A study from English language teachers’ perspective King Abdul Aziz University, Saudi Arabia; farihaa83@yahoo.comfarihaa83@yahoo.com
The purpose of the study is to explore the factors that cause language anxiety in the Saudi EFL learners and the influence it casts on communication as observed and perceived by EFL teachers. The study seeks to answer questions such as what are the psycholinguistic and socio-cultural factors, as per teachers’ perspective that cause language anxiety among ESL/EFL learners while learning and speaking English Language, especially in the context of the Saudi students. It also finds what strategies can be used to successfully cope with language anxiety. The scope of the study is limited to college and university English Teachers in Saudi Arabia. The sample size is 115(Mean age = 35 years). One hundred university English teachers (both males and females) were selected from various cultural backgrounds. Five points Likert scale questionnaire comprising twenty items was served to these 100 English teachers. In additon, 15 structured interviews were also conducted. Some English teachers believe that anxiety serves a positive outcome for the learners by giving them an extra bit of motivation to do their best in English language learning.
|
| 10:15am - 11:45am | PA25: Organizational and Counselling Settings Session Chair: Claudia Harzer |
| KOL-G-209 (Ⅲ) | |
|
|
Work and Meaning Inventory (WAMI): Progress report on the German adaptation of the WAMI University of Kassel, Germany; c.harzer@uni-kassel.dec.harzer@uni-kassel.de
The Work and Meaning Inventory (WAMI; Steger, Dik, & Duffy, 2012) is a 10-item self-rating questionnaire measuring meaning at work. Three subscales assess the degree to which work is perceived as being meaningful (positive meaning, PM), contributing to meaning in life (meaning making, MM), and serving others (greater good, GG). The German version was developed in a translation-back translation procedure. Initial data analysis (n = 252 adults) indicated satisfactory reliability and construct validity. However, inconsistencies in factor structure (i.e., PM and MM converged into one factor) required further investigation. In collaboration with Steger eight new items were included in order to improve the discrimination of PM and MM. Exploratory factor analysis of the 18 items utilizing data from a new sample (n = 367 adults) indicated a three-factor solution (i.e., Scree-test, parallel analysis). In order to develop a parsimonious measure, the best three to four items (main loadings > .45, cross-loadings < .20; full representation of intended content of scales) were selected. Fit indices in confirmatory factor analysis indicated a satisfactory fit. Currently, further data are being collected to examine replicability of the factorial structure, but also reliability and construct validity. Latest results on descriptive statistics, reliability, and validity will be presented and discussed.
Assessing the likelihood of Counterproductive Work Behaviour (CWB) - An organisational and international driven test design 1Central Test, France; 2NEC, UK; nigel@nigel-evans.comnigel@nigel-evans.com
Accurately measuring employee integrity, commonly referred to as Counterproductive Work Behaviour (CWB), is a growing concern for organizations. Case reviews and high profile media exposés show the pervasive damage companies experience worldwide as a result of employee CWB, as well as the lack of recognised overt assessments available. Globally, the measurement of obstructive work behaviours remains an underdeveloped area of testing, which contrasts to its demand. The paper outlines the development of an assessment aimed at measuring individuals’ likelihood of engaging in CWB. The design draws upon research suggesting that attitudes and subjective norms are strong predictors of intentions of engaging in CWB. As such, the assessment is designed to measure people’s attitudes and dispositions towards instances of CWBs to predict their likelihood of engaging in CWB. Six broad dimensions measured by the assessment were derived from an extensive organisational field survey conducted in three languages (English, French, and Spanish) with 109 companies from 35 different countries. The behavioural content was then referenced to ISO 26000, which details the internationally recognised standards for organisational social responsibility. Preliminary findings from statistical analyses will be discussed to show the test’s efficiency in predicting CWB as well as its international application.
Evaluation of a new instrument for assessing civic-economic competence in commercial apprenticeship 1University of Zurich, Switzerland; 2University of Konstanz, Germany; esther.kaufmann@ife.uzh.chesther.kaufmann@ife.uzh.ch
Civic-economical competence is defined as the ability to understand private, economical and economic-political problems, to judge suggested solutions for complex problems, and to develop solutions for less-complex problems. To date there has been no two-dimensional instrument for assessing civic economic competence in commercial apprenticeship developed; although based on theoretical reasons such an instrument is urgently needed for an international comparison. Additionally, there is a gender gap in economic competence, which needs to be checked for an international comparison. Hence, in our study we evaluate a newly developed two-dimensional online tool for the German-speaking commercial apprenticeship in Switzerland and Germany and check it for any gender gap. Based on our pilot studies (Germany: n = 260, Switzerland: n = 250), our developed and adapted online instrument is checked by a sample comprising 874 German commercial apprenticeship students. Our item-analysis (IRT, DIF), the modeling of the postulated two-dimension civic-economic competency and the gender gap check will be presented. The results show the usefulness of the instrument to assess civic-economical competence in commercial apprenticeship. Our presentation will also be focused on the methodological challenges faced while developing an online instrument, analyzing its psychometric properties and checking the gender gap.
Career Anchors Self-Assessment – In-depth analysis of a German version measuring 9 Career Anchors Zurich University of Applied Sciences, Switzerland; marc.schreiber@zhaw.chmarc.schreiber@zhaw.ch
The career anchors developed by Schein (1975; Schein & van Maanen, 2013) have been continuously refined, and consist of 3 components of the career-self-concept: self-perceived talent and abilities; self-perceived motives and needs; and self-perceived attitudes and values. Schein defines eight career anchors (i.e Technical/functional competence; General managerial competence; Autonomy/independence; Security/stability; Entrepreneurial creativity; Service/dedication to a cause; Pure challenge and Lifestyle) and posits that individuals’ career choices in the past and in the future are affected by the predominant career anchor of a person. In line with today’s call for qualitative and quantitative approaches in career counseling (e.g. Savickas, 2012) career anchors can and should be assessed by a questionnaire supplemented by a qualitative interview. Danziger, Rachman-Moore, and Valency (2008) recommend splitting Entrepreneurial creativity into Entrepreneurship and Creativity. Following this suggestion we will present results of a German version measuring 9 career anchors. In-depth analysis of the factor structure and the differential item functioning (DIF) were conducted with a online sample of 3000 persons. Practical implications for career counselling will be discussed. Practical implications will rely on the use of norm versus raw data and on the combination of qualitative and quantitative approaches in career counselling.
|
| 10:15am - 11:45am | PA26: Clinical Assessment 5 Session Chair: Sofia Major |
| KOL-G-221 (Ⅴ) | |
|
|
Do we have a useful tool to assess preschoolers’ social skills and problem behaviors? University of Coimbra, Portugal; smajor@fpce.uc.ptsmajor@fpce.uc.pt
During the last 30 years, there has been an increased interest on childrens' social-emotional assessment and intervention. However, it was only more recently that this awareness has been focused on preschoolers. The aim of this study is to compare 41 3-6 years-old children referred for psychological intervention for problem behavior (PB) with 41 paired typically developing children, with regard to their social skills and problem behaviors. Each child was rated by parents and teachers with the Portuguese version of the Preschool and Kindergarten Behavior Scales – Second Edition (PKBS-2). Results from t test analyses indicated fewer social skills and higher levels of problem behaviors for the PB children as rated by home and school informants, mainly associated with large effect sizes. The discriminant functional analyses revealed that 93.9% and 78.0% (based on parents and teachers’ ratings, respectively) of the children were well included in the respective group. The Over Activity, Opposition/Explosive and Social Cooperation subscales better discriminated between groups and the more frequently rated items were analyzed for both informants. Results are discussed according to the literature available on this issue and highlight the validity of the Portuguese version of the PKBS-2 to assess social skills and problem behaviors in preschool age.
The narrativity of utterances about close interpersonal relationships and level of personality organisation assessment Adam Mickiewicz University, Poland; soroko@amu.edu.plsoroko@amu.edu.pl
According to the object relation theory (in Kernberg’s view) the self-object representations serve as functioning regulators in both emotional and social domains. Thus it should be expected that the level of maturity of psychic structures (and the level of personality organisation as well) will determine the extent of the relational experiences processing. In consequence, also the narrativity of the utterances about close relationships will be affected by this. In the presented study, 95 autobiographical narratives generated by people with different levels of personality organisation (borderline, neurotic, and integrated) were analysed according to narrativity indices in two different modes (lexical; classical content analysis of narrative coherence). Inter-group comparisons revealed significant differences between borderline and integrated group (integrated group has higher coherence, specificity, human-focus and lower self-focus) and borderline and neurotic group (neurotic group has higher indices of orientation, structure, and concreteness). Results are discussed in the light of (1) the legitimation of the presented model of connections between intrapsychic structures (level of personality) and external manifestations (narratives), and (2) the usability of selected narrativity indices in clinical diagnosis, especially in understanding the processes of adaptation and self-regulation.
Psychodynamic diagnostics – Validation research based on computer modelling of handwriting psychology 1ASP/FSP/SBAP/SGG, Switzerland; 2SGG, Switzerland; m.a.nauer@bluewin.chm.a.nauer@bluewin.ch
New results in Neurophysiology obviously confirm approved theoretical models in handwriting psychology. Handwriting is brain-writing, and the patterns of micro-movements on paper reflect processes in the brain. Therefore, handwriting psychology allows highly efficient psychodynamic diagnostics while describing the dynamic and functional structure of personality. These results encourage reviewing the validity of the handwriting psychology, since the existing studies both with positive conclusions and with critical approach suffer from significant methodological drawbacks and negligence. The new approach is based on computer modelling of the handwriting psychology that ensures the objectivity and the reliability of research. It allows meeting the complexity of the topic by fundamental statistical investigations and systematic validation against various diagnostic instruments (psychological tests, personality inquiries, assessment centre etc.). Several new studies with significant results illustrate the approach.
Management of menace of drug abuse among students by understanding their personality and family dynamics: A longitudinal study GGS Indraprastha University, India; anusinghlather@ipu.ac.inanusinghlather@ipu.ac.in
This paper attempts to make a comparison between personality disposition and perception of parental attitudes toward drug abuse amongst students in 1985 and in 2015. This is a longitudinal study conducted by the authors. The sample in both the phases of study consisted of only male students between the ages of 17 and 25. In the first phase, conducted in 1985, a sample of 155 abusers and 155 non abusers was taken, out of which 117 drug abusers were from urban area and 38 were from rural area, and there were 113 non abusers from urban area and 42 from rural area. In the second phase, conducted in 2015, a total number of 100 respondents were taken out of which 50 were abusers and 50 were non abusers. Out of this sample, 41 drug abusers were from urban areas and 9 were from rural areas, and there were 36 non abusers from urban area and 14 from rural area. The tools of measurement used were Multi Phasic Personality Questionnaire by Murthy, Laxminarayan, Satyavathi (1964). In order to measure perception of parental attitudes the test used was Parental Attitude Questionnaire by Spence and Helmreich (1979).
|
| 11:45am - 12:15pm | Coffee Break |
| Lichthof/Atrium | |
| 12:15pm - 1:15pm | K6: Measuring Adaptive and Maladaptive Personality for Workplace Applications Session Chair: Johnny Fontaine Deniz S. Ones (University of Minnesota, USA) |
| KO2-F-180 (Ⅵ) | |
| 1:15pm - 2:30pm | Lunch |
| Cafeteria | |
| 2:30pm - 3:30pm | Closing Ceremony |
| KO2-F-180 (Ⅵ) | |
| 4:00pm - 6:00pm | City Tour |
Hide Presentations
Table View
