Original Photo adapted from Hansueli Kramer / CC BY
Original Photo adapted from Hansueli Kramer / CC BY
|
|
Session Overview |
|
| |
| 9:45am - 11:15am | PA14: Test Evaluation Session Chair: Samuel M.Y. Ho |
| KOL-E-18 (Ⅷ) | |
|
|
Emotional ratings of 160 Chinese words City University of Hong Kong, Hong Kong; munyinho@cityu.edu.hkmunyinho@cityu.edu.hk
There is currently no validated list of Chinese emotional words for research, despite the fact that Chinese is one of the most widely used languages in the world. This study will present the results of valence, threat, and arousal ratings of 160 Chinese words among 160 adolescents with ages between 12 and 17 in Hong Kong and Mainland China. Valence ratings showed that there were 25 positive (15.6%), 90 neutral (56.3%), and 45 (28.1%) negative words on the list, and their valence classifications were identical to their corresponding English words according to other studies. Twenty words (12.5%) were categorized as high arousal and 15 words (9.4%) as high threat. This study identified eight words with high threat, arousal, and negative valence. They were: annoyed, assault, beating, cancer, dying, horror, suffocate, and surgery. Bi-directional relationships of the three emotional dimensions were consistent with previous studies. In essence, a U-shape relationship was obtained between valence and arousal whereas linear relationships were obtained for other bi-dimensional relationships in the expected directions. Good split-half Inter-Class Correlations were obtained for the three emotional dimensions: valence (.98), arousal (.84), and threat (.96). The word list should be a useful tool to facilitate cross-cultural cognitive research.
Polish adaptation of Fatigue Assessment Scale Adam Mickiewicz University, Poland; joanna.urbanska@amu.edu.pljoanna.urbanska@amu.edu.pl
The purpose of the present study was to adapt the Fatigue Assesment Scale (FAS; Michielsen at al. 2004) for use in Poland, on the basis of theoretical principles of classical test theory as well as general requirements for adaptation (ERA/APA/NCME, 1999/2007). FAS was translated into Polish using direct and reverse translation. The study included 454 respondents (295 females, 159 males; ages from 24 to 85, M= 60) from different socio-economic groups. In general the Polish version of FAS demonstrated good reliability ratings (the Cronbach's alpha for the total scale was .86). Results show that psychometric properties of the Polish version are similar to the original version. Exploratory factor analysis showed that FAS is a homogeneous scale. Good psychometric properties allow for the conclusion that the Polish version of FAS is a suitable instrument for the assessment of the chronical fatigue at adults. Moreover, some intresting significant correlations between FAS and other scales (for example: Everyday-life Fatigue Questionnaire, Urbańska 2010; WHOQOL-Bref, Jaracz 2001) were obtained.
Stroop Color-Word Interference Test: Normative data for the Latin American adult population 1University of Deusto, Spain; 2Universidad del Norte, Colombia; 3Universidad Antonio Nariño, Colombia; 4Universidad Surcolombiana, Colombia; 5Universidad San Buenaventura, Colombia; 6Universidad Javeriana, Colombia; 7Universidad Autonoma de Manizales, Colombia; 8Virginia Commonwealth University, USA; 9IKERBASQUE. Basque Foundation for Science, Spain; dfriverac@deusto.esdfriverac@deusto.es
The objective of this study was to generate the first and most comprehensive demographic-adjusted norms for the Stroop Color-Word Interference Test in Latin-American adult population. The sample consisted of 4856 healthy adult participants from 10 countries in Latin-American representing the demographic distribution of the population. Inclusion criteria were to have a Mini-Mental State Examination (MMSE) score of≥23, have a Patient Health Questionnaire–9 (depression) score of≤4, and have a Barthel Index of≥90. 61.1% were women, the average age was 53.8±19.9 years (range 18-90), and the average education was 10.0±5.0 years. Participants completed the Stroop Test. In the results, pearson correlation coefficient and coefficient of determination showed significant effect between Stroop test scores and MMSE, age, and education (r-squared>0.430; p’s<.001). Multinomial logistic regression yielded main effects for age and education groups on the Stroop scores (r-square Cox & Snell>.322, p’s<.001). Consequently, correction tables were created in order to adjust the raw scores based on age and education. The percentiles tables were calculated based on the distribution of adjusted scores. This study is the first to create Stroop test norms in the Latin-American population and include appropriate adjustments for age and education. These data represent a critical advancement in the assessment in Latin-American adults.
Hopkins Verbal Learning Test–Revised: Normative data for the Latin American adult population 1University of Deusto, Spain; 2Instituto Nacional de Neurología y Neurocirugía MVS, Mexico; 3Universidad Interamericana de Puerto Rico, Puerto Rico; 4Instituto de Previsión Social, Paraguay; 5Centro Internacional de Restauración Neurológica, Cuba; 6Centro de Investigaciones Psicológicas y Neuropsicológicas, Bolivia; 7Universidad Cesar Vallejo, Peru; 8Virginia Commonwealth University, USA; 9IKERBASQUE. Basque Foundation for Science, Spain; dfriverac@deusto.esdfriverac@deusto.es
The objective of the study was to generate the first and most comprehensive demographic-adjusted norms for the Hopkins verbal learning test–Revised (HVLT-R) in Latin-American adult population. The sample consisted of 4856 healthy adult participants from 10 countries in Latin-American representing the demographic distribution of the population. Inclusion criteria were to have a Mini-Mental State Examination (MMSE) score of≥23, have a Patient Health Questionnaire–9 (depression) score of≤4, and have a Barthel Index of≥90. 2967 were women and 1889 were men, the average age was 53.8±19.9 years (range 18-90), and the average education was 10.0±5.0 years. Participants completed the HVLT-R. Pearson correlation coefficient and coefficient of determination showed significant effect between HVLT-R scores and MMSE, age, and education (r-squared -0.474 to 0.463; p’s<.001). Multinomial logistic regression yielded main effects for age and education groups on the HVLT-R scores (r-square Cox & Snell>.269, p’s<.001). Consequently, correction tables were created in order to adjust the raw scores based on age and education. The percentiles tables were calculated based on the distribution of adjusted scores. This study is the first to create HVLT-R norms in the Latin-American population and include appropriate adjustments for age and education. These data represent a critical advancement in the assessment in Latin-American adults.
|
| 11:45am - 1:15pm | PA18: Innovative Approaches Session Chair: Samuel Greiff |
| KOL-E-18 (Ⅷ) | |
|
|
Challenges of constructing and validating innovative assessments: The example of collaborative problem solving University of Luxembourg, Luxembourg; samuel.greiff@uni.lusamuel.greiff@uni.lu
Since the advent of computers to the field of psychological assessment, there have been tremendous changes in how psychological assessment instruments are constructed. Specifically, some psychological constructs require fast and precise tracking of reactions, which have only become reliably measurable through computer-based assessment (e.g., processing speed). In this presentation we give an overview of the construction and validation of COLBAS - a computer-based assessment tool for collaborative problem solving (ColPS). ColPS is defined as “problem solving activities that involve interactions among a group of individuals”(O`Neil et al., 2003), and its assessment requires logging and scoring of collaborative behaviors over the course of problem-solving (e.g., frequency and content of information exchange, or activities towards solving the problem). In this presentation, we propose how such behaviors can be captured and scored in ColPS assessments that simulate collaboration through computer-agents. We further discuss advantages and limitations in design, face-validity, and scalability of computer-agents as collaborators. Moreover, we report on the internal validity of COLBAS shown in the first undertaken validation study (N>450, Mage=13.5, SDage=.61, 50.2% girls) by discussing the internal structure explored in a latent confirmatory factor analysis. Finally, we outline further steps necessary to examine validity of COLBAS as ColPS assessment.
Automatic coding of short text responses via clustering in educational assessment 1Technische Universität München, Germany; 2German Institute for International Educational Research (DIPF), Germany; 3Centre for International Student Assessment (ZIB), Germany; fabian.zehner@tum.defabian.zehner@tum.de
Automatic coding of short text responses opens new doors in assessment. We implemented and integrated baseline methods of natural language processing and statistical modelling by means of software components that are available under open licenses. The accuracy of automatic text coding is demonstrated by using data collected in the Programme for International Student Assessment (PISA) 2012 in Germany. Free text responses of 10 items with 41,990 responses in total were analyzed. We further examined the effect of different methods, parameter values and sample sizes on performance of the implemented system. The system reached fair to good, and even up to excellent agreement with human codings
(Cohen's kappa: .458 ≤ κ ≤ .959). Especially items that are solved by naming specific semantic concepts appeared properly coded. The system performed equally well with sample sizes of 1661 and higher and somewhat poorer but still acceptable down to sample sizes of 249. Based on our findings, we discuss potential innovations for assessment that are enabled by automatic coding of short text responses. A latent transition analysis of the computer usage of adolescents: Longitudinal transition and testing the effect of predictors Korea University, South Korea; soyoungkim.ems@gmail.comsoyoungkim.ems@gmail.com
The present study examined the motivations of computer usage in subgroups (latent profile) and tested the effects of predictors on determining these latent profiles. Latent transition analysis was applied to the three time point data of the Korea Children and Youth Panel Survey (middle school cohorts). The results showed that there were five latent profiles for each year that could be defined based on their response patterns: game oriented group, music movie group, middle level group, SNS group, and high risk group. Female students tend to be in the mid level group compared with male students. Students with low parental control tended to be in the game oriented group, the SNS group, and high risk group. Members of the game oriented group and the mid level group have consistently shifted to the high risk group. Members of the music movie group and the SNS group rarely transformed each other. These findings will be helpful to design intervention programs for excessive computer usage.
Who is affected by FoMO (Fear of Missing Out)? – Measuring a new phenomenon in the digital era 1RFH Köln, Germany; 2HS Fresenius, Germany; christian.bosau@rfh-koeln.dechristian.bosau@rfh-koeln.de
In todays digital era people use mobile phones almost everywhere. Lately, the construct Fear of Missing Out (FoMO) is discussed as a reason for people’s mobile phone usage (JWT, 2012; Przybylski, Murayama, DeHaan & Gladwell, 2013). However, neither the measurement of this phenomenon nor its correlates to other constructs have been sufficiently discussed yet. This online-study (n=100) analyzed the relationship between the up-to-now only FoMO-scale (Przybylski et al., 2013) and the Big-5 personality inventory (Rammstedt, Kemper, Klein, Beierlein, & Kovaleva, 2012), self-esteem (Collani & Herzberg, 2003), self-monitoring (Collani & Stürmer, 2014) and affiliative tendency (Mehrabian & Ksionzkys, 1974). Furthermore, the quality of this scale by Przybylski et al. (2013) is reassessed in a CFA. Regarding convergent validity the FoMO-scale correlates with neuroticism (r=.36, p<.001), self-monitoring (other-directedness: r=.46, p<.001 & acting: r=.21, p<.05), self-esteem (r=-.41, p<.001) and sensitivity to rejection (r=.47, p<.001). Regarding discriminant validity, no correlations exist with conscientiousness, openness, and agreeableness (however: r=-.30, p<.00 with extraversion should be critically discussed). Though, a CFA-reanalysis of the FoMO-scale shows far better fit indices, if several items are removed. Likewise, the convergent validity indices improve into the expected direction. Hence, the study further improves the understanding of FoMO and its measurement.
Information search in objective personality tests: Objective assessment of perspective taking University of Salzburg, Austria; thomas.scherndl@sbg.ac.atthomas.scherndl@sbg.ac.at
Decisions are part of our daily life and the way how we make choices can tell much about us and our personality. Surprisingly, the process of making a decision has received less attention in objective personality test research than would be expected. In the present study, we investigated the usefulness of information search process data as part of an objective personality test capturing the trait ‘perspective taking’. We presented complex, multi-attribute prediction tasks to participants (n=126, 78 female, mean age = 21.4 years) and tracked their information search including search duration, number of acquired information and search direction. Additionally, we also included multi-attribute decision tasks for oneself and the scale ‘perspective taking’ of the Interpersonal Reactivity Index (IRI; Davis, 1983) to test for discriminant and convergent validity. Process data showed moderate to high internal consistency. Additionally, the questionnaire scale correlated with duration and number of information pieces inspected before a prediction task, but not with information search before decisions for oneself, thus demonstrating convergent and discriminant validity. We discuss the promise of information search as part of objective personality tests, but also mention potential pitfalls and yet unsolved problems of information search process data.
|
Hide Presentations
Table View
