Since the advent of computers to the field of psychological assessment, there have been tremendous changes in how psychological assessment instruments are constructed. Specifically, some psychological constructs require fast and precise tracking of reactions, which have only become reliably measurable through computer-based assessment (e.g., processing speed). In this presentation we give an overview of the construction and validation of COLBAS - a computer-based assessment tool for collaborative problem solving (ColPS). ColPS is defined as “problem solving activities that involve interactions among a group of individuals”(O`Neil et al., 2003), and its assessment requires logging and scoring of collaborative behaviors over the course of problem-solving (e.g., frequency and content of information exchange, or activities towards solving the problem). In this presentation, we propose how such behaviors can be captured and scored in ColPS assessments that simulate collaboration through computer-agents. We further discuss advantages and limitations in design, face-validity, and scalability of computer-agents as collaborators. Moreover, we report on the internal validity of COLBAS shown in the first undertaken validation study (N>450, Mage=13.5, SDage=.61, 50.2% girls) by discussing the internal structure explored in a latent confirmatory factor analysis. Finally, we outline further steps necessary to examine validity of COLBAS as ColPS assessment.

Automatic coding of short text responses opens new doors in assessment. We implemented and integrated baseline methods of natural language processing and statistical modelling by means of software components that are available under open licenses. The accuracy of automatic text coding is demonstrated by using data collected in the Programme for International Student Assessment (PISA) 2012 in Germany. Free text responses of 10 items with 41,990 responses in total were analyzed. We further examined the effect of different methods, parameter values and sample sizes on performance of the implemented system. The system reached fair to good, and even up to excellent agreement with human codings

(Cohen's kappa: .458 ≤ κ ≤ .959). Especially items that are solved by naming specific semantic concepts appeared properly coded. The system performed equally well with sample sizes of 1661 and higher and somewhat poorer but still acceptable down to sample sizes of 249. Based on our findings, we discuss potential innovations for assessment that are enabled by automatic coding of short text responses.

The present study examined the motivations of computer usage in subgroups (latent profile) and tested the effects of predictors on determining these latent profiles. Latent transition analysis was applied to the three time point data of the Korea Children and Youth Panel Survey (middle school cohorts). The results showed that there were five latent profiles for each year that could be defined based on their response patterns: game oriented group, music movie group, middle level group, SNS group, and high risk group. Female students tend to be in the mid level group compared with male students. Students with low parental control tended to be in the game oriented group, the SNS group, and high risk group. Members of the game oriented group and the mid level group have consistently shifted to the high risk group. Members of the music movie group and the SNS group rarely transformed each other. These findings will be helpful to design intervention programs for excessive computer usage.

In todays digital era people use mobile phones almost everywhere. Lately, the construct Fear of Missing Out (FoMO) is discussed as a reason for people’s mobile phone usage (JWT, 2012; Przybylski, Murayama, DeHaan & Gladwell, 2013). However, neither the measurement of this phenomenon nor its correlates to other constructs have been sufficiently discussed yet. This online-study (n=100) analyzed the relationship between the up-to-now only FoMO-scale (Przybylski et al., 2013) and the Big-5 personality inventory (Rammstedt, Kemper, Klein, Beierlein, & Kovaleva, 2012), self-esteem (Collani & Herzberg, 2003), self-monitoring (Collani & Stürmer, 2014) and affiliative tendency (Mehrabian & Ksionzkys, 1974). Furthermore, the quality of this scale by Przybylski et al. (2013) is reassessed in a CFA. Regarding convergent validity the FoMO-scale correlates with neuroticism (r=.36, p<.001), self-monitoring (other-directedness: r=.46, p<.001 & acting: r=.21, p<.05), self-esteem (r=-.41, p<.001) and sensitivity to rejection (r=.47, p<.001). Regarding discriminant validity, no correlations exist with conscientiousness, openness, and agreeableness (however: r=-.30, p<.00 with extraversion should be critically discussed). Though, a CFA-reanalysis of the FoMO-scale shows far better fit indices, if several items are removed. Likewise, the convergent validity indices improve into the expected direction. Hence, the study further improves the understanding of FoMO and its measurement.

Decisions are part of our daily life and the way how we make choices can tell much about us and our personality. Surprisingly, the process of making a decision has received less attention in objective personality test research than would be expected. In the present study, we investigated the usefulness of information search process data as part of an objective personality test capturing the trait ‘perspective taking’. We presented complex, multi-attribute prediction tasks to participants (n=126, 78 female, mean age = 21.4 years) and tracked their information search including search duration, number of acquired information and search direction. Additionally, we also included multi-attribute decision tasks for oneself and the scale ‘perspective taking’ of the Interpersonal Reactivity Index (IRI; Davis, 1983) to test for discriminant and convergent validity. Process data showed moderate to high internal consistency. Additionally, the questionnaire scale correlated with duration and number of information pieces inspected before a prediction task, but not with information search before decisions for oneself, thus demonstrating convergent and discriminant validity. We discuss the promise of information search as part of objective personality tests, but also mention potential pitfalls and yet unsolved problems of information search process data.