I want to believe? No, we test it!

The iCamp project is validation-driven with the overarching goal of identifying improvement suggestions to refine the conceptual models as well as technological requirements for successful online cross-cultural collaborative learning. Several objectives are to be fulfilled through three validation trials, which have different foci and scales and involve different Higher Education Institutions in Europe. The research methodology to be followed in iCamp is closely related to the design-based research, which, with the deployment of mixed methods, aims to meet dual goals of refining locally educational practice or intervention and developing more globally usable knowledge for the field. Whereas the first validation trials (Oct-Dec 2006) will mainly be exploratory, the second and third validation trials are formative (or diagnostic) and summative, respectively. iCamp will deal with the challenges not only in design and development but also in evaluation and validation. A generic evaluation framework by adapting the ValNET matrix has been developed; two overarching evaluation objects, namely Products/Resources (e.g. technological platforms, communication tools and educational material) and Results (e.g. outcomes and impacts), will be analyzed along five dimensions – technological, pedagogical, cultural, institutional and economic. Presumably, this generic framework is applicable for the three iCamp validation trials. In addition, we define a specific evaluation framework for computer-supported collaborative learning (CSCL). Evaluation of CSCL is known to be a highly complex and resource-demanding endeavour. Specifically, we will adopt mixed method evaluations by blending qualitative and quantitative approaches and by integrating social network analysis (SNA). Such triangulation of approaches is deemed indispensable for enhancing the validity and reliability of results to be inferred and for deepening the evaluators’ understanding about the issues of interest. In summary, the iCamp evaluation framework places emphasis on relational and attributional data. The former addresses the evolution of social networks and their associated factors, whereas the latter addresses the development of individual competencies/attributes and their associated factors. Apart from these two types of data, indexical data in terms of quantifiable variables and ideational data in terms of evocative and volitional constructs will be studied as well, albeit to a lesser extent.

In accord with the three areas of challenges identified by WP1, namely cross-cultural collaboration (CCC), self-directed learning (SDL) and social networking (SNW), we propose corresponding analytic and empirical evaluation frameworks. For CCC, we focus on evaluating the three aspects of the collaborative process: conversational grounding, quality of interaction and task performance. For SDL, we look into how learners make use of learning contract and assess or self-assess their SDL competencies. For SNW, we analyze three types of social network development – direct, indirect and use of resources. A plethora of data capturing and data analysis methods as well as tools are required, including observations (self- and other-oriented), questionnaire (close and open-end questions), interviews (remote computer-mediated and face-to-face), discussion forums (synchronous and asynchronous), diary writing (e.g. blogs, e-Portfolio), etc. Concomitantly, a large amount of qualitative and quantitative data will be collected. Processing of the data will be supported with the use of open source (e.g. TAWC for text analysis, Pajek for SNA) as well as proprietary software (e.g. SPSS, ATLAS.ti). Sophisticated data analysis techniques such as exploratory sequential data analysis (ESDA) for continuous observational data and interaction analysis for video data will be employed. Besides, a variety of rating scales and inventories such as Self-Assessment for Self-Directed Learning Competencies (SDLC), Cultural Perspective Questionnaire (CPQ), and Technology Acceptance Model (TAM) will be adapted for the particularities of the iCamp context. Similarly, existing coding schemes for content analysis will be modified to fit the purpose of the validation trials. Apparently, integrating multi-perspective and poly-contextual data and inferring implications from empirical findings to the future development of CSCL in the Enlarged Europe will pose real challenges for the iCamp validation team.

WP4 is intended for conducting of trials with students and educators to implement the social, pedagogical and scaffolding model as well as appropriate support insured by the technical infrastructure.

– Effie Law, Anh Vu Nguyen-Ngoc