This paper appears in the International Journal of IT Standards and Standardization Research Vol. 8, No. 2, edited by Tore Hoel, Paul A. Hollins and Jan M. Pawlowski, editor-in-chief Kai Jakobs. Copyright 2010, IGI Global, www.igi-global.com. Posted by permission of the publisher.
Reformatted for Simon Grant's publications
Jad Najjar, WU Vienna, Austria
Michael Derntl, University of Vienna, Austria
Tomaž Klobučar, Institut “Jožef Stefan”, Ljubljana, Slovenia
Bernd Simon, WU Vienna, Austria
Michael Totschnig, WU Vienna, Austria
Simon Grant, JISC CETIS, UK
Jan Pawlowski, University of Jyväskylä, Finland
Employers seek people matching particular qualifications and graduates seek jobs matching their qualifications. This market is currently managed primarily using paper certificates and heterogeneous university management systems that capture achieved learning outcomes as well as corporate information systems that capture required qualifications. In the light of current trends towards increased student mobility, employability and lifelong learning, this situation is less than satisfactory. Therefore, we propose a schema in this paper that facilitates interoperable storage and management of Personal Achieved Learning Outcomes (PALO) based on a common data model. We present use case scenarios and implementations addressing these challenges and demonstrating the added value of using such a common model.
Keywords: Competences, Learner Achievements, Learning Outcomes, Outcome-Based Learning, Skills, Standards
Nowadays, the management and interoperability of data about learning outcomes (knowledge, skill and competence) in outcome based learning are of high importance for both education and employment sectors. By managing and sharing data about their profiles, learners in higher education and lifelong learning can better plan their careers and enhance their employability potential. In order to achieve this goal, the information about learning outcomes associated to learning opportunities and units of learning as well as the learning outcomes achieved by learners need to be captured, managed and exchanged in common formats (Paquette, 2007; Van Assche, 2007; Sampson, 2009; Lindgren et al., 2004).
Currently, in the context of online learning, higher education institutions have yet adopted neither a common format for describing learning outcome definitions nor formats for describing achieved learning outcomes of learners. Such data are gathered in personal profiles and give information about the context where the outcomes were achieved (Richter & Pawlowski, 2007) by taking a learning opportunity, the achieved outcome level, and assessment information (Crespo et al., 2010; Grant, 2002).
Using a common format for describing, referencing and sharing learning outcome definitions enables exchange and comparison of learning outcome definitions across systems, domains and sectors. For instance, data about learning outcomes achieved in one course or by one student and stored at a university database may be exchanged with a Human Resources (HR) system of an employment agency. In other words, the two systems can refer to a common definition of a learning outcome with a common meaning. By this way, a lifelong learner would be able to access his/her achieved learning outcomes from the school, the university, the training agencies and the employer in one profile. This work would enable the matching between what people learn in universities (and schools) and what they need to know and be able to do at work. The gap in learner skills and knowledge may be easily identified (Paquette, 2007).
Following the European initiatives like European Qualification Framework (EQF, 2000), Europass (2010) and European Learner Mobility (EuroLM, 2009), the aim of this paper is to introduce a specification that captures information on knowledge, skills and competences achieved by a person (a lifelong learner) in higher education and training institutions or in the workplace. The specification represents data on relations between a learner's achieved learning outcomes. In addition to that, information on the context where the learning outcomes are obtained or applied is covered by schema. Evidence (assessment) records and levels (e.g. proficiency level) associated to the outcomes are also a core part of this schema. This specification is an important step towards the enhancement of the interoperability and transparency of such personal data of a lifelong learner between higher educational and workplace applications and services.
The Personal Achieved Learning Outcomes (PALO) specification presented in this paper went through at least three iterative expert evaluations by the ICOPER consortium and at relevant international workshops by standards experts, teachers and learners to make sure that it captures data needed for increased employability of learners and higher interoperability with different learning systems.
Prototypes of outcome based learning applications like widgets and modules of Learning Management Systems (LMS) are being developed, to produce and import data about achieved learning outcomes of learners in systems like Moodle (2010), Elgg (2010) and Clix (2010). The data of learner achieved learning outcomes profiles are stored in ICOPER's PALO repository. These data can be consumed by learning systems to provide learners with relevant material, recommendation of other teachers and learners based on similarity of learning outcome profiles, or to enable learners to share their achievement profiles with social or recruitment systems.
This paper is structured as follows: In the second section, related work regarding outcome-based learning, competence and learning outcome specifications is provided. Then, in the third section, the outcome based learning components and features are introduced. The fourth section provides a scenario of learner achievements in outcome-based learning. Afterwards, fifth section, the Personal Achieved Learning Outcomes (PALO) data model is presented. This data model captures information, in a profile, on learning outcomes achieved by a learner after successfully following a learning opportunity (unit of learning). The data presented in a PALO profile can be imported to different learning, training and recruitment systems or social applications like LinkedIn (2010) and Facebook (2010) for presentation purposes, or for recommendation of relevant units of learning, people or jobs. The sixth section presents prototypical implementations being developed in ICOPER (2010) project for learning management systems like Moodle (2010) or Social applications like Facebook. Conclusions and future directions are drawn at the end. A detailed description of the PALO elements, data types and values can be found at http://www.icoper.org/schema/palov1.1.
The importance of capturing data about life-long learners' knowledge, skills and competences has been stated by several national/international initiatives, standardisation bodies and researchers. The European Qualification Framework (EQF, 2000) was proposed as an instrument to make learning outcomes more transparent and comparable across Europe. This is an important step towards outcome-based learning and mobility. Following the recommendations of EQF, the European Learner Mobility (EuroLM, 2009) stated the importance of including data on intended learning outcomes to the Europass Diploma Supplement record. Personal achieved learning outcomes specifications can be seen as a complementary part to the Europass Diploma Supplement (DS). Specifications and services for management and exchange of personal achievement information should be developed to support the expression of the European lifelong learner achievement information across higher education, training and employment sectors, this should result in:
Metadata for Learning Opportunities (MLO, 2008) specification covered, to some extent, the linkage of learning outcomes to metadata of learning opportunities.
The important role of personal competence profiles in human resource management and education was emphasised by Paquette (2007), who demonstrates how a gap in learner skills can be identified in competence-based ontology-driven e-learning systems.
With a focus on linking metadata records of learning resources to intended competences of courses, Van Assche (2007) and Sampson (2009) extend the IEEE Learning Object Metadata (LOM; IEEE, 2002) records with some attributes that capture generic characteristics of an intended competence, like title, type, description, proficiency level and context.
Emphasising the importance of aligning organisation level core competences with individual level job competence, in competence management systems, (Lindgren et al. 2004) develop an integrative model of competence that incorporates a typology of competence-in-stock, competence-in-use, and competence-in-making. The authors found company systems substantiating the job-based approach with predefined job descriptions and taxonomies of formal competence create barriers to the use appropriate use of a competence management system, On the other hand, authors suggest that HR systems that adopt the skill-based approach must be accompanied by sufficient user control over the information that represents her competence.
Regarding competence standards and specifications, IEEE LTSC Reusable Competency Definitions (IEEE RCD, 2007) is a world-wide standard that defines a data model for describing, referencing and sharing competence definitions, primarily in the context of online and distributed learning. IEEE RCD specification has a rather narrow scope, focusing on representing the key characteristics of a learning outcomes, it offers technical and semantic interoperability and (to some extent) extensibility. IEEE RCD is based on the existing IMS Global Learning Consortium specification entitled IMS Reusable Definition of Competency or Educational Objective (IMS RDCEO, 2002).
HR-XML (2009) is a specification that includes the possibility of formalization and ranking of competences, supported by HR-XML Consortium. The specification is, among other things, used to capture information about evidence used to substantiate a competence, their ratings and weights and reusable data typed for referencing competences. Competence data elements of this schema are designed in order to be used in process-oriented environments and can be used to rate, measure, match and asses competence against one that is demanded, required (e.g. as a prerequisite, skill requited to take training). HR-XML has a strong focus on management and business and makes use of very clear non-extendable semantics expressed by the XML structure, which is likely to make the standard less flexible, but easier to maintain compatibility on a basic level. This is obviously a result of the firmly demarcated scope and focus of HR-XML.
As far as e-portfolio data is concerned, Leap2A (2008) sets up the framework for the e-portfolio data interoperability. The purpose of the Leap2A specification is to represent e-portfolio information collected by the individual (learner) and not the information stored by the others (institutions/teachers) about this individual. This information can be digital artefacts and users reflection to some kind of information. This specification extends ATOM syndication format to allow representation of such rich data. More analysis of competences and learning outcomes related specifications is provided in (Najjar & Klobučar, 2009).
A learner's knowledge is about what the learner knows, or is presumed to know. Knowledge or belief does not automatically turn into behaviour, but needs to be applied to be of practical use. Skills, in contrast, are about patterns of behaviour in the learner's actions. In practice, all skills are supported by knowledge. Competences are more complex; they involve the application of knowledge and skill, but they are also about the kinds of situations (context) in which the knowledge and skill are applied. These kinds of situation are patterns that are instantiated in the world just at those times and places where the competence is applicable (EQF, 2000).
To exemplify the distinction of knowledge, skills and competence, consider this medical nurse scenario:
Learning needs are the skills, knowledge and competences that need to be attained by an individual in order to function at the desired level when performing a particular task or job. Learning needs cover also the skills, knowledge and competences that need improvement (Grant, 2002). The successful involvement of a learner in the learning and assessment process enables the learner to attain those needs. Those needs are also defined in terms of the learning outcomes of a learning process according to the European Qualification Framework (EQF, 2000):
“Learning outcomes are statements of what a learner knows, understands and is able to do on completion of a learning process”.
Following the EQF, in the ICOPER (2010) project, the term learning outcomes is used to refer to knowledge, skills and competences intended or achieved by learners. This model enables management of those definitions to enable and facilitate their finding and reuse across different units of learning as intended learning outcomes. In the coming sections of this paper, we introduce a data model for describing a profile of learning outcomes achieved by an individual (learner). This data model enables learners to share their profile of achieved outcomes with third-party applications for applying to future learning opportunities or employment or for recommendation of relevant information or services.
The Learning Outcome Definitions (LOD) data model has been defined in ICOPER as a conceptual base schema for describing and sharing learning outcome definitions in the context of online and technology enhanced learning. The data model provides a way to capture the key characteristics of a learning outcome, independently of its use in any particular context or target group (persons). This model should enable the storage, retrieval and exchange of learning outcomes across systems that deal with learning outcomes data.
The LOD model has been based on the IEEE Reusable Competency Definitions standard (IEEE RCD, 2007). IEEE RCD is a widely accepted standard for describing generic learning outcomes. In a nutshell, the LOD schema includes four descriptive elements:
In universities, it is common practice for every taught module to have an associated assessment activity. This may lead to the assumption that an assessment is somehow an integral part of a learning activity (Crespo et al., 2010). However, a variety of formal, informal or self-study activities may all be preparation for the same assessment. Indeed, some assessment processes (such as psychometric tests) may be expected to be taken without any associated learning process.
Where learning and assessment are related, the logic would seem to be that the learning process aims at producing the intended learning outcomes in the learner, while the assessment method is designed to assess the learner's attainment of those outcomes. Formal assessment produces some formal result, according to a formally defined process. That result may fall into a certain pattern (pass/fail, or being over or under a certain percentage mark, for instance) and this is taken as an evidence for whether the learner has attained the intended learning outcome. The evidence is recorded and issued by the institution that has offered the learning opportunity.
One of the useful applications of technology based on these models is for matching people with opportunities, that is, trying to get people into situations to which they are suited.
What is called here a personal achievement is intended to represent the way in which people claim that they are good at some thing. Achievements may have different forms and characteristics, e.g.:
Most formally defined opportunities — opportunities of employment as well as learning — have some kind of selection process, and that process typically specifies the kinds of knowledge, skill or competence that people are expected to have. A process of comparison can be undertaken between achievement and requirement. Does the assessment result in the achievement fit into the required pattern? Do the knowledge, skill or competence claimed to be achieved match the ones which are required? Do the contexts referred to in the achievement match the contexts expected in the requirement? The greater clarity about this matching process that is achieved through modelling can potentially serve as the basis for a greater degree of effective automation in the initial stages of recruitment or selection.
Let us motivate our work on developing a model that captures learner's achieved learning outcomes by a scenario. The scenario gives an insight into outcome-based learning from a learner's point of view. It also serves as the basis for describing concrete examples of data model elements in the next section.
Peter Smith is a motivated young computer professional from the United Kingdom, working as a programmer in a big software company. Already as a teenager he was dreaming about becoming an entrepreneur in the area of computer games development and founding his company before the age of 30. The computer science programme Peter attended at a local university lacked many learning outcomes Peter needs to obtain while pursuing his goal, especially non-computer science related ones, for example in management and finances. Peter also has to regularly update his computer science knowledge and skills to be able to follow the rapid scientific and technological development.
In an attempt to organize an individual learning path, Peter wants first to analyse his knowledge, skills and competence gaps and clearly define his learning needs in terms of learning outcomes. A free on-line service at a career development agency helps him analysing those gaps by automatic-semantic matching of his achievements stored in his personal achieved learning outcome profile to the data the agency has for different occupations and positions. His profile includes both obtained learning outcomes formally assessed by educational institutions and other achievements obtained at work or in informal learning; for example, developed open source games. Peter updates his profile by importing missing learning outcomes as intended learning outcomes.
Based on the identified learning needs, he decides that his first goal is to obtain competences in project management on EQF level 6. Since Peter's university studies he has been maintaining his personal learning environment (PLE) that supports self-directed learning and collaboration. The PLE's tools and services enable Peter to find learning opportunities that best suit his intended learning outcome at different educational institutions as well as other users. They also keep notifying Peter about the changes and upgrades of the computer science programme from his alma mater and suggest topics he should learn to stay in touch with the latest developments. His achievements are taken into account during selection of the best suited learning opportunities. As Peter does not speak other languages than English and German, the search engine, for example, shows only courses in those two languages when presenting search results.
From the list of found courses Peter selects a blended course on project management that also enables him to obtain competences in group leadership. As the course is given by Vienna University of Economics and Business (WUW) he takes an opportunity to use his three months stay in Vienna also for learning purposes. Peter uses a variety of social software tools from his PLE when interacting with a teacher, his assistant and other learners. For the final assessment in the course he has to prepare a small project and lead a group of peer students who will help him implementing the project. His project management and group leadership competences are assessed by the learner supporters and peer learners.
Data collected about personal achieved learning outcomes are generated by the learning system after Peter has successfully completed the course, as shown in Figure 1. Based on the assessment results the university issues Peter a certificate in German that proves obtained learning outcomes, and registers Peter's achievement in the form of an achievement record. The achievement states attainment of all learning outcomes Peter obtained in a certain context in the course, and can be verified on the basis of an assessment record (e.g., certificate). The achievement is stored in Peter's personal achieved learning outcome profile in WUW's PALO repository. Peter can import the achievement also into his personal profile and enhance it with English descriptions.
Figure 1: Generation and use of personal achieved learning outcomes data.
The Personal Achieved Learning Outcomes (PALO) data model is a simple schema proposed to capture information on knowledge, skills and competences achieved by a person (a learner), and the relations between those outcomes. Furthermore, information on the context where the learning outcomes are obtained or applied, evidence records (assessment), and levels (e.g. proficiency level) associated to the outcomes are also part of this schema.
One of the main challenges of communities and systems that deal with learning outcome information is the interoperability issue. Different communities and systems may use different data models to represent information on skills, knowledge or competence obtained by a person or that is required for a job or a task. The PALO specification is a step towards a common model supporting the exchange of such data, to enhance interoperability of personal learning outcome information between, for example, learning management systems, e-portfolios, social applications and recruitment systems.
The PALO data model enables capturing the following information:
The PALO data model covers (with some customization) data elements and concepts related to learning outcomes from other specifications like:
The data collected in person's PALO profile can be used in different ways:
In this section, we briefly describe the main PALO data elements and types. Figure 2 presents a graphical representation of the Personal Achieved Learning Outcomes (PALO) data model, for a detailed description see http://www.icoper.org/schema/palov1.1.
Figure 2: The PALO data model.
There are 6 main elements in the model:
All six elements are uniquely identified; the context and the level by a combination of a scheme and its value, and the other four element by means of URIs. Figure 3, using a concrete example, demonstrates how the PALO specification (shown in Figure 2) can represent the data about the learning outcome achievements of a lifelong learner like Peter, the character presented earlier in the scenario section.
Figure 3: Concrete example of schematic representation of personal achievement profile and learning outcomes
Current work in ICOPER (2010) includes building software prototypes, services and modules that use the PALO model to capture, store and exchange learning outcome data between systems. The prototypes use the Open ICOPER Content Space — OICS (Totschnig, et al., 2009) as an open repository containing different kinds of educational resources collected from distributed providers, ranging from small-scale assets like pictures and documents to complete units of learning, as well as PALO learning outcome profiles of learners and teachers. There are two types of prototype being developed:
The PALO model is primarily targeted towards run-time applications: the idea is that the applications use and update the PALO profile of a learner, e.g. when evidence or context for the achievement of an intended learning outcome is available. Therefore, applications that use PALO are dependent on units of learning that have their intended learning outcomes explicitly stated. This is where the design-time applications come into play. The design-time prototypes being developed enable learning designers to enrich units of learning with definitions of intended learning outcomes, assessment resources, learning resources, and so forth.
To “pick up” as many existing users as possible, the plan was to enable the design and delivery of outcome based learning by extending functionality of existing systems that are already used by teachers and learners. To illustrate this approach of implementing PALO, this section presents a set of extensions to Moodle Learning Management System (Moodle, 2010), which were implemented as prototypes:
This is a Moodle module that enables searching and importing units of learning from the OICS. These units of learning come with intended learning outcomes in their metadata. A screenshot clipping of the search interface is displayed in Figure 4. The screen shows a list of units of learning that have “school geography” in their metadata. In addition it is possible to filter the query by selecting a specific portion of metadata to be queried for the given search string.
Figure 4: Unit of learning search interface in Moodle.
Figure 5 shows a screenshot of the details of a unit of learning after the user clicked on one of the items in the query results. The section most relevant to PALO is the list of intended learning outcomes connected to this unit of learning.
Figure 5: Unit of learning detail display based on available metadata.
After importing the unit of learning, Moodle “knows” what outcomes are to be added to the PALO profiles of learners who pass the course assessment. In the optimal case this can achieved fully automatically, if the intended learning outcomes are linked to the learning assessment in the metadata of the unit of learning. For instance, the unit of learning may include an IMS QTI compliant assessment resource (e.g. a multiple choice test on geography basics) that can be used by Moodle to do the actual assessment and subsequently automatically create an assessment record with information like scores, assessing body and verification date (see Figure 2) that forms evidence for the achievement of specific learning outcomes.
|
After the unit of learning is successfully imported into Moodle, several additional benefits of PALO can be exploited: the system can keep a PALO profile of intended learning outcomes of units of learning that are — or have been — taught by the logged-in teacher user. If this is available for a large group of users (potentially also outside the current Moodle instance), Moodle can recommend other teachers who have similar profiles of taught learning outcomes. This enables teachers to create practitioner communities around the learning outcomes they teach simply based on the intended learning outcomes linked to courses where they have a teacher role in Moodle. A screenshot of a Moodle sidebar widget that recommends similar teachers is given in Figure 6. Another, potentially, even more powerful recommendation approach, can be implemented for the learners: based on outcomes of courses currently being attended, and outcomes already in the PALO profile of a learner, Moodle can instantly recommend other learners, e.g. learners with similar background knowledge who are currently working on achieving the same or similar outcomes. Depending on the richness of user profile data, the recommendation approach can be simple or sophisticated, e.g. filtering the recommendation by physical proximity. Additionally, Moodle can recommend units of learning which address the same learning outcomes as the ones attached to units of learning that the student is currently using. This way, students can access additional resources during learning. A screenshot of a Moodle widget recommending related units of learning is shown in Figure 7. |
Figure 6: Moodle widgets displaying recommended units of learning and recommended teachers. 
|
Figure 7: Moodle widgets showing personal learning outcomes and related units of learning. 
|
After a unit of learning is completed in Moodle, the system can, depending on the local policy (e.g. after explicit clearance is given by the teacher, the institution, and/or the student), update the PALO profile of the learner in the PALO repository. In addition, newly achieved learning outcomes can for instance be published to social networking sites like LinkedIn (2010) and Facebook (2010). To demonstrate this, an Atom XML binding (ATOM, 2010) of the PALO schema was defined. This binding is used to publish PALO profiles to Facebook using a custom Facebook application. The application retrieves the user's public PALO profile and displays it in a separate tab of the personal profile page in Facebook. The screenshot in Figure 8 shows a screenshot mockup of the user's public PALO profile in Facebook.
Figure 8: Facebook profile extended with learning outcomes.
The Moodle extensions presented in this serve as a hint to the potential world of powerful person-centred, outcome-based applications that can be developed building on PALO. On the other hand, the use of PALO data in social applications support the portability and transparency of personal achieved learning outcomes data.
The knowledge, skills and competences achieved by the learner play an increasingly important role as professional life introduces new lifelong formative challenges. In this paper, we introduced a data model for capturing information that enables management and exchange of personal data on achieved learning outcomes. The Personal Achieved Learning Outcome (PALO) schema describes the relations between achieved learning outcomes, context where outcomes are achieved or applied, and evidence records of the obtained learning outcomes. Information on levels like proficiency level of learner mastering for the learning outcomes are also captured.
The PALO schema went through at least three iterative expert evaluations by the ICOPER consortium and at relevant international workshops by standards experts, teachers and learners to make sure that it captures data needed for increased employability of learners and higher interoperability with different learning systems.
Currently, prototypes of outcome based learning applications like widgets and LMS modules are being developed. These prototypes produce data about achieved learning outcomes of learners in systems like Moodle (2010), Elgg (2010) and Clix (2010) and store it in ICOPER's PALO repository. These data can be consumed by learning systems to provide learners with relevant material, recommendation of other teachers and learners based on similarity of learning outcome profiles, or to enable learners to share their achievement profiles with social or recruitment systems.
Future work will include the dissemination and validation of the prototypes in real-world environments. The PALO model is being prepared to be discussed in the CEN Workshop on Learning Technologies for standardization, as an EU specification for capturing data of personal achieved learning outcomes.
This work was supported by the ICOPER Best Practice Network, which is co-funded by the European Commission under the eContentplus programme, ECP 2007 EDU 417007. We thank Piotr Peszko for creating and providing the mock-up screens.
ATOM (2010) The Atom Syndication Format. Retrieved 3 February 2010 from http://www.ietf.org/rfc/rfc4287.txt
CEN (2010) CEN Workshop Learning Technologies. Retrieved 5 February 2010 from http://www.cen-isss-wslt.din.de.
Clix (2010) Clix IMC Learning Management System. Retrieved 3 February 2010 from http://www.im-c.de/en/products/learning-management-system/product-overview/what-is-clix
Crespo, M. R, Najjar J., Derntl M., Leony D., Neumann S., Oberhuemer P., Totschnig M., Simon B., Gutiérrez I., Delgado K. C. (2010) Aligning Assessment with Learning Outcomes in Outcome-based Education. In Proceedings of the Engineering Education Conference (EDUCON), 14-16 April, Madrid, Spain.
ELGG (n.d.) Social Networking Platform. Retrieved 3 February 2010 from http://elgg.org/
EQF (2000) The EQF for lifelong learning. Office for the publication of the EC. ISBN 978-92-79-0847-4.
EuroLM (2009) European Learner Mobility. Retrieved 5 February 2010 from http://wiki.teria.no/confluence/display/EuropeanLearnerMobility
Europass (2010) Europass - Initiative for Transparency of Qualification and Mobility of Citizens in Europe. Retrieved 3 February 2010 from http://europass.cedefop.europa.eu/
Facebook (2010) Facebook Social Network. Retrieved 3 February 2010 from http://www.facebook.com/
Grant, J. (2002) Learning needs assessment: assessing the need. BMJ 2002;324:156–159. doi:10.1136/bmj.324.7330.156
HR-XML (2009) HR-XML Competency data model. Retrieved 21 August 2010 from http://ns.hr-xml.org/schemas/org_hr-xml/3_0/.
ICOPER (2010) ICOPER Best Practice Network. Retrieved 3 February 2010 from http://www.icoper.org/.
IEEE (2002) 1484.12.1 — IEEE Standard for Learning Object Metadata. New York: IEEE.
IEEE RCD (2007) IEEE LTSC Reusable Competency Definitions (RCD). Retrieved 3 February 2010 from http://www.ieeeltsc.org/working-groups/wg20Comp/.
Leap2A (2008) The Leap2A specification for e-portfolio portability. Retrieved 21 August, 2010 from http://wiki.leapspecs.org/2A/specification
Lindgren R., Henfridsson, O., Schultze, U. (2004) Design Principles for Competence Management Systems: A Synthesis of an Action Research Study. MIS Quarterly, 28 (3), 435–472
LinkedIn (2010) LinkedIn Social Network. Retrieved 3 February 2010 from http://www.linkedin.com/
Moodle (2010) Moodle Learning Management System. Retrieved 3 February 2010 from http://moodle.org/.
Najjar, J. & Klobučar, T. (2009) ISURE: Report of study of learning needs building blocks and the standards to be used. ICOPER Project Deliverables.
Paquette, G. (2007) An Ontology and a Software Framework for Competency Modeling and Management. Educational Technology & Society, 10 (3), 1–21.
Richter, T., & Pawlowski, J.M (2007, October) The Need for Standardization of Context Metadata for e-Learning Environments. Proceedings of the e-ASEM Conference, Seoul, Korea.
Sampson, D. G. (2009) Competence-related Metadata for Educational Resources that Support Lifelong Competence Development Programmes. Educational Technology & Society, 12 (4), 149–159.
Totschnig, M., Klerkx, K., Klobučar, T., Law, E., Simon, B., Ternier, S. (2009) Open ICOPER Content Space: Implementation of 1st Generation of Open ICOPER Content Space including Integration Mini Case Studies. ICOPER Project Deliverables.
Van Assche, F. (2007). Linking Learning Resources to Curricula by using Competencies. In Proceedings of the First International Workshop on Learning object Discovery Exchange, Crete, Greece.
This paper appears in the International Journal of IT Standards and Standardization Research Vol. 8, No. 2, edited by Tore Hoel, Paul A. Hollins and Jan M. Pawlowski, editor-in-chief Kai Jakobs. Copyright 2010, IGI Global, www.igi-global.com. Posted by permission of the publisher.