Peer Production in Citizen Science A Guide

Knowledge Management

These guidelines are based on my experiences implementing a peer production approach with the personal science community. They are related to the discussion in Chapter 5.1 and can be found in Appendix A.11.1 of my thesis.

Process Recommendations for Designing Knowledge Management Systems for and with Communities of Practice in Citizen Science

  • Understanding objectives. What is the purpose of the knowledge management system? Why are existing solutions not sufficient? Try to understand the general problem that the approach is thought to solve. Then try to understand this problem in more detail.
  • Understanding and involving users. To this end, try to engage with a variety of community stakeholders, from newcomers to experienced members. Individuals with community management experience might have valuable insights into failed attempts, as well as common issues and motivations, and can inform the prioritization of issues. It might also be insightful to talk with people who have stopped engaging in the CoP, or people who have not (yet) started to understand barriers. Try to understand the target audience and think of both the consumers and contributors. Who enters and consumes information, what are their incentives, and why would they use your system instead of existing ones?Creating a solution that caters to every user group might not be achievable. To navigate this, consider giving precedence to specific user groups and customizing the solution to match their needs. Including potential future users at every phase of the design process can go beyond seeking feedback, and might also include design or leadership roles. Either way, try to maintain transparency about how individuals can get involved and try to document the design process as openly as possible to make it easier for new people to engage.
  • Prototyping and iterating. It can be easier to start discussions and gain feedback when interacting with prototypes, than by contemplating abstract ideas. Statements of community members such as interview extracts or forum contributions can serve as discussion starters in the beginning, non-functional and functional prototypes at later stages. Be open to early and frequent iterations, presenting unfinished ideas, and being willing to discard or revise approaches as needed. Introducing a functional prototype early in the process can provide valuable insights into real-world usage under realistic conditions. However, a drawback of this approach is that implementing changes in an already operational system can be more challenging. In that case, it is important to ensure that measures are in place to prevent the loss of user-generated content.
  • Structuring information. A suitable information architecture is important so people can discover relevant information. Keep in mind that your imagination of a reasonable organization of the information might not represent mental models held by other community members. The same counts for information architectures that are already in use in other systems. If you start building a knowledge management system from existing resources, be open to adapt information architectures or information representation according to user feedback or observations. You can for example start by using a category structure that is already in use if you have a functional prototype, and see during usage how well it fits with the generated content, then adapt it as necessary. Try to provide tools that allow users to influence or adapt information architectures, e.g. by letting them use custom tags. If there is already a certain quantity of content, there are user study methods like card sorting or tree testing to develop or test information architectures with users. start somewhere, e.g. something already in use and test it. There might be various ways to structure knowledge efficiently, and some ways might be better for one user group or another. If possible, offer several complementary pathways to provide alternative pathways for discovery.
  • Tacit knowledge. Learning and sharing experiences about a particular practice is frequently at the center of a community of practice. A significant portion of this information resides in tacit knowledge, encompassing skills, know-how, and contextual nuances. Presenting this information in a beneficial manner might not be immediately apparent. A format effective in one context might not translate well into another context. Allowing users to contribute diverse types of artifacts and soliciting feedback from current users on running a user study can aid in assessing usability and identifying potential issues. Making artifacts editable can provide users with the opportunity to distill important information. In any context but particularly in contexts related to research and citizen science it is also vital to consider quality standards, along with mechanisms in place for their oversight, and how users can engage in this process. Measuring success. Developing ways to measure success can be a complex task, especially for early stages of the project, in which the knowledge management system does not have a large user base. Meaningful measures may vary for each project. For example, try to understand which functionalities it provides that did not exist before, and in which use cases, contexts and by which user types the system is used.
  • Embracing change. Knowledge relevant in communities of practice, especially if it is an emerging practice, will change frequently. Therefore, view the knowledge management system as a dynamic entity, with its structures evolving in response to usage patterns over time. Try to see the knowledge management task as partly independent from its technical implementation. Insights about mental models, architectures or formats might be transferable to other systems, and the content within the knowledge base can become a valuable object of research by itself.