Empowering citizen science through free and open source GIS

Abstract

This article presents an overview of the GIS components of the COBWEB software framework, designed in order to support a collaborative citizen science environment. It describes the overall architecture of the system, focusing in new developments of existing Free and Open Source GIS software.

Introduction: VGI & citizen science

Volunteered Geographic Information (VGI) is a term coined by GoodChild [1] in 2007, to describe a recent phenomena taking place in the GIS community; it refers to the widespread engagement of large numbers of individuals, often without any formal qualifications, in the creation of geographic information. Wikimapia [2], which adopts some procedures from Wikipedia to build a gazetteer and OpenStreetMap [3], an international effort to create a free source of map data, are well-known examples of VGI. Usually, data is classified depending on two factors: if the data is actively generated and if the data is georreferenced or not. More definitions about crowded data can be found at [4]. On this usecase, we will focus on georreferenced data voluntarily generated for scientific purposes.

Technological and societal developments, such as the ubiquity of internet and mobile devices, and the low cost of location devices, are driving forces behind the rise of VGI. There is one category of GIS, which had a significant development in the last two decades, and has been particular relevant for the context of public participation; this is Internet GIS, or Web GIS, sometimes also referred as Web Mapping, or Geo Web [5]. The architecture of Internet GIS is two tiered, comprising a server, which runs on a computer server and can handle multiple requests, and a range of networked clients. These clients can be as simple as a web page, as the most complex functionality is built into the server. One of the notable features of this architecture is that it has the potential for a very large user base with a very low cost per user, which is certainly appealing for the purpose of VGI. The third generation of web mapping, the Geo Web 2.0, relies on the Web 2.0, including technologies such as SOAP, AJAX and APIs [6].

Citizen science, on the other hand, can be defined as the set of scientific activities in which non-professional scientists voluntarily participate in data collection, analysis and dissemination of a scientific project [7]. Although not all, a great part of the information collected through classic citizen science, community science and citizen cyberscience projects has a geographic nature. The intersection between citizen science and VGI is called Geographical Citizen Science (see Fig. 1), a term created by Haklay in 2013 [8] to describe citizen participation in scientific projects involving collection and/or analysis of geospatial information.

Fig. 1

Geographical Citizen Science. Standing in the intersection of VGI and Citizen Science

A key problem on using crowdsourced data for scientific purposes is that you cannot rely on the quality of the data input. Sometimes due to unvoluntary mistakes made by the volunteers and sometimes due to malicious interventions. Some mechanisms are enabled to verify and validate the data. On COBWEB, part of this quality assurance is done by automatic filters on the data collected, based on known statistic distributions. Outliers are discarded, but even then, specific domain filters are needed for each usecase, even requiring manual intervention on not clear data. Asking different participants to collect the same data on similar time and location also help improving the quality assurance.

The ubiquity of geography in citizen science is explained by its focus in environmental issues, such as pollution, habitat and biodiversity [9, 10]. One of the early examples of citizen science, the Christmas Bird count [11], located observations within a grid with 100m, and sometimes 1Km of accuracy. Although geographic information can be (and still is) collected accurately using paper maps, on the rest of this section we will focus specifically on citizen cyberscience, which takes advantage of the GIS developments encapsulated in the Geo Web 2.0. According to [8], citizen cyberscience can be further divided into volunteered computing, volunteered thinking and participatory sensing.

In Geographical Citizen Science, volunteers can take an active or passive role [8]. An active role requires a consciously contribution to the observation or analysis; an example would be to take a picture of a bird, tagging it and submitting it to the project’s endpoint. On the other hand, a passive role does not require any active engagement in the data collection, and the user acts like not much more than a living observation platform; an example would be to have a daily route recorded, using an app and a location device.

Another classification [8] splits Geographical Citizen Science, according to the explicit nature of geographic information. While in geographically explicit projects, the activity targets directly the collection of geographic information, for instance the location of a species observation, in the case of geographically implicit projects the aim could be to collect an image of a bird species; although this image could be produced geotagged, acquiring location properties was not the original aim of the project.

It is easy to perceive how VGI can play a key role in science projects where resources are geographically spread, and thus require a large, but sparse, network of volunteers. Even if these volunteers are not professional scientists, they are able to contribute with free labor, skills, computer power, and in some cases, even funding.

COBWEB, which stands for “Citizen Observatory Web”, is an European research project focused upon the UNESCO World Network of Biosphere Reserves. Its generic goal is to facilitate effective citizen participation in environmental monitoring and governance, and for that purpose it has the specific goal of enabling citizens to collect environmental data using mobile devices [12]. According to the classifications presented on the previous paragraphs, COBWEB users play an active role in collecting explicit geographic information.

On Fig. 2, it is presented a diagram with a generic workflow of the citizen science use cases in COBWEB.

Fig. 2

What is COBWEB. Description of the data workflow and user involvement in the COBWEB platform

In this diagram, we may observe that the system data sources are either automated sensor devices (using the SOS protocol¹), or surveys uploaded by citizens. One of the key issues affecting surveys uploaded by users is the reliability of these data. Although in some cases we may have information about the participants, for most use cases all we know is their willingness to share information. In those cases, the spatial dimension of the data is particularly relevant, as location acts as a key to provide context information, even in cases where there this no metadata attached to the observations.

Fortunately, most modern mobile devices include a sensor (e.g.: GPS) that helps positioning the information. By combining the location stamp with the information provided by the surveyor, we are able to develop strategies to establish a degree of confidence in the reliability of the contributed data. We can compare this data with nearby data participants that have have been collected by other participants in a similar time frame. Should data from different sources confirm these observations, the level of confidence automatically increases.

Related to this issue, we may raise the question whether it is better to increase the density of collection around the same spot, or to have a more sparse data collection, which covers a larger area; this is basically a trade-off between reliability and coverage rate.

The objective of COBWEB is to provide a reusable framework that covers all the process of creating crowd sourced data: from the app that the participants use to collect the data to the final conflation analysis that makes sure the data is reliable and has a minimum quality level.

Implementation

Architecture

In order to achieve the goals described on “Introduction: VGI & citizen science” Section, a generic platform which assembles different technologies with different readiness levels [12] was developed. These components, depicted on Fig. 3, include a web and a mobile frontend to interact with the users, a quality assurance [13] and conflation backend [14] which interfaces with the sensors, and a few utilities (e.g.: PCACPI, Authoring tool). At the core of the system, there is a backend based on Free and Open Source geospatial technologies. This backend receives the geospatial data from the different input sources and serves these data using open standards, making it discoverable and available to the general public. On this paper the authors will focus mostly on these core technologies, and describe how the use of existing mature Free and Open Source GIS powered the geospatial capabilities of the COBWEB framework. More specifically, there will be a detailed explanation of the developments in one particular technology that was improved in the context of this project.

Fig. 3

COBWEB architecture. Overview of the technological components of the platform, and their interactions with users

Use of GIS technologies in the COBWEB framework

Part of the COBWEB framework is hosted on a simplified GeoCat Live environment [15], a quick-path solution for publishing spatial data and metadata on the cloud (see Fig. 4). Live takes advantage of common cloud techniques such as server virtualization, load balancing and service monitoring, in order to provide managed servers with optimal configurations, to support continuous availability of large crowd sourced datasets. Apart from GeoNetwork, whose developments we will describe in more detail in the next sub section (see GeoNetwork developments), the software stack includes other GIS technologies.

Fig. 4

Live infrastructure. Overview of the major components

Geoserver [16], a Java based server-side software, was used to serve spatial data using OGC standards (WMS, WMTS and WFS). Its focus on interoperability, supports the creation of maps using a wide range of clients. The PostgreSQl database management system [17], extended with PostGIS [18] in order to support spatial data types and operations, was used as storage engine. PostgreSQL is a mature project, with a focus on extensibility and standards-compliance. Within the scope of spatial database engines, PostGIS offers one of the most extensive feature list, in addition of being supported out-of-the-box by a number of third-party products (e.g.: QGIS, Geoserver, Mapserver) [18]. Both projects, Geoserver and PostGIS, are under the umbrella of the The Open Source Geospatial Foundation (OSGeo) [19].

In the context of COBWEB, it was developed a storage middleware which abstracts access to Cloud Storage providers; this was called Personal Cloud API, or PCACPI. Along with this middleware, it was developed an extensible viewer, which adopts the Leaflet libraries [20], in order to provide interactive mapping capabilities. Leaflet is a good example of third-generation web mapping, taking full advantage of the capacities of the Geo Web 2.0, for performance and usability; it is well-known by its simplicity, and compatibility with mobile devices.

In the case of the GIS components described in the previous paragraphs, the key functionality provided by the original software fitted the needs of the COBWEB framework, and therefore there was no need for additional changes in the source code; these products were integrated in GeoCat Live in their original, unmodified, versions. That was not the case of GeoNetwork, as it is the basis of the COBWEB portal, and this portal had specific needs which were not covered by the generic GeoNetwork OpenSource. In the next sub section we describe the developments that were made, in order to accommodate those needs.

GeoNetwork developments

GeoNetwork openSource is a standardized and decentralized catalog to manage geospatial resources [21]. It is based on the concept of distributed data and information ownership, and in practice it works as a gateway to access geospatial data and products through its metadata. The compliance with international standards for metadata (e.g.: INSPIRE [22]², ISO³) [23] is one of the strengths of GeoNetwork, and for that reason it has been adopted by many public authorities and international and European organizations, to support the development of their Spatial Data Infrastructures (SDI) [21].

The COBWEB framework follows the INSPIRE directive by using metadata schemas compliant with it. The metadata editor, which allows the data to be classified on the catalog, provides many wizards and helpers to help the editors to create valid metadata. This metadata is also validated against the ISO and INSPIRE schema rules to make sure any INSPIRE catalog reader can process and understand the data included on our portal. This way, all data generated on the COBWEB platform is easily reusable by third parties.

The web portal, which allows users to view, manage and edit the metadata over the internet, is one of the components of GeoNetwork. On version 3x this UI was completely redesigned and it was refactored to use Angular.js and the OpenLayers3 libraries, which provide a richer web experience. This portal was adopted as the main entry point to COBWEB instances; the idea was that citizens could use the website to register and join events, download the app and visualize contributed data. Although GeoNetwork is a generic product and therefore can be used in many different contexts, it lacks specific functionality to support this particular use case. The approach took by COBWEB was to extend GeoNetwork, in order to provide this functionality, and later include some of these improvements in the official branch (this outcome is discussed on “Why FOSS” Section). We decided to based all the developments on GeoNetwork 3x, and in that way contribute to test and improve this version.

Along with the user interface, GeoNetwork offers another OGC protocol to search the catalog: CSW, Catalog Service for the Web. On the COBWEB usecase, this protocol allows third party developers to search for data and even create a whole new user interface (mobile or desktop) for specific surveys.

The diagram on Fig. 5 shows the code repositories, for the different blocks which relate to GeoNetwork. We discuss these developments in the next paragraphs.

Fig. 5

Code Repositories. Repository structure of the developments in GeoNetwork (green) and related products (red)

The GeoNetwork portal website is a Spring MVC application. The changes for accommodating the COBWEB portal included a complete re-design, template changes, adding of new templates, controller changes, and the introduction of an AngularJS and Bootstrap modules. As at the time, GeoNetwork did not have a metadata detail page, and as this was one of the requirements for presenting the surveys, this functionality was implemented. An interesting feature is that the detail page was implemented to use the fields available through the “q” service, the Lucene-based search engine. As a consequence no additional request have to be made in order to get the metadata details, and the q-schema is always similar for each type of metadata. The detail page also provides some additional information, such as the relation to other metadata, and a link is provided to request the full metadata.

Another feature of the detail page is an additional “show map” button, which enables adding metadata with a service link to the GeoNetwork map viewer. As the service url is only defined in the context of the ISO19119⁴ profile from Geoserver harvest, we added support for mapping all the records that do not have a service link, but have an ISO19119 service metadata attached, by retrieving the url and adding it to the “show map” button.

One interesting exploratory feature for citizen scientists, is to be able to retrieve additional information about records on the map. In order to enable that, we gave support to the getFeatureInfo WMS and WFS request in the map viewer, so that clicking features on the map triggers a bubble with the results about those feature. This bubble is styled according to the common data format of COBWEB, allowing the portal to offer a more customized user experience.

Although most of the improvements resulted in adding features to GeoNetwork, in some cases we removed elements which are unnecessary in this context, in order to simplify the portal. This was the case of the metadata editor, which can be adjusted through the configuration options. The final result was a simpler editor, with new templates to support COBWEB types of metadata (e.g.: survey, field session, map) and a convenience button to publish metadata directly from the editor form.

The GeoNetwork community module was developed in order to give support to groups (e.g.: surveys), and group roles in GeoNetwork. Users can subscribe to groups, and within a group they can manage a set of private and public resources (download app, view datasets). The group administrator can invite and manage users in the group. Most of this customizations have already been added to GeoNetwork master branch as a way to improve it. Other customizations have been left only on the specific COBWEB repository, like a button to “join” a group on each survey page.

COBWEB uses the Security Assertion Markup Language (SAML) for authentication across services through a single sign-on (SSO). This is an XML-based, open-standard, data format for exchanging authentication and authorization data between an identity provider and a service provider, in this case between users and the COBWEB platform. Although GeoNetwork had some support for SAML (contributed by GeoSolutions), additional work was required in order to extend this basic authentication support and allow complex attributes on the SAML authentication.

GeoViqua is another research project, developed under the umbrella European Community’s Seventh Framework Program [24]. Its main goal was to augment the GEOSS Common Infrastructure (GCI) with innovative quality-aware visualization tools and geo-search capabilities. One of the project’s outputs was a ISO19139 profile, which incorporates ISO19157⁵ for encoding the quality of the information; being able to qualify uncertainty in observations is an important feature for citizen science, thus this profile was adopted by the COBWEB project [25]. As it was not yet supported on version 3x of GeoNetwork, it was developed a schema plugin in order to incorporate this functionality.

As mentioned before, apart from adding new features, the COBWEB developments contributed to improve GeoNetwork by polishing features and raising awareness of minor bugs, which were subsequently fixed.

Availability and requirements

On Table 1, we can see a summary of different aspects, required to run and develop the COBWEB fork of GeoNetwork.

Table 1 Availability and requirements of the COBWEB fork of GeoNetwork

Results & discussion

COBWEB in action

The COBWEB portal allows coordinators to create and invite users to a citizen science event for collecting data (e.g.: survey) within the scope of a project. In Fig. 6, we can see a screenshot of the landing page, for a desktop and a mobile device.

Fig. 6

Landing page. Desktop and mobile versions of the COBWEB portal

After creating an account, the participants can login to the portal, browse the citizen science projects, and request to join them, either as anonymous, private or registered users (see Fig. 7). They are then able to participate in an event, by downloading an app, and data (as GeoPackage⁶) for offline usage (see Fig. 8); this enables them to collect observations on the field, even in the advent of no internet connection. The information collection cycle ends, when participants upload their observations to the COBWEB framework, and the uploaded data is available on the portal using OGC standards (e.g.: WMS, WFS, SOS). On Figs. 9 and 10, we can see a screenshot of the results browser and a detail page which contains a link to download the survey data.

Fig. 7

Project. Screenshot of the project information page where the user can request to join the project

Fig. 8

Event. Screenshot of the page of an event, where the user can download the event related files

Fig. 9

Data browser. Screenshot of the page, where the results of the surveys are presented

Fig. 10

Survey detail. Screenshot of a survey detail page, where the generated data is available for download

Through the user panel, users can update their account details and (re)view their contributions. The coordinator can manage users, events and datasets.

Why FOSS

Free and Open Source Software (FOSS), also called Free Software is a concept created by Stallman in 1984 [26], as part of a manifesto designed to guarantee software users, four basic freedoms. These are the freedom to run, modify and redistribute copies of a program, in its original or modified versions. These freedoms were designed to promote software evolution, through bug fixing and capabilities improvement, and they are enforced through a method called copyleft, which is implemented through a family of software licenses (the most popular one being the GPL licence⁷).

Free software is highly represented in the world of GIS, and currently there are a number of extremely sophisticated software projects that implement every component of the GIS software stack, including spatial storage, Web GIS, desktop GIS and even mobile GIS. Unlike some years ago, nowadays these projects are very complete and they have nothing to envy from proprietary systems, in terms of usability, as well as functionality [6]. Even well established proprietary software producers such as ESRI, which still attain a considerable share of the GIS market, are now releasing a limited number of products under a Free license⁸.

FOSS shares some common aspects with citizen cyberscience, in the sense that they both are bottom-up initiatives, involving a community of tech-savy users, which are not necessarily professionals of that field; thus, it should not come as a surprise that the citizen science community has been deeply involved in FOSS, and vice-versa [6].

Free cost is a strong argument for the adoption of FOSS (although by definition, that is not necessarily always the case⁹), but there are many other aspects which are sometimes overlooked. We already mentioned bug fixing and feature adding, as a direct consequence of the FOS model. Although it is true that these improvements require some developing knowledge, the rest of the users may support the developers by testing, and issuing bugs; the results of this combined effort, should benefit the overall community. This kind of approach tends to result in a dynamic pace of development, and sometimes FOSS evolves quicker than proprietary software [27].

In GIS the access to the source code is particularly relevant, because some algorithms are complex and it may be necessary to review them in order to fully understand them, something which is simply not possible with proprietary software [27]. In addition, proprietary software also falls short in the distribution of newly implemented models, since the original software is required in order to run those models [28].

As described on “Use of GIS technologies in the COBWEB framework” Section most COBWEB GIS core technologies are Free and Open Source; in one hand this enabled us to benefit from very sophisticated components, in this case at no cost, (e.g.: data storage engine, server side software); and on the other hand, it eased the distribution of software among project participants (e.g.: apps). On Table 2 we may find a listing of the license of the different GIS components within COBWEB; it should be noted that in most cases, they adopt the original GPL license.

Table 2 Software licenses of the FOS GIS components of the COBWEB framework

In addition to the FOSS GIS components, we used a proprietary software called GeoCat Bridge “Use of GIS technologies in the COBWEB framework”. This is an ArcGIS extension, which allows users to publish any ArcGIS project in a range of OGC-based services, using a stack of Free and Open Source components. Although Bridge itself is not free, as it is restricted by the ArcGIS proprietary license, this extension plays an important role in unlocking users from a proprietary software system, by allowing them to easily publish their information in FOSS, using open standards. As the producers of Bridge were part of the COBWEB consortium, it was possible to update the software in order to communicate with the COBWEB middleware services and COBWEB authenticating protocol, and it was also possible to use it free of charge, within the context of the project. Were they not part of the consortium, due to its license it is not clear that Bridge could be used in the project, at least on this basis.

As described through “Implementation” Section, all components were integrated in COBWEB, in their unmodified form, apart from GeoNetwork. On “GeoNetwork developments” Section we detail which were these developments which were related to the need of having a portal where surveys could be created, and users could contribute to these surveys and share the results with the community. If GeoNetwork would not have a GPL license (or other FOS license, for that matter) it would not be guaranteed that we would have access to the source code, in order to customize it to the needs of this specific use case. It would also not be guaranteed that we could share the resulting framework, as a deliverable for the citizen science community, in general. In that sense, we may say that the license of GeoNetwork has a played a key enabling role in this project, both in terms of development, as well as in terms of results delivery.

However, GeoNetwork’s COBWEB developments do not only contribute just to the COBWEB project and the citizen science community, and that is another benefit of the FOS model. On Fig. 11, we can see a diagram that depicts COBWEB developments, in relation to the GeoNetwork OpenSource project. In this diagram we can note that there are different levels of integration with the trunk, according to the interest that these changes may have to the generic GeoNetwork community, which includes people who may not necessarily be interested in citizen science. For instance the “GeoNetwork community module” was included as a plugin, as it was not clear, at least at the time, that it should be part of a generic catalog product.

Fig. 11

GeoNetwork developments. Contributions of COBWEB to GeoNetwork OpenSource

With this approach, users interested in community functions (e.g.: create groups, invite users for these groups) can take advantage of this feature, without the drawback of bulking the core product with code that is only used in a restricted number of use cases. GeoNetwork already uses a plugin mechanism for giving support to schemas, so the profiles used by COBWEB were added as plugins, which may be useful to other use cases beyond the scope of this project, or even beyond the scope of citizen science.

The adoption of GeoNetwork 3x, which was still at its early stages¹⁰, promoted a lot of needed testing. This testing raised awareness over minor bugs, which were then fixed and these fixes were contributed back to the trunk.

These improvements, together with some generic features that were added within the context of the project and merged back into trunk, came to benefit the overall GIS community.

Finally, the look and feel of the portal itself, which is based on a fork of GeoNetwork, was never contributed back to trunk. This is because this portal is too specific of this use case, and it does not make sense to have it in the generic catalog. However, this project raised awareness to the fact that there are many GeoNetwork users who want custom versions of the portal. This helped on the development of an easier framework for developing custom user interfaces for the GeoNetwork catalog.

Current status and future developments

As we mentioned before, COBWEB includes a large number of components, with different licenses and different readiness levels. When a software component was available with a usable license, we preferred to incorporate it in the framework, rather than starting something from scratch. If a particular functionality was lacking on that software, we have chosen to engage on the development of this functionality, and in those cases where it was relevant, to merge the new functionality back into the main project. This approach was particularly feasible, since core contributors/maintainers of relevant FOS projects were involved in COBWEB.

In the case of GeoNetwork, which we explained with detail in “GeoNetwork developments” Section, some outcomes of COBWEB are already available to GeoNetwork users, as plugins, or as improvements in the trunk. Other improvements, such as community functions, or support to the SOS protocol in the map viewer, are still to be incorporated in GeoNetwork OpenSource.

In terms of promoting the use and evolution of the resulting software, thus ensuring an optimal uptake of the results, there are two main strategies that could be followed; one focuses on the promotion of the COBWEB framework, as a whole, among the citizen science community, while the other focuses on the promotion of individual components. Historically, developer oriented communities are more sustainable than use case communities, because they can foster reusable technologies, thus having the potential to become larger. Moreover, creating new communities is an important effort, which would require a substantial contribution from COBWEB partners, that should endure after the end of the project.

By deliberately selecting, mature, software components with a FOS license, we aimed to take advantage of those existing communities, in order to persist the life of the COBWEB components beyond the duration of the project.

Conclusions

User engagement and user participation play an essential role in citizen science. Due to its high level of interactivity which produces a high engagement with the end user, Web GIS has been at the core of many citizen science initiatives.

In this article we have presented a framework designed to support a set of citizen science use cases, related to environmental monitoring and governance [29]. This framework successfully took advantage of existing Free and Open Source GIS components, in order to support different aspects of the solution (e.g.: database, catalog server, map server, web map). Beyond demonstrating the importance of FOS licenses in enabling us to use these components, we have illustrated how this type of license allowed the customization of the software in order to support particular needs of the project, and how the GIS community could also benefit from some of those improvements. Finally, and most importantly, these licenses are responsible for ensuring the continuous use and update of the software, beyond the life of the project.

In this way, we hope to have demonstrated how Free and Open Source GIS can have a key role in supporting and further enhancing, citizen cyberscience.

Endnotes

¹The Sensor Observation Service (SOS) is an OGC standard, which allows querying sensor data and metadata, as well as representations of observed features, through a web interface. See: http://www.opengeospatial.org/standards/sos.

²The INSPIRE directive was established in 2007 by the European Community, to support the development of compatible Spatial Data Infrastructures by member states. See: http://inspire.ec.europa.eu/index.cfm.

³International Organization for Standardization: http://www.iso.org/iso/home.html.

⁴ISO 19119:2016 - Geographic InformationServices: http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=59221.

⁵ISO/TS 19139:2007 - Geographic information – Metadata – XML schema implementation: http://www.iso.org/iso/catalogue_detail.htm?csnumber=32557.

⁶GeoPackage is an OGC standard for defining a platform-independent data format, for GIS. See: http://www.geopackage.org/spec/.

⁷GPL was written by Richard Stallman in 1984; the complete license agreement can be found here [30].

⁸The refereed products are the ESRI “GIS Tools for Hadoop” [31], released under an Apache license.

⁹It is important to note the difference between “free” software and “free of cost”, as FOSS can be distributed for a fee, and software that is gratis may not guarantee the four freedoms of the FOSS model.

¹⁰GeoNetwork OpenSource v3.0.0 was released in late April, 2015.