Promoting the capture of sensor data provenance: a role-based approach to enable data quality assessment, sensor management and interoperability

Since the inception of the world-wide web, geo-scientists have been putting data online for sharing and discovery. Machine-to-machine harvesting of data is enabled when data are sufficiently described in community-adopted standards frameworks. But with the discovery of data comes questions. To understand data, better metadata are needed. Most of the effort towards the generation of metadata has focused on who, what, when and where observations are made. However, little effort has been directed toward providing sufficient content to determine precisely how an observation was made. Information that would be meaningful in assessing discovered data would include sensor characteristics, sensor configuration that can significantly affect observational accuracy and precision, and sensor maintenance – or lack of maintenance - that can help explain otherwise inexplicable shifts or long-term trends in data. But how do we associate this information to the data and enable it to be discoverable and accessible in our machine-to-machine systems. We need to get this information out of the manuals and field logs and PDFs and into community-adopted, standards-based frameworks, which are machine-actionable.

The Open-Geospatial Consortium (OGC) Sensor Web Enablement (SWE) [1] provides a community-adopted framework that supports the ability to fully describe processes used in creating observations and the sensors used. It can also support the tasking of sensors and defines machine-actionable access to web-accessible observations. The OGC SWE standard Sensor Model Language (SensorML) provides a means to describe sensors and processes in machine-harvestable encodings [2]. In SensorML, all components are defined as processes. Some processes, such as sensors and actuators, are physical while others are computational. Additionally, these processes can be indivisible components such as a single detector or algorithm, or aggregate such as a complex sensor system or a process chain [3]. SensorML provides a general framework to describe such properties as inputs, outputs, parameters, components, capabilities and characteristics. It can also be used to describe procedures and history using its event list. The terms in SensorML are not domain specific, so tools and profiles can provide enhanced support for specific communities or sensor types. Because SensorML treats all components as processes, the over-arching SensorML model provides an appropriate framework that can describe the provenance of an observation including sensor characteristics, the deployment environment, as well as its data processing and quality control and assessment tests performed.

In a NOAA-IOOS funded project called Q2O (QARTOD-to-OGC) [4], a SWE model was created to describe in SensorML all information needed in assessing data quality of an observation. The model describes the sensors, qc-tests, qc-flags and processing used to create derived products. A complete description of the content-rich model is described in [5]. From this activity, we recognized that the first step in the capture of the observational provenance is to encourage the manufacturer to describe the sensor in machine-harvestable SensorML. These descriptions must include capabilities (e.g., operational ranges), characteristics, input (observable properties), output (including units, accuracy and precision), as well as manufacturer contact information. The content can be used to enable automated QC, such as selection of appropriate precision and accuracy, as well as validating data using specified operational ranges. It is a small but significant step in automating the capture of observational provenance.

A recent NSF EarthCube Integrative Activity called X-DOMES (Cross-Domain Observational Metadata for Enviro-Sensing) was funded to develop a community of participants and to build the necessary tools to enable role-based creation of SensorML with links to resolvable terms. The tools are designed to be configurable to meet the needs of different users (manufacturers, field operators, data managers) who are not experts in SensorML and the Semantic Web [6]. The project is engaging sensor manufacturers (providers) and data facilities (consumers) across the geo-sciences in order to assess their usability to fully describe and harvest descriptions using the tools. An editor/viewer was developed which facilitates the creation and viewing of SensorML (Fig. 1); a SensorML Registry & Repository was created to enable SensorML to be registered and reference-able (IRI); and, an Ontology Registry & Repository (ORR) [7] was created to enable easy creation of resolvable definition of terms to be embedded in the SensorML documents using the Online SensorML editor, via a SPARQL query [6].

In “Definition of role-based content creation” section, we describe the roles of content creators and how the content can be associated. “Results and discussion” section describes the role and significance of building communities to promote the adoption of this approach in a consistent manner. The use of the various components of software by different members of the community are illustrated in Fig. 2.

A network of stakeholders has been established to further develop the model and to encourage the adoption of these best-practices. With the funding from the NSF-EarthCube X-DOMES project, the authors are updating and creating the suite of tools enabling manufacturers to create accurate, consistent description of sensors in SensorML, while providing registered and linked vocabularies developed by the community of users. The authors are also working with the Ocean Data Interoperability Platform (http://www.odip.org) and towards the development and adoption of standardized SWE Marine profiles through a recently organized working group [9].

Interested communities can participate by joining the Earthcube X-DOMES Network (http://earthcube.org/group/x-domes) or through the Earth Science Information Partnership (ESIP) Enviro-Sensing Cluster (http://wiki.esipfed.org/index.php/EnviroSensing_Cluster). The communities provide the development team with cross-domain, cross-agency input to guide in our development of tools and give us the ability to test the integration of products into existing and emerging data management systems through our collaboration with a few data facilities, such as the NSF-funded R2R (Rolling Deck to Repository), BCO-DMO (Bio-Chemical Oceanographic-Data Management Office) and Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI).

Access to the tools are available on the xdomes.org site (http://xdomes.org) and links to tutorial videos can be found on the ESIP X-DOMES website (http://esipfed.org/earthcube-xdomes).

These tools and the associated standards they promote were designed to be for international adoption and are applicable across domains. As the process of registering observational sensor models descriptions in machine-harvestable frameworks grows, manufacturers will feel compelled to participate since they would lose exposure to their potential market if their products are not in the registry.

The capture of knowledge is more difficult the further one gets from where the decisions are made. The more automated the capture of information and the earlier this information is captured, the more accurate and complete it will be. The social barriers of getting sensor manufacturers and field operators involved in the generation of this information can be overcome by providing access to simple tools that enable them to contribute without being aware of the technologies involved in enabling interoperable solutions for capturing metadata. By implementing community-adopted standard frameworks (OGC/W3C), we can easily broker [10] across other adopted standards (e.g., ISO/FGDC). As data are shared across a broader community and as data are less associated with those who created it, it is imperative to be able to understand and assess it for reuse by expectation of a broader set of descriptions of how the observation came to be.