Definition of role-based content creation
Sensor manufacturer role
Typically, the sensor manufacturer has the most comprehensive knowledge of how a sensor works and what is important in assessing data quality. By assuming the role of describing the sensor in SensorML, accurate and complete creation of content is more easily implemented and knowledge is captured where it is best understood. By creating this content in standards-based encodings and registering the content for access from an authoritative, freely and openly available service, anyone who purchases a sensor described by the original equipment manufacturer (OEM) can reference a vetted, community-adopted description of a particular sensor model. This can lead to more accurate and more fully-described information about the technology used in creating our observations.
Description of the sensor model
A sensor model can be described broadly and will be referenced as a unique type of sensor. Some sensor models have options that will be defined as such. The SensorML file describing the Original Equipment Manufacturer (OEM) model is “owned by” the manufacturer, who can register it to provide a persistent, version-controlled, resolvable link to its content. The manufacturer has the responsibility to create, maintain, and register the OEM SensorML document that will have specific content, including at a minimum:
Additional information will include characteristics of the sensor (e.g. size, electrical requirements), capabilities (e.g. sensitivity, accuracy), identifiers (e.g. model number, sensor name), classifiers (e.g. intended application, sensor type), and other properties. By registering the sensor in a registry such as the X-DOMES SensorML Registry and Repository (SRR), the content is accessible and can be referenced via a URL (Uniform Resource Locator). The SRR also enables control of the content through authoritative ownership, version control and provides a mechanism for discovery.
Sensor manufacturer of a particular sensor
The manufacturer will also create many instances of a sensor model that will be purchased and deployed. The manufacturer can create an Instance SensorML file that describes each particular sensor, as built and factory-configured. The content of this SensorML description includes at a minimum:
-
A URN to uniquely define this instance of the sensor
-
a declaration that this is a typeOf the OEM model (Fig. 3), thus enabling inheritance of the SensorML descriptions in the OEM model documents, and
-
Contact information of the sensor owner
Additional information will include, for instance, a serial number, calibration curve or date of last calibration, configuration settings, and more.
A URN for a model could be “urn:mfgr:model” and the instance “urn:mfgr:model:serNum”, providing them each with a discoverable unique id.
The SensorML description of the instance is transferred to the owner of the instrument who becomes the curator of the document. It is the responsibility of the sensor owner to deploy a mechanism to register and reference the document (via a registry or a web service) and to associate it with the data that it produces.
Sensor user roles
Configuration of the sensor
Each sensor typically has options available to the user to specify in order to meet specific deployment requirements. These options are also important to associate with the data, as they often affect operational range, accuracy and other parameters that affect how data can be interpreted. Currently, the X-DOMES project is exploring mechanisms to enable field operators to be able to add these descriptions to their sensor descriptions using the SensorML Editor/Viewer. Unless parameters are specified in the Instance SensorML, the operational descriptors will be inherited from the OEM descriptions.
Deployment of the sensor
Once an instrument is purchased and configured, it is ready for deployment. There are standard descriptions that must be included such as location, orientation, station IDs, feature of interest, data ownership, etc. But, there are also many things that happen that can significantly impact data quality that have not been typically included in metadata. In particular, sensor preparation with regard to assuring quality measurements should be noted. For example, calibration of a sensor may happen in situ or at some point in time in a data stream. A sensor face may periodically be cleaned or fouling may be noted in the sensor deployment description. All this information must be fully described in machine-harvestable frameworks. The SensorML Editor/Viewer provides an easy way for the operators to insert history events into the combined ConDep (Configuration/Deployment) information SensorML document.
Processing for quality control and derived products
Quality control tests are typically applied after data are collected and derived products are often computed from collected data. In the Q2O project [4], examples of SensorML process descriptions demonstrate how to describe quality control tests and processing along with associated URLs for further describing computational methods. This enables a data provider to provide options for data services (SOS) and provides a data consumer the option of requesting, for example, only ingesting data that passed particular QC tests or the unfiltered raw observations. Access to complete data processing descriptions enables a consumer to better understand how to interpret data that they receive. For example, if one is looking for extreme events, one might look at the processing descriptions to learn what threshold values were used to determine out-of-range data and whether these data were removed or replaced. If threshold value used is below the threshold value of your definition of ‘extreme event’, you cannot assume there were no occurrences of these events, but you can only note that they would not show up in this particular data set. The ability to automate descriptions of the processing chain while implementing it is important to assure accurate and complete descriptions. The authors encourage exploration of methods to automate the capture of process descriptions, as processing steps are being defined by those who best understand the methods being implemented. The processing descriptions must also include authoritative references to established methods of computation. And any seasonal or situational changes in parameters used in each processing step should be time-stamped and noted as an event in the process descriptions. Fig. 4 shows a SensorML description of a common quality control test. Like the OEM and Instance SensorML documents, this will become a component describing the system that created an observation.
The SensorML online editor enables the user to specify the type of SensorML component such as a physical component (OEM document) or process model selection (procedure), that is being created. The software uses RelaxNG [8] profiles to guide SensorML development, leading to more consistent content (Fig. 1 lower panel). Profiles can be created to standardize metadata for a particular community or manufacturer, the description of a particular sensor type, or the inputs, outputs, and parameters of a particular QA/QC test component. These profiles both simplify and standardize the creation of a SensorML documents to describe the various components. The X-DOMES community strives to encourage the development of profiles that enable non-experts, such as sensor manufacturers and data managers, the ability to create content from templates. Thus, through the development of example profiles for OEM, Instance, processing, deployment, platforms etc., a community-adopted set of templates can be registered, managed and assessed for use by the cross-domain enviro-sensing community.
Bringing it all together within an sensor observation service
Thus, the full description of the provenance of a particular observation can include the OEM description of the sensor model, the description of the particular configured sensor Instance, the list of QA/QC tests that have been applied along with their configuration, and the explicit chain of processes applied to obtain the collected observations. When the SensorML files are complete, the provenance for a given observational offering can be accessed using the DescribeSensor request defined within the OGC Sensor Observations Service (SOS) standard.
The Q2O project demonstrated how to pull these documents together though one should note that the descriptions in Q2O were SensorML 1.0 while SensorML v2.0 provided significant improvements to support inheritance of associated SensorML through use of the ‘typeOf’ association. Within XDOMES, SensorML version 2.0 was used, which led to a better separation of the OEM sensor model description from the description of the sensor Instance.
The content is created as stand-alone documents that need to be registered, so that they are web-accessible and persistent. The X-DOMES SensorML Registry and Repository (SRR) can be accessed through the xdomes.org site. The SRR enables content to be maintained and provides a Uniform Resource Locator (URL) for access. The documents can be referenced and harvested as part of a DescribeSensor response and included as a component of a SensorML process-chain. The links can also be referenced in the O&M encodings. Or, the documents can be included in non-SWE data management systems, since the content is registered and accessible via its URL. For example, if a data provider associated a set of data with a particular sensor and sensor model, a data facility could harvest the referenced SensorML document to incorporate information needed within their specific data management system automatically, since it is based upon a standards-based, community-adopted standard (SensorML).
The X-DOMES project has focused on tools to create the OEM/Instance content. The tools currently can also be used to create process descriptions. Tools are also needed to more easily connect the components into a process-chain that can fully describe how they connect to show process lineage from observable property to observation, thereby enabling quality assessment.