CityJSON: a compact and easy-to-use encoding of the CityGML data model

Main criticism of CityGML as an encoding

CityGML files are known to be very difficult to parse and to extract information from. We briefly describe the main issues, at three levels: XML, GML, and CityGML.

XML. As mentioned in the Introduction, when JSON suffices, it is usually preferred by developers over XML. This preference is mainly because JSON is far simpler than XML, which reflects the fact that JSON is a data format while XML is a markup language, and it is thus much easier to write software for JSON than XML. Some common arguments in favour of JSON include its smaller file size, greater ease of reading (by humans), greater ease of parsing (by software), and the general difficulty of dealing with malformed XML (which is common). JSON is also based on simple data types and data structures that are available in almost all programming languages, and thus mapping the content of a file to a data structure that can be easily queried is trivial. In fact, many programming languages treat JSON as a near-native type and can read and write to it (ie serialisation and deserialisation) without the need of external libraries. By contrast, XML usually needs to be parsed in a process that requires the use of libraries and which creates a hierarchy of more complex objects, and this structure still needs to be traversed according to the logic of CityGML.

GML. While GML allows us to represent geometries, the fact that there are many different ways to store the same geometry is a big handicap in practice. A vivid example is shown by [23]: a simple square can be stored in at least 25 different variations in GML. The large number of possible variations means that a developer needs to figure out all possible variations for every geometry type and write code to handle them appropriately. The variations also increase with the dimensionality, so the storage of a solid would have even more variations (because polygons are used as building blocks). Apart from the large efforts required to find and handle all possible configurations, this is a situation that causes differences in how different software packages handle different datasets.

CityGML. CityGML builds upon XML and GML, and so it inherits most of the advantages and disadvantages of these formats, but some specific features of CityGML are also problematic. First, one thing to notice is that a city can be large and as a consequence CityGML files tend to be massive (1GB+ are common). As this means that they often do not fit in the memory of a computer, CityGML software sometimes needs to do more complex processing than it would be required otherwise (e.g. dynamically reading from and writing to a database). Second, as it can be seen in Fig. 1, the hierarchy for a single simple building can become rather deep, which translates into many classes which are nested hierarchically in XML. This makes files very difficult to read by a person (which nullifies a big advantage of XML), and the many different classes might need specialised code to be written for each. Third, CityGML makes extensive use of XLinks (XML Linking Language). While these are in theory powerful, in practice the links need to be resolved, which is problematic, especially for large files, or when references are external URIs (i.e. pointing to objects not in the file). Several XML libraries and software do not resolve XLinks. Furthermore, many of the key features of CityGML are based on XLinks, e.g. they are necessary for semantic surfaces.

The consequences of the above is that software support for CityGML is lacking. As a telltale example, there are still no full JavaScript parsers for CityGML (which are necessary in order to exchange and process files on the web), and thus the efficient exchange and processing of CityGML models on the web is very difficult, if not impossible. We emphasise the word “full” here, given that there are existing JavaScript parsers but these are rather limited, as they are usually hard-coded for specific files or files written by a specific program. When used with other files, they might therefore ignore some parts of a file or simply crash, because a particular CityGML representation has not been accounted for.

City objects are “flattened out”

The property ~CityObjects~ contains a dictionary where the properties are the identifiers of the city objects (IDs). The schema of CityGML has been flattened out and all hierarchies removed. Figure 2 shows the city objects that are supported in CityJSON, both 1st- and 2nd-level city objects are stored in the dictionary ~CityObjects~.

As an example, for a building containing 2 parts, the 3 objects will be represented at the same level and linked by their IDs.

Each city object can have a "parents" and/or a "children" property, and this is how in the snippet the building ~id-1~ is linked to its 2 parts. The fact that a dictionary is used means that developers have direct access to the city objects through their IDs (and also in constant time if a hashmap is used to implement the dictionary).

A city object can be of any of the types defined in Fig. 2, and each of them must have the same structure, and at a minimum contain a "geometry" property. If attributes are to be stored, they have to be in the "attributes" property. This simplifies the work of the developer because there is a single point of entry for all geometries and attributes, unlike with CityGML.

Geometry

CityJSON defines the same 3D geometric primitives used in CityGML, with the same restrictions for linearity/planarity. However, since they are rarely used in a 3D context, Point and LineString only have their Multi* counterparts; a single Point is a MultiPoint with only one object. When a geometry is defined, it must contain a value for the LoD. In order to avoid ambiguities, we encourage the use of the refined LoDs, as defined in [4], over the five standard CityGML ones. City Object can have several LoDs, and thus CityJSON, as is the case for CityGML, allows us to store concurrently several LoDs for the same object.

It should be noticed that CityJSON uses a different approach from (City)GML to store the (x,y,z) coordinates of geometric primitives. A geometric primitive does not list all the coordinates of its vertices, rather the coordinates of the vertices are stored in a separate array (the "vertices" property of the CityJSON object), and geometric primitives refer to the position of a vertex in that array.

The indexing mechanism of the format Wavefront OBJ⁴ is reused, because it has been used for many years, with success, in the computer graphics community. There are several advantages to this approach. First, the files can be compressed: 3D vertices are often shared by several surfaces, and repeating them can be costly (especially if they are very precise, often sub-millimetre is used). Second, this increases the topological relationships that are explicitly stored in the file, and several operations can be sped up and made more robust (e.g. are two buildings adjacent?). Third, it is very easy to convert to a representation listing all coordinates; the inverse is not true.

The geometry is based on an enumeration of the vertices forming each ring of a surface, as follows. A ~MultiSurface~ has an array containing surfaces, where each surface is modelled by an array of arrays, the first array being the exterior boundary of the surface, and the others the interior boundaries. A ~Solid~ has an array of shells, the first array being the exterior shell of the solid, and the others being the interior shells; each shell has an array of surfaces, modelled in the exact same way as a ~MultiSurface~. Notice that unlike with (City)GML, there is only one variation per geometry type, which (greatly) simplifies the life of developers.

Semantic surfaces

In one given city object (say a ~Building~), several surfaces can have the same semantics (think for instance of a complex building that has been triangulated, there can be many triangles for one given surface). Because of this, a semantic surface, which is a pivotal concept in CityGML, becomes a JSON object that is stored separately from the geometry of a city object. By doing so, a semantic surface object has to be declared only once, and each of the surfaces used to represent it can point to it. This is achieved by first declaring all the semantic surfaces in a "surfaces" array, and then declaring an added "values" array that links each surface to its corresponding semantic surface using their respective positions in the arrays.

Geometry templates

CityGML’s Implicit Geometries, better known in computer graphics as templates, are one method to compress files since identical geometries (e.g. benches, lamp posts, and trees), need only be defined once (and translations/rotations/scaling are applied). In CityJSON, they are implemented slightly differently than in CityGML: they are stored at one specific location in the file, and each template can be reused. In CityGML, one reuses the geometry used for another city object, and thus there is no structured way to store them, and furthermore, one has to search for them in the file (with XLinks) because they can be located anywhere (the link could even point to an external reference that needs to be resolved).

A given city object can have a geometry of type "GeometryInstance" (instead of those defined above), which defines the (x,y,z) location, a link to the geometry template, and the transformation matrix.

Appearance

Both textures and materials are supported, and the same mechanisms as CityGML are used for these. The material is represented with the X3D specifications⁵, as is the case for CityGML. For the texture, the COLLADA specifications⁶ are reused, as is the case for CityGML.

Just as for the geometry templates, all material and textures must be located at the same entry point in a CityJSON file; this is in contrast to CityGML where they can be located anywhere.

Schema validation

CityJSON uses schemas defined in JSON Schema⁷ to document its data model and to validate whether a CityJSON file respects the allowed structure and syntax. All the city objects, their attributes, the allowed geometries, and other constraints are defined in schemas that are openly available at https://cityjson.org/schemas/.

CityGML support

CityJSON implements most of the data model, and all the CityGML modules have been mapped to CityJSON objects. However, for the sake of simplicity and efficiency, some modules and features have been omitted and/or simplified. If a module is supported, it does not mean that there is a 1-to-1 mapping between the classes and features in CityGML and CityJSON, but rather that it is possible to represent the same information, but in a different manner. CityJSON is thus conformant to a subset of CityGML, although technically only CityGML files (encoded with the XML format) can be conformant to the specifications of CityGML [22, Clause 2 about Conformance].

Compression of CityJSON files

To reduce the size of a file, it is possible to represent the coordinates of the vertices with integer values, and store the scale factor and the translation needed to obtain the original coordinates (stored with floats/doubles). If compressed, a CityJSON file contains a "transform" property:

Several file formats use this, for instance LAS [1] and TopoJSON [7]. For CityJSON, it typically compresses the files by around 5–10%; we give below examples with real-world datasets. It should be noticed that it also makes files more “robust”, in the sense that the coordinates are not prone to rounding because of floating-point representation in a computer [10]. This is the favoured way to store CityJSON files.

Handling and streaming (large) CityJSON files

One drawback of representing geometries by having references to a list of vertices is that large files are difficult to handle (one needs to read all of the file in memory to reconstruct the geometries) and that streaming of large files is thus complicated.

There exists a misconception that CityGML, since it uses the Simple Features paradigm [20], can be easily and directly streamed. We claim that while it is easier, this is not completely true. CityGML files also often contain references between objects in a given file (XLinks), and before this file can be streamed, these references need to be resolved and the objects copied to the location pointing to it. This also increases the size of the file.

Isenburg and Lindstrom [12] proposes to reorganise the order of the information in the file so that the vertices are not all at the end, they rather are located close to the geometries that need them. Special tags in the file informs us about the fact that a vertex will not be used anymore, thus allowing us to free the memory.

This cannot be used with the current structure of CityJSON, but we propose instead to partition a CityJSON file into several files. The rule can be based on a spatial partition, on the type of city objects, or simply randomly. It suffices to update the list of vertices and the indices, which is a simple operation. The open-source software cjio has an implementation of this.

Partitioning a given CityJSON file into several usually will not increase the storage. There will be several properties (e.g. the CRS, metadata, etc.) that will be repeated for each of the files, but the indices in each file will be smaller (always starting at 0), and thus in practice we have noticed that the size will actually decrease.

Support for metadata

CityGML has very limited support for metadata [16]. Only a few elements are supported, such as the bounding box and the CRS, and most elements are on the city model level and not on the module or city feature level. While there exists a metadata ADE for CityGML⁹, in CityJSON metadata is incorporated into the core schema. CityJSON metadata is developed with ISO 19115 (the metadata standard specifically for geographic information developed by the International Organization for Standardization) as the base and further includes elements important for 3D city models, such as the levels of detail present, extensions (and their metadata), presence of textures and/or materials, etc. It also supports metadata at the city model level, the module level and the city feature level.

This is the only addition that CityJSON makes to the CityGML data model.

Software to read/write CityJSON

There are already several software programs to create, parse, visualise, and edit CityJSON files. These were written in different languages (mostly Java, C++, Python, and Ruby) and have been coded during the development of the CityJSON specifications; our workflow involved testing new features to ensure that in practice they are implementable.

The structure of a CityJSON file has been developed so that the developer who wants to parse the file does not have to use an auxiliary data structure to index and extract information from the file. One example is that all city objects are indexed in a dictionary (by their identifier), which allows the developer to have direct access to them; this is particularly useful because the city objects have been flattened out, and a ~Building~ refers to its ~BuildingPart~s by their identifiers. Many other features of CityJSON are based on the simple indexing of objects in an array (templates, textures, materials, etc.), and thus they can be accessed directly by their index in the array.

We provide in this section an overview of a few software implementations, but this list is not exhaustive.

citygml4j: an open-source Java class library and API for facilitating the reading/writing/editing of CityGML files. Starting from version 2.6.0, it supports parsing and writing CityJSON, and all of the features of CityJSON are supported. It can automatically convert CityGML toCityJSON (and vice-versa); the datasets used for the experiments in this paper have all been automatically converted with citygml4j. [https://github.com/citygml4j/citygml4j]

cjio: a Python command-line interface (CLI) program to process and manipulate CityJSON files. The different operators can be chained to perform several processing operations in one step (thus avoiding saving several temporary files). Examples of operators are: creating a subset given certain rules, validating with the CityJSON schemas, merging several files in one, reprojecting to a different CRS, and modifying the paths for the textures. [https://github.com/tudelft3d/cjio]

azul: a modern 3D viewer for macOS, written in Swift and C++. It supports the primitives and semantical surfaces in CityJSON; textures, material and geometry templates are currently not supported. The datasets in Fig. 3 are visualised in azul. [https://github.com/tudelft3d/azul].

	CityGML				CityJSON
	Size(a)	No space(b)	LoD	Texture	Size-float(c)	Size-int(d)	Compr.(e)
Den Haag(1)	23 MB	18 MB	2	Material	3.1 MB	2.9 MB	6.2
Montréal(2)	56 MB	42 MB	2	Yes	5.7 MB	5.4 MB	7.8
New York(3)	590 MB	574 MB	2	No	110 MB	105 MB	5.5
Railway(4)	45 MB	34 MB	3	Yes	4.5 MB	4.3 MB	8.1
Vienna(5)	37 MB	36 MB	2	No	5.6 MB	5.3 MB	6.8
Zürich(6)	435 MB	423 MB	1	No	127 MB	100 MB	4.4

^(a)size does not take into account the size of the textures files (PNG, JPG, etc) since CityJSON refers to the same ones

^(b)the carriage returns, tabs, and spaces are removed, for a fair estimation of the compression factor

^(c)coordinates represented as double/float

^(d)coordinates represented as integer (compressed files)

^(e)compression factor = CityGML(no spaces) / CityJSON(size-int)

⁽¹⁾tile 01, https://data.overheid.nl/data/dataset/ngr-3d-model-den-haag

⁽²⁾tile VM05, https://tinyurl.com/y8eglpmn

⁽³⁾LoD2 tile DA13, https://www1.nyc.gov/site/doitt/initiatives/3d-building.page

⁽⁴⁾CityGML v2 demo Railway, https://www.citygml.org/samplefiles/

⁽⁵⁾the demo file, https://tinyurl.com/yaopvy6w

⁽⁶⁾version with Max height, https://data.stadt-zuerich.ch/dataset/geo_3d_blockmodell_lod1

3dfier: a software to automatically construct 3D city models from 2D GIS datasets and elevation datasets (LiDAR). The polygons are lifted to their elevation, and their semantics is taken into account. One of the output formats of 3dfier is CityJSON. [https://github.com/tudelft3d/3dfier]

QGIS plugin: a simple QGIS plugin to load CityJSON files has been developed in Python. The city objects are loaded as features in layers and can be divided and styled in different layers according to their object type; their geometry can be visualised both in the 2D and 3D view, while their semantic information can be displayed in the attribute table. [https://github.com/tudelft3d/cityjson-qgis-plugin].

val3dity: a validator for the 3D geometries defined in ISO 19107 [13]. Written in C++. CityJSON fully supported. Full details of the implementation in [17] and [18]. [https://github.com/tudelft3d/val3dity]

CityJSON web-viewer: a simple web-based viewer written in JavaScript. Anyone can simply open a local file and visualise it, all the operations are done locally in the browser. It does not support attributes querying or other queries at this moment, but demonstrates that simple tools can be built quickly if the encoding is simple. [https://viewer.cityjson.org/]

Experiments with real-world datasets

To demonstrate and test the software packages mentioned above, we have taken a few subsets of openly available datasets stored in CityGML, and converted them automatically (with citygml4j) to CityJSON. The datasets used are shown in Table 1 and Fig. 3.

These datasets were reconstructed with different methodologies and utilising different software, and they cover a wide-range of possibilities: textures, no textures, material, geometry templates, different LoDs, etc.

The first thing to notice is that CityGML files, as downloaded, often contain several carriage returns, extra spaces, and tabs, and these can significantly increase the file size (and have no use for computers). We have therefore removed all of these to provide a fair comparison of the file size with CityJSON (which do not contain any either). While this might seems unimportant, one can observe that for the CityGML datasets we used, the compression obtained is already large, e.g. for the Montréal dataset we obtain 25%.

Notice also that the compression obtained by encoding the vertices in integers (see the section about compression of CityJSON files, above) can also be significant.

If we compare CityGML files without any spaces or carriage returns to the CityJSON files (integer coordinates), the average compression factor is about six (it varies between 4.4 and 8.1). This varies because of several reasons: (1) if several geometries are shared/adjacent, in CityJSON the vertices are merged; (2) generic attributes are very verbose in CityGML, and in CityJSON they do not occupy extra space, they are considered simply as an attribute; (3) simple files like Zürich contain only simple LoD1 blocks, and there is no semantics or any other features used (e.g. geometry templates); (4) some of the compression is obtained because in the original CityGML file each polygon has a gml:id, and this is lost during the translation (we believe this ID has little meaning in practice, and is stored simply because the export function created it).

Observe that the presence of textures and materials does not seem to affect the compression factor, this is explained by the fact that the sizes of the textures is not taken into account (both CityGML and CityJSON simply refer to the files on disk), and more or less the same mechanism is used.

We have tried with several other different datasets, and we have obtained similar results.