Nov 172014
 

We are pleased to announce that a new Ordnance Survey licence agreement for Digimap is now available.

The new End User Licence Agreement (EULA) can be viewed in the Digimap Help Pages. As part of the new licence arrangements, end users need to agree to the EULA to access the Ordnance Survey data through the Digimap Service. Initially this will need to be done every time you access the collection via a popup after you select an application, however we are working on a new registration system that will mean you only need to do this once.

There is also an updated list of FAQs for the licence.

However, if you have any questions relating to the licence please do not hesitate to contact the Digimap support team.

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302

What’s changed in the new agreement?

See below for key changes. Supplementary information is available from the EULA, the Ordnance Survey website and the EDINA helpdesk.

1. The following new products have been introduced to the Digimap Ordnance Survey collection under the new licence:

  • OS MasterMap® Integrated Transport Network Layer™ (ITN) Layer Urban Paths Theme.
  • OS MasterMap® Topography Layer will also contain the Sites Layer.
  • OS Terrain® 5, introduced as the new high resolution height dataset, replacing Land-Form PROFILE®.
  • PointX Points of Interest (PoI).

These will be made available through the Digimap service as soon as possible.

2. Changes to image publication size restrictions for external use

Image publication size restrictions have changed significantly with many being removed. Please refer to the Your Obligation/Restrictions section of the EULA, specifically Clause 5.1.4 and 5.2.

3. Public sector data sharing rights

In certain situations, it is now possible for you to share Digimap Ordnance Survey data with government departments and for government departments to share their Ordnance Survey data with you. For further details please see FAQs 27 & 28 in Licence FAQ section and consult the EULA Clause 4.

4. Authorised Data Handler

Where a Digimap Authorised User needs to use Ordnance Survey data in a secure data service, the new Digimap Licence now permits that via a separate Digimap Data Handler Agreement. Please see EULA Clause 3.1.4.

5. Educational navigation product/service

For Educational Use, you are now permitted to build Educational navigation apps. See the EULA, Clause 3.4 for the scope of what is permitted.

6. Use of YouTube and Twitter

You may now publish Ordnance Survey data, in line with strict terms detailed in EULA Clause 5, on YouTube and Twitter.

7. There is a change in the copyright notice that must be included on any maps

You must include the following acknowledgements on any display or reproduction of the Ordnance Survey Licensed data:

© Crown Copyright and Database Right [insert date]. Ordnance Survey (Digimap Licence)

Where the date to be inserted should be the current year.


For HE institutions:

If you are responsible for your HE institution subscribing to Digimap Ordnance Survey Collection you should already have received an email from Jisc Collections detailing how your institution can agree to the new Sub-Licence. Full details can be found on the Jisc Collections website.


If you have any further questions then please contact the Digimap Support team:

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302

 

 

Share
 November 17, 2014  Posted by at 3:52 pm Access, Digimap News, New Features Tagged with: ,  No Responses »
Nov 132014
 

We’ve recently written blogs on visualising OS MasterMap® Topography Layer Building Height Attribute (BHA) data in ESRI’s ArcGIS and ArcGlobe and also QGIS. These blogs have proved very popular so we have written a 3rd instalment on how to achieve similar results using Autodesk products using AutoCAD Map3D and InfraWorks.

Please see the previous post for information on BHA data coverage, an explanation of the different height attributes supplied by Ordnance Survey (OS) and for details of further information sources; including the excellent Getting Started Guide produced by OS. Please remember this is an alpha release of the data and OS do not guarantee that BHA is error free or accurate. Additionally the dataset is not yet subject to update and maintenance.

Getting Started

Download the following datasets for your area of interest from Digimap using the OS Data Download application:

  1. OS MasterMap® Topography Layer: select ‘GML’ as the format for your data rather than ‘DWG’ as we need to get to some of the raw data values stored in the GML file, and limit your download to just the Buildings using the Layers drop-down in the basket.
  2. OS Terrain™ 50 DTM: this will be used as the base (surface) heights for the area;
  3. BHA data (BHA data is found in the ‘OS MasterMap’ group): select CSV as the format;
  4. Optionally download any additional data you may wish to use as a backdrop draped over the DTM surface, in this example we’re going to use OS MasterMap® 1:2,000 Raster but other datasets could be used.

Preparing BHA data for use

If your downloaded BHA data is made up of more than one CSV file we recommend merging them all together in to a single CSV file first to make subsequent processing easier and quicker. Use a text editor such as Notepad or TextPad rather than Excel, as Excel can change the formatting of numbers which contain leading zeros.

Each object in MasterMap Topography Layer has a unique identifier called a Topographic Identifier, or TOID for short. TOIDs supplied by OS take the format of a 13 or 16 digit number prefixed with ‘osgb’ e.g. ‘osgb1000039581300′ or ‘osgb1000002489201973′. Some applications, including AutoCAD Map 3D , automatically strip off the ‘osgb’ prefix and add three leading zeros to any TOID that has only 13 digits to make them all 16 characters long. In order to make it easier to join BHA data to building features in MasterMap® Topography Layer the BHA files supplied by EDINA have two TOID values:

  • os_topo_toid_digimap is the TOID formatted to match TOIDs in AutoCAD Map 3D, ArcGIS and in the File Geodatabase format supplied through Digimap.
  • os_topo_toid is the original TOID as supplied by Ordnance Survey

You should check the TOID values in your MasterMap data and those in the BHA data to ensure that there is a common field that you can use to match on; we will use os_topo_toid_digimap as this field in the BHA data matches the TOID values in the MasterMap data when used in AutoCAD Map 3D.

Open the merged CSV file in Excel. To ensure that the data is displayed correctly in Excel you should import the data as follows:

  1. Open a blank Excel document then use the ‘From Text’ option which can be found on the Data ribbon.  This allows you to specify the correct field delimiter and data types for the TOID columns, ensuring they are imported as text fields, show image.
  2. Import the data as a ‘delimited’ file, show image.
  3. Specify ‘Comma’ as your delimiter, show image.
  4. On the Text Import Wizard – Step 3 of 3 window select the first column in the ‘Data Preview’ section and set the ‘Column data format’ to ‘Text’. Repeat this step for the second column. This ensures that Excel treats the two TOID columns as text rather than numbers so doesn’t strip off the leading zero’s from any of the values (which are needed when joining the data to the building features in MasterMap later on), show image.
  5. Press Finish to complete the process, after which your data in Excel should look like the image below with the TOID values in the first column all 16 characters long and including three leading zero characters where necessary:Excel showing TOID values imported as text
  6. AutoCAD Map 3D requires a ‘named range’ of cells to connect to. To create this highlight/select all cells that contain data in the workbook and using the ‘Name Box’ give this selection of cells a name. In the screen grab below we have called the selection ‘BuildingHeightValues’ (note your name cannot include spaces):
    Excel named range
  7. Save your file as an .xslx file.
  8. The next step is to use the Windows ODBC Data Source Administrator to create a connection that points to this .xlsx file. Open the ODBC Data Source Administrator, the easiest way of doing this is to use the Windows search tool to search for ‘ODBC Data Source’.
    ODBC Data Connection Administrator
  9. On the ‘User DSN’ tab press the ‘Add…’ button to create a new ODBC connection:
    Add ODBC Connection
  10. Select ‘Microsoft Excel Driver’ and press the Finish button.
  11. Give your connection a name in the ‘Data Source Name’ field, and using the ‘Select Workbook…’ button browse and select the .xlsx file created above.
    ODBC Select File
  12. Press OK and the newly created User DSN will be listed:
    ODBC Data Source Administrator
  13. Press OK to close the ODBC Data Source Administrator.

Preparing MasterMap Topography Layer GML data for use

  1. Open AutoCAD Map 3D.
  2. At the command prompt type: MAPIMPORT, or select ‘Map Import’ from the Insert menu.
    AutoCAD Map 3D Map Import
  3. Browse to the .gz file downloaded from Digimap, ensure the ‘Files of type’ drop-down is set to ‘OS (GB) Mastermap (*.gml, *.xml, *.gz)’.
    AutoCAD Map 3D Browse
  4. Import just the TopographicArea layer by deselecting all other layers in the import dialog.
    AutoCAD Map 3D Import
  5. Click on the word <None> in the Data Column for the TopographicArea layer.
  6. In the Attribute Data window select ‘Create object data’ and press OK.
    AutoCAD Map 3D Attribute Data
  7. Select ‘Import polygons as closed polylines’ and press OK.
    AutoCAD Map 3D Closed Polylines
  8. The data will be imported in to your current map window. Note you may need to select View > Extents to see the data.
  9. The data needs to be converted to an .sdf file to allow the Building Height Attribute data to be joined to it.
    1. At the command prompt type: MAPEXPORT
    2. Select ‘Autodesk SDF (*.sdf)’ as the file type.
    3. On the Feature Class tab, click on the ‘Select Attributes…’ button.
    4. In the Select Attributes window select ‘Object Data’ and press OK.
      AutoCAD Map 3D Select Attributes
    5. On the Map Export window press OK to export the data.
  10. Connect to the .sdf file just created:
    1. In the Task Pane select Data > Connect to Data…
      AutoCAD Map 3D Connect To Data
    2. Select ‘Add SDF Connection’.
      AutoCAD Map 3D Add SDF Connection
    3. Give your connection a name and browse to the .sdf file exported in the previous step.
    4. Click the Connect button to establish the connection.
    5. Press the ‘Add to Map’ button to add this data to your current map window.
      AutoCAD Map 3D Add to map

You have now added MasterMap buildings to your current map window, the next step is to connect to the Building Height Attribute (Excel spreadsheet) and join it to the building features in MasterMap.

Joining Building Height Attribute to buildings in MasterMap

  1. Connect to the BHA spreadsheet using the ODBC connection:
    1. In the Task Pane select Data > Connect to Data…
    2. Select ‘Add ODBC Connection’
    3. Give your connection a name and select the Data Source Name created above using the ‘…’ button next to the ‘Source’ field.
      AutoCAD Map 3D select DNS
    4. Press the ‘Test Connection’ button.
      AutoCAD Map 3D create ODBC connection
    5. The table in the bottom half of the window will display all named ranges in your spreadsheet, we called our named range ‘BuildingHeightValues’. Before you can select this range for use in AutoCAD Map 3D you need to select a column to use as the ‘Identify Property’. To do this click on the text that says ‘<Click to select>’.
      AutoCAD Map 3D create ODBC connection
    6. In the drop-down that appears put a tick in the box next to the value ‘os_topo_toid_digimap’.
      AutoCAD Map 3D create ODBC connection
    7. Now you can tick the box next to the named range in the spreadsheet and press the ‘Connect’ button.
      AutoCAD Map 3D create ODBC connection
    8. The connection details will be displayed.
      AutoCAD Map 3D create ODBC connection
    9. The Data Connection window can now be closed.
  2. In the Task Pane right click on the MasterMap data and select ‘Create a Join…’
    AutoCAD Map 3D create a join
  3. Join detailsIn the ‘Create a Join’ window:
    1. select the building height data (in the Excel spreadsheet) as the ‘Table (or feature class) to join to’;
    2. select ‘TOID’ in the left hand drop-down menu;
    3. select ‘os_topo_toid_digimap’ in the right hand drop-down menu;
    4. select ‘Keep only left-side records with a match’ in the ‘Type of Join’ section;
    5. press ‘OK’ to create the join.
  4. To verify that the Join has worked, open the data table for the MasterMap data, this is done in the Task Pane by selecting the MasterMap data then pressing the ‘Table’ button. The table will be displayed, scroll to the right to see the joined building height values:
    AutoCAD Map 3D attribute table
  5. The final step is to export the joined data as a new .sdf file which we can then visualise in 3D in InfraWorks. This is done by either right clicking on the MasterMap layer in the Task Pane and selecting ‘Export Layer Data to SDF…’ or by using the ‘Export to SDF’ function on the ‘Vector Layer’ ribbon in the ‘Save’ group.

Visualising the data in 3D using Autodesk InfraWorks

So far we have downloaded OS MasterMap® Topography Layer and BHA data for the same area and joined the two together to create a new dataset containing just the building features which now include the various height attributes published by OS. We also downloaded additional data to use as a backdrop draped over the DTM surface, in this example we will use OS MasterMap® 1:2,000 Raster, but OS VectorMap® Local Raster or OS 1:25,000 Scale Colour Raster would also be suitable depending on the scale of your study area.

Visualising the data in 3D is achieved using Autodesk’s InfraWorks product . The steps below describe how to use the application to create a 3D model:

  1. Open InfraWorks and create a new model.
  2. Specify a location to save the model and give it a name:
    InfraWorks New Model
  3. Click and drag the OS Terrain 50 DTM in to InfraWorks; the file to drag is the one with the .asc extension.
  4. In the Data Source Configuration window, ensure the Type is set to ‘Terrain’ and Coordinate System is set to ‘BritishNatGrid’:
    InfraWorks Data Source DTM
  5. Press the ‘Close & Refresh’ button; the DTM should be displayed:
    InfraWorks showing DTM only
  6. Click and drag the final .sdf file created in the final step of the previous section which contains the heighted building data (i.e. the .sdf file created after joining the MasterMap buildings to the Building Height Attribute data spreadsheet).
  7. In the Data Source Configuration window, set the ‘Type’ drop-down to ‘Buildings’ and select a suitable ‘Roof Height’ attribute using the drop-down on the Common tab. As with previous blogs we have used the RelH2 attribute as we found this gave the best overall representation of building heights relative to each other:
    Data source configuration
  8. On the ‘Geo Location’ tab select ‘BritishNatGrid’ as the coordinate system:
    Data source configuration
  9. On the Source tab select ‘Drape’ from the drop-down under the ‘Draping Options’:
    Data source configuration
  10. Press the ‘Close & Refresh’ button, the buildings should now be displayed on top of the DTM, you may need to pan or zoom to view the data:

    InfraWorks with 3d buildings draped over OS Terrain50

    OS Terrain™ 50 with buildings from OS MasterMap® Topography Layer extruded on top using Building Height Attribute data.

  11. To give some more context to the visualisation you can drape additional raster layers on top of the DTM such as OS MasterMap® 1:2,000 Raster. This is done by selecting all the raster files and dragging them in to the InfraWorks window.
  12. In the Data Source Configuration window ensure ‘Type’ is set to ‘Ground Imagery’, and one the ‘Geo Location’ tab select the ‘BritishNatGrid’ Coordinate System:
  13. Select the ‘Close & Refresh’ button and the map data will be draped over the DTM surface:

    Infraworks with Terrain50 DTM, MasterMap 1:2,500 Raster and Heighted Buidlings

    OS MasterMap® 1:2,000 Raster draped on top of OS Terrain™ 50, with buildings from OS MasterMap® Topography Layer extruded on top using Building Height Attribute data.

The finished visualisation

The screen grab below shows the final visualisation centred on Biggar using OS MasterMap® 1:2,000 Raster as the surface layer.

InfraWorks visualisation

OS MasterMap® 1:2,000 Raster draped on top of OS Terrain™ 50, with buildings from OS MasterMap® Topography Layer extruded on top using Building Height Attribute data.

Share
Nov 062014
 
QGIS 3D visualisation with trees

Our recent blog post about visualising OS MasterMap® Topography Layer Building Height Attribute (BHA) data in ArcGIS and ArcGlobe prompted a number of questions about whether it’s possible to do something similar in open source software. In this post we’ll show you how to achieve something similar using QGIS and the freely available Qgis2ThreeJS plugin.

Please see the previous post for information on BHA data coverage, an explanation of the different height attributes supplied by Ordnance Survey (OS) and for details of further information sources; including the excellent Getting Started Guide produced by OS. Please remember this is an alpha release of the data and OS do not guarantee that BHA is error free or accurate. Additionally the dataset is not yet subject to update and maintenance.

Getting started

  1. Download the following datasets for your area of interest from Digimap using the OS Data Download application:
    1. OS MasterMap® Topography Layer: select the ‘File Geodatabase’ format for your data as this format does not require any conversion to use it in QGIS;
    2. OS Terrain™ 50 DTM: this will be used as the base (surface) heights for the area;
    3. BHA data (BHA data is found in the ‘OS MasterMap’ group): select CSV as the format;
    4. Optionally download any additional data you may wish to use as a backdrop, such as OS VectorMap® Local Raster or OS 1:25,000 Scale Colour Raster;
  2. Open QGIS and load in the OS MasterMap® Topography Layer, OS Terrain™ 50 DTM and your backdrop map data.

Preparing BHA data for use

If your downloaded BHA data is made up of more than one CSV file we recommend merging them all together in to a single CSV file first to make subsequent processing easier and quicker. Use a text editor such as Notepad or TextPad rather than Excel, as Excel can change the formatting of numbers which contain leading zeros.

Each object in MasterMap Topography Layer has a unique identifier called a Topographic Identifier, or TOID for short. TOIDs supplied by OS take the format of a 13 or 16 digit number prefixed with ‘osgb’ e.g. ‘osgb1000039581300′ or ‘osgb1000002489201973′. Some applications, such as ArcGIS, automatically strip off the ‘osgb’ prefix and add three leading zeros to any TOID that has only 13 digits to make them all 16 characters long. Additionally this same formatting is applied to the File Geodatabase format of MasterMap supplied through Digimap. In order to make it easier to join BHA data to building features in MasterMap® Topography Layer the BHA files supplied by EDINA have two TOID values:

  • os_topo_toid_digimap is the TOID formatted to match TOIDs in ArcGIS and in the File Geodatabase format supplied through Digimap.
  • os_topo_toid is the original TOID as supplied by Ordnance Survey

You should check the TOID values in your MasterMap data and those in the BHA data to ensure that there is a common field that you can use to match on; we will use os_topo_toid_digimap as this field in the BHA data matches the TOID values in the MasterMap data downloaded in File Geodatabase format from Digimap.

Before BHA data can be loaded in to QGIS it is necessary to create a small text file (called filename.csvt, where ‘filename’ is the name of your BHA csv file) that specifies the data type of each field so that QGIS handles it correctly. Specifically the .csvt file is used to ensure that QGIS treats the two TOID  values as text rather than numbers, and all height values as numbers. The steps required are detailed below:

  1. Create a new file called filename.csvt (replacing ‘filename‘ with the name of your BHA csv file) in the same folder as the BHA csv file you wish to import.
  2. Open the file in a text editor such as Notepad or TextPad.
  3. Copy and paste the following text in to the file:
    "String","String","Integer","Date","String","Real","Real","Real","Real","Real","Integer"
  4. Save your changes to the file. Ensure it is saved in the same folder as the CSV file you wish to import.
  5. Add your BHA CSV file to QGIS through the Add Vector Layer function; this will add the data as a table in the QGIS project.

Creating a heighted buildings dataset

In order to create a new heighted buildings dataset from the building features in OS MasterMap Topography Layer and the BHA data we use the GIS ‘join’ function. A join links these two datasets together through a common unique identifier (the TOID) resulting in a set of buildings with height values stored as additional attributes.

  1. Bring up the Layer Properties dialog for the Topographic Area layer in the MasterMap data either by double clicking on the layer in the Layer panel or by right clicking on the layer and selecting Properties from the pop-up menu.
  2. Select the ‘Joins’ tab on the left hand side to display the join panel:
    QGIS Join Window
  3. Press the green plus button to add a new join:
    1. QGIS add vector joinSelect your BHA dataset as the ‘Join Layer’.
    2. Select the correct TOID field that matches the TOIDs in your MasterMap data; as mentioned above we’re using os_topo_toid_digimap as the formatting of this matches the TOIDs supplied in the MasterMap data downloaded from Digimap in File Geodatabase format.
    3. In the ‘Target Field’ select the attribute column that contains the TOIDs in your MasterMap® data; by default this is called ‘TOID’ in MasterMap downloaded from Digimap.
    4. Leave the checkbox selected to ‘cache join layer in virtual memory’ as this will speed up query and display of the data.
    5. Press OK to create the join.
  4. Press OK on the Layer Properties dialog to close the window.
  5. Open the attribute table for the TopographicArea MasterMap layer to verify the join has worked. You will see the additional BHA columns at the end of the attribute table. Note you will see a lot of ‘null’ values in these additional columns as BHA values are only available for ‘building’ features (the TopograhicArea feature class contains features for everything, not just buildings).
  6. Having joined the datasets together, before we can create a heighted buildings dataset we need to select only those buildings which now have height information. This is done using the QGIS ‘Select features using an expression’ button: QGIS select by expression button
  7. QGIS select by expression windowWe are looking to select only features which have a value for the height attribute we wish to use in the 3D visualisation. As mentioned in the previous post, we have found that the ‘RelH2′ attribute provides a good representation of the height of buildings relative to one another. The expression used is shown below. Note the field name, in quotes below, is automatically created by QGIS by adding the BHA table name (NT27) to the attribute column name (RelH2) with an underscore between them:
    "NT27_relh2" IS NOT Null

    QGIS save vector layer

  8. Having selected just the buildings that include height information we can now export these features as a new dataset by right clicking on the TopographicArea dataset in the Layer panel and selecting ‘Save As…’ from the pop-up menu.
  9. Save the dataset as a new Shapefile in a suitable location, selecting the checkboxes ‘save only selected features’ and ‘add saved file to map’.
  10. The newly created heighted buildings dataset will be added to your QGIS project; now it’s time to visualise it in 3D.

Visualising the data in 3D

So far we have downloaded OS MasterMap® Topography Layer and BHA data for the same area and joined the two together to create a new dataset containing just the building features which now include the various height attributes published by OS. We also downloaded OS Terrain™ 50 DTM to use as the surface heights, 1:25,000 Colour Raster and OS VectorMap® Local Raster to drape over the surface.

Visualising the data in 3D in QGIS is achieved using the Qgis2ThreeJS plugin, which can be installed using the QGIS plugin manager if you don’t have it already. The steps below describe how to use the plugin to create a 3D model:

  1. Ensure you have all the data loaded in to your QGIS project that you wish to include in the 3D model, as a minimum you should have your DTM, the heighted buildings dataset and a suitable map layer to drape over the DTM.
  2. Turn off all layers in the Layers panel apart from the surface you wish to drape over the DTM; the buildings will be styled using the Qgis2ThreeJS plugin.
  3. Launch the plugin, which can be found on the Web toolbar.
  4. Using the ‘DEM’ panel of the plugin select your DTM data as the ‘DEM Layer’, leave all other settings at their default values. Tip: by default the surface has a vertical exaggeration of 1.5, if you wish to reduce or increase this, the setting is configured on the ‘World’ panel of the plugin.
  5. Qgis2ThreeJS plugin windowIn the ‘Polygon’ panel of the plugin select your heighted buildings dataset and complete the following settings:
    1. Z coordinate: set to ‘Height from surface’ – this will ensure the buildings sit on the DTM surface.
    2. Under ‘Style’: ensure the ‘Object type’ is set to ‘Extruded’ and select the height attribute you wish to use for the extrusion using the ‘Height’ drop down; as mentioned above, we’re using the RelH2 attribute from OS which is in the column ‘NT27_relh2′ in our data.
    3. Select suitable colours and transparency, we used a medium grey colour with 10% transparency to give a glasshouse effect.
  6. Optionally specify an ‘Output HTML file path’ to save the resultant files. Whilst you’re experimenting we recommend you leave this blank and the plugin will save the data in a temporary location, when you’re happy with the result you can use this setting to save your final visualisation.
  7. Press ‘Run’ to create the 3D model. Once it’s finished processing the model will open in your default web browser.

The plugin outputs an HTML file, along with a small number of accompanying files. The HTML file requires a WebGL compatible browser (WebGL is a method of generating dynamic 3D graphics using JavaScript), most modern browsers are WebGL compatible including IE 11, FireFox, Chrome, Safari and Opera, the Can I Use site offers further information on browser compatibility.

As the files are output as a web page, you can share the results of your work with colleagues without them needing to have any specialist GIS software, however you are not permitted to make the website publicly available as the html and javaScript files contain map data rather than just images of maps.  The Licence does not permit the sharing of licensed data from Digimap with anyone other than registered users of the service: Digimap Licence

Tips

  • The plugin uses the extents of the current QGIS map canvas, so the bigger the area being displayed, the bigger the generated 3D scene and the slower it will display. We have found that areas of up to 10km² display okay, anything bigger tends to be a little slow to respond.
  • If you wish to define specific extents for your 3D scene instead of using the map canvas extents this can be done on the ‘World’ panel of the plugin.
  • Applying a vertical exaggeration to your buildings is achieved through the ‘Multiplier’ setting on the ‘Polygon’ panel of the plugin.
  • You can create 3D models of multiple layers. For example in the screen shot below the trees were created by selecting the ‘Positioned Non Coniferous Trees’ from the OS MasterMap Topographic Point layer. These were then added to the QGIS project twice. Using the plugin one of these layers was extruded as a brown cylinder with a radius of 0.75m and a height of 3m to form the trunk; the other was extruded as a green sphere with a radius of 4.5m and a z coordinate of 4.5m (i.e. the height above the ground surface of the centre of the sphere) to form the tree canopy:
QGIS 3D visualisation with trees

OS VectorMap® Local Raster draped on top of OS Terrain™ 50, with buildings from OS MasterMap® Topography Layer Building Height Attribute and Positioned Non Coniferous Trees extruded on top

The finished visualisation

The screen grab below shows the final visualisation centred on the south side of Edinburgh using OS 1:25,000 Colour Raster as the surface layer.

3D visualisation

1:25,000 Colour Raster draped on top of OS Terrain™ 50, with buildings from OS MasterMap® Topography Layer extruded on top using Building Height Attribute data.

Share
Oct 312014
 

To get the most from OS MasterMap data it is usually best to convert it from its supplied format, GML, to a format better suited to the software you are going to use it in. For our CAD users Digimap has been offering the DWG format for several months (see previous blog post); now we are offering a format that makes the data easier to use in GIS software.

Although Shapefile is still the most commonly requested GIS format it cannot handle the large file sizes (over 2GB) that could be requested from the Data Download service; an area of 100km2 in an urban centre would exceed this limit.  We therefore turned to the File Geodatabase format that works in the two most commonly used GIS software applications, ArcGIS and QGIS. There is more information about Geodatabases on Wikipedia here: Wikipedia -ArcGIS Geodatabase

To select the File Geodatabase format, add some MasterMap to your basket in Data Download. Once in the basket you can click the down arrow in the Format column and change it from the default GML to File Geodatabase:

File Geodatabase in Data Dowload

Geodatabase formats are the recommended formats  for use in ArcGIS software, being the most efficient for data storage and analysis. The format supports the use of .lyr files for styling the data and EDINA has provided some for the MasterMap data downloaded from Digimap.

The data can be added to a map in ArcGIS and QGIS in the usual way, though in QGIS you need to use Add vector layer –> Directory rather than a Database as you may assume.

Opening a Digimap File Geodatabase in QGIS

Styling information for both ArcGIS and QGIS has been provided by EDINA in .lyr and QML formats, these can be found at the bottom of Digimap’s help page for OS MasterMap. These representations give enough information to view the data in a style that OS MasterMap is commonly viewed in. The help page also contains a link to the official Ordnance Survey SLD styling information, please see the PDF that comes with the SLD files for information on how to use them.

There are help pages for adding styling information here:

If you require any help on using File Geodatabases from Digimap or any other dataset or format then please get in touch:

  • Phone: 0131 650 3302
  • Email: edina@ed.ac.uk
Share
Oct 222014
 

In March 2014 Ordnance Survey (OS) published an alpha release of the much anticipated Building Height Attribute (BHA) dataset, which is an enhancement to OS MasterMap Topography Layer. You can read all about it in their blog post. In this blog we’re going to show you how to integrate the BHA dataset with buildings in the OS MasterMap Topography Layer to create a heighted buildings dataset and visualise it in 3D. We used ArcGIS 10.2 and ArcGlobe to do this but other software could be used.

The first alpha release of BHA included buildings covering approximately 8,000km2 of the country. A second alpha release of BHA was published in July 2014 which covers around 10,000km2 of the major towns and cities in Great Britain. OS publish an interactive map which shows the extents of the areas covered by the alpha release, so you can check if your area of interest is included.

A note of caution, this is an alpha release of the data and OS do not guarantee that BHA is error free or accurate. Additionally the dataset is not subject to update and maintenance. However in time OS intend to include BHA in OS MasterMap Topography Layer so in future it will be supplied and maintained as a part of the Topography Layer.

Attributes supplied by OS

BHA attributesA number of attributes are provided for each building as shown in the image :

  • ground level [AbsHMin]
  • the base of the roof [AbsH2]
  • highest part of the roof [AbsHMax]

Using these three values two additional relative heights are calculated:

  • relative height from ground level to the highest part of the roof [RelHMax]
  • relative height from ground level to base of the roof [RelH2]

Data availability

OS publish the data as a single CSV file containing over 20 million records. This is a very large dataset and can cause data management problems in a desktop environment so EDINA have split the dataset up using the OS 5km grid allowing you to download the data in tiles for your study area. The data is available in CSV and KML formats. To use the data in GIS or CAD packages you should download the data in CSV format; KML is used to visualise the data in Google Earth.

OS 5km gridThe ‘Show Grid/Overlay’ menu on the right hand side in the Data Download application displays the OS 5km grid. This will draw a grid with each square containing the OS 5km tile reference, as shown in the image.

Please note: BHA data is not currently available for the whole country, you should consult the interactive map published by the OS to see if data exists for your area of interest.

Using the data

OS provide an excellent Getting Started Guide which explains in detail the process of getting BHA data in to GIS for subsequent analysis. The main steps are described below but please refer to the Getting Started Guide for full details.

The data is supplied as CSV files. Each record in the file has a unique TOID which can be used to join the data to building features in OS MasterMap Topography Layer.

Getting started
  1. Download OS MasterMap Topography Layer data for your area of interest from Digimap using the OS Data Download application. Select the ‘File Geodatabase’ format for your data as this is a native ArcGIS format and doesn’t require any conversion.
  2. Download BHA data for your area of interest from Digimap using the OS Data Download application (BHA data is found in the ‘OS MasterMap’ group), selecting CSV format.
  3. Open the OS MasterMap Topography Layer data in ArcGIS.
Preparing BHA data for use

If your downloaded BHA data is made up of more than one CSV file we recommend merging them all together in to a single CSV file first to make subsequent processing easier and quicker. Use a text editor such as Notepad or TextPad rather than Excel, as Excel can change the formatting of numbers which contain leading zeros.

Each object in MasterMap Topography Layer have a unique identifier called a Topographic Identifier, or TOID for short. TOIDs supplied by Ordnance Survey take the format of a 13 or 16 digit number prefixed with ‘osgb’ e.g. ‘osgb1000039581300′ or ‘osgb1000002489201973′. ArcGIS automatically strips off the ‘osgb’ prefix and adds three leading zeros to any TOID that has only 13 digits to make them all 16 characters long. In order to make it easier to join BHA data to building features in MasterMap Topography Layer the BHA files supplied by EDINA have two TOID values:

  • os_topo_toid_digimap is the TOID formatted to match TOIDs in ArcGIS
  • os_topo_toid is the original TOID as supplied by Ordnance Survey (this should be used in other GIS packages such as QGIS which do not modify the TOIDs in MasterMap Topography Layer)

Before BHA data can be loaded in to ArcGIS it is necessary to create a small text file (called schema.ini) that specifies the data type of each field so that ArcGIS handles it correctly. Specifically the schema.ini file is used to ensure that ArcGIS treats the two TOID  values as text rather than numbers. The steps required are detailed below:

  1. Create a new file called schema.ini in the same folder as the BHA csv file you wish to import.
  2. Open the file in a text editor such as Notepad or Text pad.
  3. Copy and paste the following text in to the file:
    [bha_filename.csv]
    Format=CSVDelimited
    ColNameHeader=True
    Col1=OS_TOPO_TOID_DIGIMAP Text
    Col2=OS_TOPO_TOID Text
    Col3=OS_TOPO_VERSION Long
    Col4=BHA_ProcessDate DateTime
    Col5=TileRef Text
    Col6=AbsHMin Double
    Col7=AbsH2 Double
    Col8=AbsHMax Double
    Col9=RelH2 Double
    Col10=RelHmax Double
    Col11=BHA_Conf Long
  4. The first section of code, in square brackets shown in red above, refers to the name of the csv file you wish to import. You should modify this filename so that it references your BHA csv file.
  5. Save your changes to the file. Ensure it is called schema.ini and is saved in the same folder as the csv file you with to import.
  6. Add your BHA csv file to ArcGIS through the Add Data function; this will add the data as a table in the map document.
Creating a heighted buildings dataset

ArcGIS JOIN windowIn order to create a new heighted buildings dataset from the building features in OS MasterMap Topography Layer and the BHA data we use the GIS ‘join’ function. A join links these two datasets together through a common unique identifier (the TOID) resulting in a set of buildings with height values stored as additional attributes.

  1. Right click on the Topographic Area layer in the table of contents > Joins and Relates > Join. This will bring up the Join Data window which can be completed as shown. Remember to join to the TOID in the csv file that is formatted to match the TOIDs displayed in ArcGIS (os_topo_toid_digimap).
    Tip: to create a dataset which just includes the heighted buildings select ‘Keep only matching records’.
  2. Having joined the datasets together we can then export the result as a new Feature Class in our File Geodatabase for subsequent use and analysis. This is done by right clicking on the Topographic Area layer in the table of contents > Data > Export Data…  give your new dataset a suitable name and select your existing File Geodatabase as the destination.
Visualising the result in ArcGlobe

So far we have downloaded data from OS MasterMap Topography Layer and BHA data for the same area and joined the two together to create a new dataset containing just the building features which now include the various height attributes published by OS. Now the fun begins!

We can easily visualise the heighted buildings dataset in 3D using ArcGlobe or ArcScene. The following steps describe how to import the data in to ArcGlobe.

  1. Download the OS Terrain 50 DTM for your area of interest from Digimap using the OS Data Download application. This will be used as the base (ground) heights for the area to provide a more accurate terrain model than is available by default in ArcGlobe.
  2. Open ArcGlobe and add in the DTM. You will be asked if you wish to use the DTM as and ‘image source’ or an ‘elevation source’. You should select the ‘elevation source’ option:

ArcGlobe add DEM window

  1. The Geographic Coordinate Systems Warning dialog will appear as OS MasterMap Topography Layer data is in a different coordinate system (British National Grid) from that used by ArcGlobe (WGS 84):

ArcGlobe Geographic Coordinate Systems Warning

  1. You should specify the transformation used to ensure that the data is accurately positioned on the globe. Using the Transformations… button you should specify the ‘OSGB_1936_To_WGS_1984_Petroleum’ transformation:

ArcGlobe Geographic Coordinate System Transformation

  1. Adding your heighted building dataset from your File Geodatabase is achieved through the Add Data button. Once added you may need to zoom to the layer to view it: right click on the layer in the table of contents > Zoom To Layer.
  2. By default the data is not extruded vertically so appears flat on the earth’s surface. To visualise the buildings in 3D right click on the layer in the table of contents and select Properties and then click on the Globe Extrusion tab.
  3. Select the ‘Extrude features in layer’ checkbox and then in the ‘extrusion value or expression’ box enter the following:
[relh2] * 1.5

ArcGlobe layer properties

This will extrude the buildings using the RelH2 attribute with a vertical exaggeration of 1.5 times (i.e. buildings will be shown 1.5 times their actual height). We found using RelH2 (the relative height from ground level to base of the roof) provides a more useful visualisation over RelHMax (the relative height from ground level to the highest part of the roof) which can lead to some overly tall looking buildings where they include towers that extend significantly beyond the height of the rest of the roof.

The end result

The image below shows an area of Edinburgh including Edinburgh Castle with Arthurs’ Seat in the background. Aerial imagery from ArcGlobe is draped over OS Terrain 50 data for the region with heighted buildings drawn on top. Using the tools in ArcGlobe it is easy to explore the landscape, navigating across the surface and examining the relationships between buildings in the built environment.

BHA data in ArcGlobe

Further information

OS published Release Notes for the alpha releases of BHA. Additional information can be found in Annexe D of the OS MasterMap Topography Layer User Guide and Annexe E of the OS MasterMap Topography Layer Technical Specification.

 

Share
Sep 252014
 

Even though we update our mapping data throughout the year we still make a large number of changes over the summer.

Along with the general updates to MasterMap and a few other products the biggest change over the summer has been to swap the 1:10 000 Scale Raster maps to VectorMap Local Raster maps in Digimap Roam. Ordnance Survey have retired the 1:10 000 data, so we have removed it from the online maps, though you do still have a couple of options if you are a fan of the product:

  1. You can view the VectorMap Local data styled to look like the 1:10 000 maps at two zoom levels in Roam; remember that on the Map Content tab you can remove the contours for a very close representation of the mapping.
  2. We still make the 1:10 000 data available in Digimap’s Data Download interface, and will continue to do so for at least this academic year.

The table below details all of the latest changes we have made showing the currency dates of all Ordnance Survey datasets in Digimap.

Digimap Roam

Ordnance Survey product Data Currency Last updated in Roam
OS MasterMap® Topography Layer May 2014 July 2014
OS VectorMap® Local July 2014 September 2014
OS VectorMap® Local Raster July 2014 September 2014
1:25 000 Scale Colour Raster March 2014 March 2014
1:50 000 Scale Colour Raster June 2014 July 2014
Meridian™ 2 January 2014 March 2014
OS Terrain™ 50 July 2013 October 2013
OS VectorMap® District Raster September 2014 September 2014
Strategi® January 2014 March 2014
1:250 000 Scale Colour Raster June 2014 September 2014
MiniScale® January 2014 February 2014

Data Download

Ordnance Survey product Data Currency Last updated in Data Download
OS MasterMap® Integrated
Transport Network™ (ITN) Layer
May 2014 September 2014
OS MasterMap® Topography Layer May 2013 September 2014
OS MasterMap® Topography Layer Raster June 2013 October 2013
Backdrop Mapping
* 1:10 000 Scale Raster June 2013 October 2013
OS Street View® April 2014 April 2014
1:25 000 Scale Colour Raster March 2014 March 2014
OS VectorMap® Local Raster July 2014 September 2014
OS VectorMap® District Raster September 2014 September 2014
1:50 000 Scale Colour Raster June 2014 July 2014
1:250 000 Scale Colour Raster June 2014 June 2014
MiniScale® January 2014 January 2014
Land and Height Data
* Land-Form PROFILE® November 2009 September 2011
* Land-Form PROFILE® DTM November 2009 September 2011
OS Terrain™ 50 July 2014 September 2014
OS Terrain™ 50 DTM July 2014 September 2014
* Land-Form PANORAMA® November 1993 September 2011
* Land-Form PANORAMA ® DTM June 2006 September 2011
Vector Data
OS VectorMap® Local July 2014 September 2014
OS VectorMap® District September 2014 September 2014
Meridian™ 2 January 2014 February 2014
Strategi® January 2014 February 2014
Boundary and Location Data
Boundary-Line™ May 2014 May 2014
Code-Point® August 2014 September 2014
Code-Point® Open August 2014 September 2014
Code-Point® with polygons April 2014 July 2014
OS Locator™ May 2014 May 2014
1:50 000 Scale Gazetteer June 2014 June 2014

* Land-Form PANORAMA®, PROFILE® and 1:10 000 Scale Raster products have been withdrawn by Ordnance Survey and are no longer updated.

Other Updates

Remember there have also been substantial changes made to the Geology and Marine data holdings in recent months:

Please also note that there are a lot more formats available than before so you may not have to convert the data after you have downloaded it:

It is now easier to see which formats are available for the data product you are downloading and change it if you need to, this blog post shows you how:

Please let us know if you have any questions about the new data or any other changes and additions to the Digimap service

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302

 

Share
 September 25, 2014  Posted by at 11:34 am Data Changes & Additions, Digimap News, New Features Tagged with: ,  No Responses »
Sep 242014
 

We have updated all the new style Download interfaces in Digimap to make it easier to change the options for the data you have selected. When you have added some data to your basket you can now see more clearly what can be changed; the version (date and style), the format and the layers.

New Download Basket

Where these options are highlighted in blue with a next to them you can click them and get a drop down menu with the alternatives.

New Download basket with open menu

Make the changes you need, add a name,  then click on the Request Download button to order your data.


 

We have also made a slight change to the Product Information pages for some of the data products too.  Where the data is available on tiles you can now click a Show Grid button to see the tile outlines on the map.  When you make a selection on the map you will get all the grid tiles that are partially or entirely within your orange selection area.

How to View a Grid in Data Download

This feature is particularly useful for data products like the 1:10,000 and 1:25,000 scale geology data and the Marine HydroView Charts, where the data isn’t a continuous coverage and your selected area may contain no tiles. When you click the Show Grid button it will also automatically open the menu from the right of the map which allows you to change the grid shown or switch it off.

Remember, you open the Product Information panels by clicking on the blue Info links in the list of data products to the left of the map.

Please let us know if you have any questions about these changes or anything else:

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302
Share
Aug 062014
 

We have now added detailed 1:10,000 and 1:25,000 geological mapping data and 1:625,000 hydrogeological data to Geology Download. The mapping data is the most detailed geological mapping available from the British Geological Survey (BGS), consisting of four polygon layers:

  • bedrock geology (‘solid’);
  • superficial deposits (‘drift’ or Quaternary)
  • mass movement (mostly landslide)
  • artificial (or man-made ground)

…and one linear features layer, including:

  • thin beds (such as coal seams and fossil bands)
  • faults
  • mineral veins
  • some landforms

The products cover around 30% of the UK most of the which is provided by the 1:10,000 scale data (approx 28% of the UK), as the coverage is not complete we have added overlays to Geology Download to guide you to the areas where it can be found. The BGS have more map tiles in production for this dataset and we will add them to the service as they become available to us. The coverage of the data can be seen in the Download interface below as the dark blue areas of Great Britain.

Geology Download showing 1:10,000 scale data availability

The 1:25,000 has a much smaller coverage, but with little overlap with the 1:10,000 it makes a good companion dataset to fill in the gaps. The 1:25,000 data also fills the gaps in the 1:50,000 data in Wales, again the coverage can be seen in the interface below as the dark blue areas.

Geology Download showing 1:25,000 scale data availability

Grids / Overlays for Geology DownloadTo access the coverage maps click on the double arrow to open the Show Grid / Overlay panel on the right side of the map; you can then add the grids to the map to see where data is available. We highly recommend you do this before selecting your area for download as it is not possible to add data to your basket if you choose a product with no coverage for your that area. If you find that the Add to Basket button remains grey after you have selected your products it probably means that one or more of them do not have coverage for the area you have chosen.  If you click on the grey button an error message will tell you which products are unavailable, either remove these from the order or switch on the overlay and choose a different location that has the data.

GEology Download Products in July 2014In addition to the 1:10,000 and 1:25,000 scale mapping datasets we have also added the BGS OpenData 1:625,000 scale digital hydrogeological data to Geology Download. This data can be used to indicate the aquifer potential of an area in generalised terms. Areas are categorised into one of three groups:

  • those in which intergranular flow in the saturated zone is dominant
  • those in which flow is controlled by fissures or discontinuities
  • less permeable formations including aquifers concealed at depth beneath covering layers

The 1:625 000 scale data may be used as a guide to the aquifers at a regional or national level, but should not be relied on for local information.

The addition of these new datasets brings the total number of different BGS products in Geology Download to 14.  We hope to add the detailed mapping datasets to Geology Roam later in the year as well as some of the other types of data as overlays or basemaps.

If you have any questions about the new datasets or any requests for geology data we don’t yet have then please get in touch:

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302
Share
 August 6, 2014  Posted by at 10:00 am Data Changes & Additions, Digimap News Tagged with: ,  No Responses »
Jul 292014
 

Marine Digimap has updated datasets in its brand new Marine Download facility.  The new interface provides a single place to get both Hydrospatial data and charts. The Hydrospatial data is now the newer Hydrospatial One dataset from SeaZone which also provides access to much higher resolution gridded bathymetry data, one arc second instead of six.  The Charted Raster dataset has been replaced with Hydroview Charts; these are still the same scanned Admiralty Charts though they are much more recent and include some small scale charts covering whole oceans (still not for navigation though!).

Version of Marine Downloader Launched in July 2014

Marine Download overlays listMarine Download works in the same way as Data Download, Environment Download and Geology Download; you select your area, pick your products and download the  data. The main difference is with datasets as these don’t all form continuous coverage of the UK Coastal waters. We have therefore included overlays that show where the tiles of each dataset can be found. These are very useful when choosing your data, just open the “Show Grid / Overlay” menu on the right of the map and pick the overlay for the data you are selecting.

Remember there can be a lot of overlap between the Hydroview Charts and also large areas with no data, you will get all the charts that present for the area and scale band that you choose. If there is no data for the area you have selected then you will not be able to add anything to the basket, if you try you will get an error message informing you of the products with no coverage.

For a full description on how to use the new interface, have a look at the help page: How to use Marine Download

List of Hydroview Charts in Marine DownloadThe data in Marine Download is broken into three categories, Hydrospatial, Hydroview Charts and Bathymetry. In the Hydrospatial category the same themes as before can be found, the Hydroview Charts are broken up into categories depending on their scale including those extra charts with wider ocean or global extents.

The old interfaces for downloading data, Hydrospatial Download and Charted Raster Download, will be withdrawn in the coming weeks, if you have teaching materials that include them then please update these as soon as possible.

 

If you have any questions about the new download or the withdrawal of the old interfaces then please get in touch:

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302
Share
 July 29, 2014  Posted by at 2:37 pm Data Changes & Additions, Digimap News, New Features Tagged with: ,  No Responses »
Jul 152014
 

We are getting a few reports from users experiencing issues when printing PDF maps using Chrome web browser. Unfortunately, the map watermark and any semi-transparent annotations are being printed as solid filled features. We have discovered that this occurs when the PDF is viewed and printed from within the Chrome browser, using Chrome’s in built PDF plugin.

We have investigated this problem and discovered that the bug is with Chrome and occurs with all PDFs with semi-tranparent layers not just those created by Digimap. The problem only happens on Windows computers with versions of Chrome released since the end of April (Chrome 34.0.1847 onwards).

Digimap users with Chrome should see a warning when they open the Print… window, informing them about the problem.

Print interface showing chrome alert

As the alert message suggests you can disable the plug-in to prevent it from opening PDFs or you can use the following workaround.
Printing from Chrome alternative method

  • Create your printable map as normal it will appear at the bottom of your web browser.
  • Don’t click the file name to open the PDF, click the little arrow and choose ‘Open with system viewer’.
  • Providing it is installed on your machine, the map will open in Adobe Reader which will print the semi transparent layers correctly.

We hope that the issue is resolved by Google who develop the Chrome browser, it has been reported to them. Should you wish to disable the plug-in entirely then you can follow the instructions with the image below. You can also download the map as a file to your computer, and then open it directly with Adobe Reader, this is usually the best option anyway as it allows you to print the PDF multiple times or store it if you don’t want to print it just yet.

Disabling Chrome PDF plugin

  • Type Chrome://plugins into the web address bar in chrome, this opens a list of all the plug-ins you have installed for Chrome.
  • Scroll down to the Chrome PDF Viewer on the Plug-ins page and click on the blue “Disable” link.
  • You will no longer have the option to view PDFs in the web browser.
  • Use this page to reactivate the plug-in if you want to reverse the change in the future.

 

If you need any assistance with opening your PDF map, please contact the EDINA helpdesk:

  • Email: edina@ed.ac.uk
  • Phone: 0131 650 3302
Share