Measuring Galileo Performance: Navigation and Timing Performance Figures of Merit | Part 2 - Inside GNSS - Global Navigation Satellite Systems Engineering, Policy, and Design

Measuring Galileo Performance: Navigation and Timing Performance Figures of Merit | Part 2

This article provides an overview of the current Galileo system deployment status and performance trends since the Initial Open Service declaration on December 15, 2016. The system performance is assessed through a set of navigation and timing performance figures of merit. The outstanding performance results along with the acceleration in the constellation and ground segment deployment confirm that the Galileo system is on track towards Full Operational Capability. That Initial Open Service Declaration in late 2016 marked a historical milestone in the Galileo program. Here the authors will illustrate that Galileo has arrived and that it works.

Despite the not fully deployed constellation and ground infrastructure, a reliable service provision and compliance to the declared performance targets have since become a priority. This represents a challenge for all European GNSS stakeholders, including the European Commission, European GNSS Agency (GSA), and European Space Agency (ESA), who now have to commit to a service from a system still under deployment with further upgrades ongoing, ensuring both incremental system verification and sustained service provision. Nevertheless, it was fundamental to assert Galileo’s presence in the exploding market of mobile devices and location-based services, including dual frequency chips recently embedded in smartphones, where the Galileo E1/E5 signals are already playing a major role. The majority of the last generation smartphones are now Galileo-enabled (see www.usegalileo.eu, Additional Resources), making the Galileo engineering efforts started more than a decade ago a reality in the hands of often-unaware users. With this article we want to show that Galileo is definitely present and it works.

Galileo System Status

Let’s start with a brief summary of the system configuration enabling the Initial Services provision. The latest satellites launches in 2016-2018 (L8, L9, and L10) in the quadruple configuration with Ariane 5 have significantly boosted the deployment of the Galileo Space Segment. The numbers speak for themselves: 26 satellites built and launched in a 7-year time frame. As of August 2018, 17 satellites are currently usable with the Signal In Space flagged as healthy. The latest constellation information can be retrieved from the European GNSS Service Centre website (see Additional Resources).

The latest Galileo satellites, GSAT0219, GSAT0220, GSAT0221, and GSAT0222, were successfully launched with Ariane 5 (Figure 1) from Europe’s Spaceport located in French Guiana on the July 25, 2018. After their commissioning, the Galileo system will reach full worldwide coverage and availability with all constellation slots filled.

As outlined in Table 1, in parallel to the momentum in space, the Galileo ground infrastructure is going through important upgrade milestones, resulting in enhanced core processing facilities, and finally achieving the full redundancy and business continuity of the Galileo Control Centres in Italy and Germany (GCC-I and GCC-D). More operational stations have been added to the worldwide distributed network of Galileo Sensor Stations (GSS), Up-Link Stations (ULS), and Telemetry Tracking and Control (TT&C) stations. New versions of the Ground Mission Segment (GMS) and Ground Control Segment (GCS) are expected to be deployed soon on the operational chain, contributing to further performance and operability improvements.

System Performance Monitoring Platforms

Continuous Galileo system performance monitoring and analysis is being carried out by several entities and independent processing facilities across Europe, with the main contributors as follows;

  1. The Galileo Service Operator (GSOp) Spaceopal GmbH is in charge of the online mission monitoring embedded in the Galileo Control Centres (GCC-Italy and GCC-Germany).
  2. GSA is managing the Galileo Reference Centre (GRC) in Noordwijk, The Netherlands. The GRC primary function is to allow GSA, as Galileo Service Provider, to independently monitor the performance of the system and of the Galileo Service Operator (P. Buist et alia). The GRC was officially inaugurated on May 16, 2018. The current GRC v0 nucleus is already supporting the publication of Initial Open Service Quarterly Performance Reports on the European GNSS Service Centre (see Additional Resources).
  3. ESA, as Galileo system architect, is supporting GSA and GCCs with the continuous monitoring of the system performance and functional behavior in near real time, supporting the diagnosis and resolution of system anomalies, managing system upgrades and associated System Integration and Verification (SIV) activities, as well as leading an end to end system reliability characterization in support of Safety of Life (SoL) applications.

ESA is supervising two independent system performance monitoring facilities, which have supported the Galileo system performance verification since its early stages of development:

  1. The Time and Geodetic Validation Facility (TGVF) has its core Galileo Processing Centre (GPC) located at the European Space Agency (ESA) European Space Technology and Research Centre (ESTEC), in Noordwijk, The Netherlands, under a consortium led by GMV Madrid, Spain.
  2. The Galileo System Evaluation Equipment (GALSEE) platform was developed by Thales Alenia Space Italy (TAS-I) and is hosted in Rome, Italy. TAS-I is also leading Galileo System Engineering Technical Assistance (SETA), an industrial consortium supporting ESA in all system engineering and verification activities.

As part of the Galileo SETA, it is important to mention the contribution provided by the performance team at Airbus Defence & Space (Ottobrunn, Germany) through their in-house expertise and Performance Assessment Facility (M. Kirchner et alia). The performance results provided in this article are a combination of outputs from ESA, TGVF, GALSEE, and Airbus.

As illustrated in Figure 2, TGVF is a distributed system comprising a worldwide network of Galileo Experimental Sensor Stations (GESS), the Orbit Determination and Time Synchronization Validation Facility (OVF), the Time Validation Facility (TVF), and the Galileo Processing Centre (GPC). Within the GPC, the Data Processing and Analysis Facility (DPAF) is the core software element generating the system performance figures of merit in near real time and retrieving data collected in the centralized Data Server Facility. It is important to highlight that GESS stations are independent from the operational Galileo Sensor Stations (GSS).

TGVF is now evolving as a key experimentation and validation platform for state of the art algorithms, supporting the prototyping of advanced Orbit Determination and Time Synchronization (ODTS) algorithms, as well as the Integrity Support Message for A-RAIM.

The GALSEE high-level architecture is shown in Figure 3 (G. Galluzzo et alia). A key differentiator between GALSEE and TGVF is the availability of a direct interface with the Galileo Mission Segment in GALSEE, enabling in-depth troubleshooting of critical ground elements, such as the Orbitography and Synchronization Processing Facility (OSPF) and the Message Generation Facility (MGF).

TGVF is primarily used to assess the broadcast navigation data accuracy as received by the end user, whereas GALSEE focuses on the functional verification of the on-ground ODTS processing leading to the navigation message generation.

Performance Figures of Merit

Although users are ultimately interested in their final positioning performance, i.e., how accurately a GNSS can pinpoint them on a map, for the GNSS service providers a direct commitment in the position domain remains the most challenging. An interesting analogy is provided in the GPS Standard Positioning Service (SPS) Performance Standard. The GNSS system is compared to an electricity provider that can only commit to the level of performance its interface is able to provide, for example, in terms of voltage and frequency at the socket output. The users will then decide how to toast their bread or which program to set for their washing machines. Similarly, the GNSS can only control and commit on the errors that originate from the system itself, therefore excluding local receiver effects such as multipath and interference, as well as propagation effects, despite the fact that these can contribute significantly to the final user solution.

This is the rationale for the derivation of a set of performance figures of merit conveying only the system driven performance, as outlined in Table 2. Performance metrics and calculation methods have been harmonized to a large extent to those established in GNSS international standards, following the guidelines provided by the United Nations International Committee on GNSS (ICG).

Nevertheless, as system architect, ESA is also conducting in-field tests to demonstrate the system and signals usability in real environments, from open sky conditions to urban canyons, exploiting both professional and mass-market receiver technology. Typical Position, Velocity, and Time (PVT) results will be provided for illustration purposes as an indication of attainable performance in the positioning domain.

These figures of merit along with the committed Minimum Performance Levels (MPL) are currently being revised for the upcoming Galileo Enhanced Service provision phase, targeted in 2019 (see presentations by E. Chatre and A. Mozo in Additional Resources).

In addition to the results presented in this article, the Initial Services Quarterly Performance Reports published by GSA on the European GNSS Service Centre website can be referred to for a routine and detailed review of the full set of figures of merit presented in Table 2.

Navigation Message Data Accuracy

The broadcast navigation message accuracy is measured through the Signal In Space Ranging Error (SISE), defined as the difference of the satellite position and time as broadcast by the navigation message and the true satellite position and time, projected on the user-satellite direction. In agreement with the Galileo OS-SDD, the ranging accuracy is calculated as the 95th percentile, per satellite and per constellation, of the instantaneous SISE time series over a 30-day period. Additional conditions include the computation of the error as an epoch-wise global average within each satellite footprint, the usage of healthy Signal In Space only, and exclusion of propagation and receivers effects. More details about the SISE calculation methodology can be found in Additional Resources. SISE is one of the most important GNSS figures of merit, conveying the system driven accuracy of the orbit and clock predictions in the broadcast navigation message.

The long term historical SISE plots shown in Figure 4 and Figure 5, respectively, computed per satellite and constellation over 30-day sliding windows, showed a non-stationary behavior with constant improvement over time, spanning a period of almost four years, from September 1, 2014 to May 31, 2018. SISE has only recently reached a stabilization plateau with a constellation value of about 0.45 meters (95%, global average). This has been achieved through a combination of several factors, including the increased number of operational satellites, enhanced versions of the Ground Mission Segment, and higher uplink rate of the navigation message (lower Age of Data). The gaps in the plots of Figure 4 and Figure 5 in September 2016 are related to system testing activities as per NAGU 2016040. The small bump around May 2017 was caused by a ground segment hardware failure as reported in Service Notice SNGU 2017001, Navigation message not refreshed, published by the European GNSS Service Centre on May 17, 2017. The failed equipment was promptly replaced, and with the current redundancy of the Galileo Control Centers, this type of anomaly is no longer expected. A large SISE deviation is observed for GSAT0203 (E26) in June 2017. This was caused by an anomaly that occurred shortly after a planned clock maintenance activity. The Signal In Space was not flagged in a timely fashion as unhealthy to protect the users. The onset of anomalies and isolated large SISE errors is expected to gradually decrease over time as new ground segment upgrades and operational improvements are deployed, contributing to the overall system reliability towards the full services phase in 2020.

The overall Galileo SISE performance, in terms of 95% statistics, is already at least a factor of two better than other GNSS (O. Montenbruck et alia). This is mainly driven by the on-board clock stability in combination with the high uplink rate of the Galileo navigation messages. As shown in Figure 6, the cumulative Age of Data (AoD) for all satellites computed since the Initial Services declaration resulted in a mean of 45 minutes, opposed to about 12 hours for GPS (see Additional Resources).

Galileo AoD has decreased significantly in recent years, as illustrated in Figure 7, with a reduction by a factor of two in the AoD 95% value, from 275 to 140 minutes, before and after Initial Services. AoD is expected to decrease further with the on-going ground segment and uplink station enhancements. The uplink and refresh rate of the broadcast navigation message is optimized by the ground segment based on the uplink station availability. The uplink rate can be relaxed, for example, in the case of on-ground maintenance activities.

System Availability

Figure 8 shows the trend of the Galileo healthy Signal In Space (SIS) availability calculated as a monthly constellation average. The results provide an average above the 87% minimum annual target set in OS-SDD. As mentioned before, the total number of healthy satellites is expected to increase soon once L10 commissioning is completed.

According to the Galileo OS-SDD, the availability of the OS SIS is defined as the percentage of time a user is able to receive a healthy SIS as a global average, either single frequency (E1, E5a, E5b) or dual frequency (E1-E5a, E1-E5b), from at least one satellite above a minimum elevation angle of five degrees from any point in the coverage area. With the current available constellation as per Table 1, the expectation for this parameter is 100%. In other words, there is always a Galileo satellite in view broadcasting healthy SIS worldwide. This parameter was devised at the early stages of Galileo constellation deployment, when only a limited number of satellites were available and it is less meaningful at this stage of the deployment.

Let’s move on to another Initial Service figure of merit, the horizontal positioning service availability, computed considering only system contributions from healthy Galileo SIS above a minimum elevation of 5 degrees, as a percentage of time of the average location with a horizontal positioning error less than or equal to 10 meters (95%), both for single and dual frequency users. The position solutions are calculated starting from a minimum of three satellites in view assuming a receiver operating in altitude hold mode, using the previous altitude estimation or approximation available. A similar approach is used for the derivation of the HDOP ≤5 availability.

Recent analyses (Figure 9) for the month of June 2018 result in a mean HDOP≤5 availability of 92.0% and a mean horizontal positioning ≤10 m availability of 96.4%. For the same period the mean PDOP≤6 availability is 73.2%.

The system availability will improve gradually with the inclusion of more operational satellites in the Galileo constellation. As shown in Figure 10, the evolution of the PDOP≤6 and positioning service availability for both single and dual frequency users will already exceed 90% with 22 satellites. This working point is expected to be reached soon, once the on-going commissioning activities for L10 satellites are completed. The availability of positioning service is computed through a service volume simulation for rural environment conditions (see OS-SDD for more details).

System Timing Performance Metrics

The system timing performance monitoring is currently being monitored by the TGVF Timing Validation Facility (TVF), using inputs from different UTC(k) European laboratories, led by GMV Spain and the German National Metrology Institute (PTB). In addition to TGVF-TVF, an independent analysis of Galileo timing performance is provided by the Istituto Nazionale di Ricerca Metrologica (INRiM, Italy), in the frame of ESA system verification activities. Galileo-GPS calibrated receivers are connected to the local realization of UTC(k) for the assessment of UTC and GPS to Galileo (GGTO) dissemination accuracy.

The OS SIS UTC time dissemination accuracy is evaluated through the measurements of a standard timing/calibrated laboratory receiver, resulting in minor local error contributions to the overall UTC dissemination error budget. The computation of daily UTC mean values further reduces the impact of short term variations.

As shown in Figure 11, from the start of the Initial Services on December 15, 2016 to June 30, 2018, the UTC dissemination error using UTCrapid as reference is around 8.4 ns (95th percentile), well below the Initial Services target of 30 ns.

Similarly, the accuracy of the broadcast GGTO is computed as the 95th percentile of the difference between the broadcast GGTO prediction conveyed by the Galileo navigation message and the estimated GGTO. As for the UTC dissemination accuracy, only system contributors are considered; user environment and receiver contributions are excluded. As shown in Figure 12, the overall broadcast GGTO accuracy during the period December 15, 2016 – June 30, 2018 was 6.9 ns (95th percentile), also within the corresponding 20 ns specification for Initial Services. GGTO predictions are currently generated using the observations of combined Galileo-GPS calibrated timing receivers located at the two Galileo Control Centers. Further optimization of the GGTO prediction algorithm is on-going in order to mitigate some of the outliers observed in the period reported above.

Figure 13 shows the operational timeline of the master clocks for the Galileo constellation. Most of the satellites are currently operating with Passive Hydrogen Masers (PHM) as prime clock. With 26 spacecraft in orbit, and considering the quadruple redundancy implemented on each satellite, the total number of Galileo atomic clocks delivered to space since 2011 amounts to 104. A limited number of units have shown failures, however this is mitigated by the quadruple redundancy in place and by the recent refurbishment of some clock components performed by the satellite manufacturer, already implemented in L9 and L10 satellites.

Figure 14 illustrates a recent Allan deviation characterization for the Galileo satellite clocks, showing an overall excellent frequency stability and compliance with requirements. In terms of broadcast clock prediction accuracy, the Galileo constellation shows similar or better performance than GPS and other GNSS in the nominal range of the navigation message Age of Data (<3 hours), as reported in several studies (P. F. Madrid Navarro et alia; G. Galluzzo et alia).

In-Field Galileo Performance Evaluation

In addition to the system performance at the Signal In Space interface and associated figures of merit, ESA is also analyzing the attainable end-user performance through field tests conducted in different environments, from open sky to urban canyons, as well as with a variety of hardware setups.

Position, Velocity, and Time (PVT), User Equivalent Ranging Error (UERE), and Time To First Fix (TTFF) are some of the metrics evaluated for static, pedestrian, and vehicular users, using both professional and mass-market receivers (Figure 15) (E. Breeuwer et alia).

Monthly horizontal and vertical positioning accuracy are monitored continuously for fixed user positions at the Galileo Experimental Sensor Stations comprising the TGVF worldwide network (Figures 16 and 17).

Table 3 provides a summary of the detailed positioning accuracy figures obtained at the GESS sites with Galileo-only measurements for the full month of June 2018, using F/NAV E1-E5a dual frequency service, and limited to PDOP≤6 intervals. A simple least square PVT algorithm is used to generate these solutions. The cumulative errors for all the stations in the horizontal and vertical domain were 2.65 meters and 4.42 meters (95%), respectively.

An example of a vehicular test with Galileo-enabled smartphones is shown in Figure 18. The positioning error was calculated against a precise reference generated with a professional inertial navigation system (SPAN) integrated in ESA’s Navigation Testbed Vehicle. This was one of the tests conducted in May 2018 in the frame of the Galileo App 2017/2018 competition, involving Young Graduate Professionals (YGT), trainees, and students at ESA. The challenge was to design an Android smartphone app allowing users to compute and visualize their position based on Galileo measurements only, as well as the possibility of selecting a combination of GNSS constellations to assess their performance (see Additional Resources).

Conclusions

Galileo has been in use since December 15, 2016 with in-field-proven performance. The main system performance trends have been shown in this article, from ranging and timing accuracy to system availability and positioning accuracy. ESA’s key assets for system monitoring and troubleshooting have been presented. The results are fully in line with the Minimum Performance Levels (MPLs) established in the Galileo Open Service Definition Document. The GNSS user community can be confident that Galileo is working nominally and is well on track toward Full Operational Capability.

Acknowledgments

The authors would like to thank the European GNSS Agency (GSA) and the European Commission for the continuous cooperation on the definition, engineering, and monitoring of the Galileo services.

In addition to the teams from ESA Directorate of Technology, Engineering and Quality (TEC), and Directorate of Navigation (NAV), the authors would also like to acknowledge the contributions of the industrial teams involved in the support and development of the facilities mentioned in the article:

  1. TGVF team led by GMV Spain, in particular the Operations Team at the ESTEC Galileo Processing Centre
  2. Galileo System Performance Team at Airbus Defence & Space in Ottobrunn, Germany
  3. Thales Alenia Space in Rome, Italy and the Galileo System Engineering Technical Assistance (SETA) subcontractors

Additional Resources

[1] An Analysis of GPS SPS Performance for 2017, The University of Texas at Austin, 2017

[2] Breeuwer, E. et alia, “Galileo Works! Results of the Galileo In-Orbit Validation Test Campaign,” Inside GNSS, March/April 2014

[3] Buist P., Mozo, A., and Tork, H., “Overview of the Galileo Reference Centre: Mission, Architecture and Operational Concept,” Proceedings of the ION GNSS+, Portland, OR, September 2017

[4] Chatre, E., “Galileo System Status,” Proceedings of ION GNSS+, Portland, OR, September 2017

[5] ESA website, ESA Trainees Compete in Inaugural Galileo App Contest, http://www.esa.int/Our_Activities/Navigation/ESA_trainees_compete_in_inaugural_Galileo_app_contest

[6] European GNSS (Galileo) Initial Services – Open Service – Service Definition Document (OS-SDD), Issue: 1.0, December 2016

[7] European GNSS Service Centre website, https://www.gsc-europa.eu

[8] Galluzzo, G., Rodriguez, R. L., Morgan-Owen, R., Binda, S., Blonski, D., Crosta, P., Gonzalez, F., Garcia, J. Molina, Otero, X., Sirikan, N., Spangenberg, M., Spinelli, E., Swinden, R., and Wallner, S., “Galileo System Status, Performance Metrics and Results,” Proceedings of the 2018 International Technical Meeting of The Institute of Navigation, Reston, VA, January 2018

[9] GNSS Compare, Winner of the ESA Galileo App Competition 2018, https://gnss-compare.readthedocs.io

[10] GPS Standard Positioning Service (SPS) Performance Standard, 4th Edition, September 2008

[11] Kirchner, M. et alia, “Galileo System Performance,” Proceedings of NAVITEC, Noordwijk, The Netherlands, 2016

[12] Montenbruck O., Steigenberger P., and Hauschild A., “Multi-GNSS Signal-in-Space Range Error Assessment – Methodology and Results,” Advances in Space Research, Volume: 61, Issue: 12, 15 June 2018Mozo, A., “What’s Next? Galileo Services Roadmap,” 1st Galileo User Assembly, 28-29 November 2017, Madrid, Spain

[13] Navarro Madrid, P. F., Roldán Gómez, P. J., et alia., “Observing the Behavior of High Stable Galileo Satellite Clocks and Exploring Potential Associated Benefits,” Proceedings of ION GNSS+, Portland, OR, September 2017

[14] useGalileo.eu, Find a Galileo-enabled Device to Use Today, https://www.usegalileo.eu/

Authors

Gaetano Galluzzo is the Galileo Service Performance Engineer in the Commercial User Segment and Navigation System Validation Section at the European Space Agency (ESA) – European Space Technology and Research Centre (ESTEC) in Noordwijk, The Netherlands. His current responsibilities in the Galileo Project Office include the Galileo system performance verification, development of system monitoring platforms, and performance metrics standardization.

Stefano Binda is System Performance Validation Principal Engineer in the Galileo Project Office at ESA – ESTEC in Noordwijk, The Netherlands. He is the Technical Officer of the Time and Geodetic Validation Facility (TGVF). He contributes to preparation and execution of the system level verification campaigns. He also oversees projects to study innovative and competitive products in the PNT domain, specifically in timing and authentication.

Daniel Blonksi is the System Performance Engineer in the Galileo Project Office at ESA – ESTEC in Noordwijk, The Netherlands. He coordinates all system performance activities in the Galileo Project.

Francisco Gonzalez is the Navigation Performance and Verification Engineer in the Galileo Project Office at ESA – ESTEC in Noordwijk, The Netherlands. He coordinates the Galileo on-board clock characterization activities.

Enrico Spinelli is the System Performance Verification Engineer the Galileo Project Office at ESA – ESTEC in Noordwijk, The Netherlands. He is the Technical Officer of the Galileo System Evaluation Equipment (GALSEE). He is also the main point of contact for aviation and standardisation related matters, and oversees projects to study innovative concepts and products in the PNT domain.

Richard Swinden is the Navigation System Engineer in the Commercial User Segment and Navigation System Validation Section at ESA – ESTEC in Noordwijk, The Netherlands. His current responsibilities in the Galileo Project Office include the development, integration, and verification of the Time and Geodetic Validation Facility (TGVF), as well as the coordination of in field GNSS receivers testing activities.

Rafael Lucas Rodriguez is the Galileo Services Engineering Manager in the Galileo Project Office, Directorate of Navigation at ESA – ESTEC in Noordwijk, The Netherlands.

Marco Falcone is the Galileo System Manager in the Galileo Project Office, Directorate of Navigation at ESA – ESTEC in Noordwijk, The Netherlands.

 

IGM_e-news_subscribe