# Evaluate TPS XML API

Source: [https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html](https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html)

Before testing the Terrestrial Positioning Service (TPS) XML API on a large scale, it's necessary
      to evaluate the accuracy and yield of Wi-Fi and cellular ID-based services correctly.

## Perform field test and collect data

Source: [https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html](https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html)

The best approach for testing the TPS XML API is to conduct testing in the field where real-world
      errors can be determined. Qualcomm has the tools and applications required to help with field
      testing, data collection, and analysis.

Consider the following guidelines and recommendations to achieve consistent and reliable test
      results:

- Test in various morphologies such as dense urban, suburban, deep indoor, and indoor.
    - A mix of 70% indoors and 30% outdoors is recommended to account for devices being
            indoors most of the day.
- A statistically significant sample size is required to get an accurate picture of the
        location services.
    - At a minimum, take several thousand samples in a given city. If this is not achievable
            due to limitations in resources or individual city access, contact the Qualcomm TPS team
            to determine a fair sample size and location criteria for the specific test area.
- When choosing test points, ensure that an accurate ground truth location can be determined
        and recorded.
    - Use street corners or other recognizable landmarks for testing. Online satellite and
            vector map services can help to determine ground truth.
- After selecting the test point, ensure that the device isn't in motion and record the
        following information:
    - Time
    - Real location (ground truth), where the test sample is taken via a mapping
            service
    - Result of location system
- After recording all the information of the test location, compare each location result to
        the ground truth to determine individual location errors.
    - Compile these individual location errors into a Cumulative Distribution Function
            (CDF). See [Analyze test results](https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html#Guidelines_to_analyze_test_results_29) for more
            information.

## Use GNSS/GPS for ground truth

Source: [https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html](https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html)

With recent improvements in GNSS and receiver technologies, GNSS/GPS is often used to determine
      accurate ground truth. However, using GNSS/GPS for ground truth isn't advised in many cases as
      it can be inaccurate in urban areas due to the urban-canyon effect of multipath. In such
      cases, it's possible that the GNSS receiver suggests that its accuracy is good, but in reality
      it isn't.

Performing field tests particularly on a large scale may not always be possible due to
      constraints in resources, time, and cost. However, high-quality GNSS reference points may
      mostly be available in areas where network location is least valuable such as outdoors in
      areas with clear view of the sky.

When determining the quality of a location system using GNSS/GPS, consider the following
      guidelines and recommendations to achieve consistent and reliable test results. These results
      are used for replaying RF signal environment scans and comparing them to GNSS as a proxy for
      accurate ground truth.

- Select a bounding box for the test area with enough device traffic and assumed access
        point/cell coverage to remove signal scan and data processing variability.
    - This bounding box can be in metropolitan areas or suburban environments where
            network-based locations are most valuable to a system but also have the potential for
            accurate GNSS reference locations.
- Ensure that the GNSS samples used indicate that the location result is highly accurate.
    - Use GNSS samples where recommended GNSS uncertainty is ≤ 10 m and the number of
            satellites reporting is ≥ 8.
    - Remove GNSS samples that indicate that the device is likely in motion. This includes
            GNSS attributes such as speed < 0.5 m/s.
- Ensure that there is minimal delta time between the GNSS fix used and the Wi-Fi/network
        information replayed for that sample.
    - For example, the recommended delta time between the GNSS fix and the corresponding
            Wi-Fi scan is ≤ 1.
- Ensure that the data use for replays is as recent as possible.
    - In general, Wi-Fi access points are moved and reused, and cell identifiers can be
            rotated or changed by operators. These updated locations are reflected in the most
            recent database of location system providers.
    - Using the most recent possible data reduces the risk that access points and cell IDs
            may have either moved or changed during the time between data collection and the
            replayed requests.

## Analyze test results

Source: [https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html](https://docs.qualcomm.com/doc/80-42216-1/topic/26_Guidelines_to_evaluate_TPS_XML_API.html)

Comparing results from different location systems can be difficult due to varying algorithms and trade-offs between yield and accuracy. Here, yield is the percentage of time a location is returned and accuracy is the distance to ground truth.

A few common industry standard methods for improving accuracy metrics over yield are as
      follows:

- Don't include an error value for individual location requests that produced no or poor
        results.
- Don't include an error value for individual location requests for single access point
        locations or low number of access points.

Qualcomm implements algorithms and fallback logic to maximize yield and provides the best location solution suitable for all circumstances. Compared to returning a failed location in most cases, this solution provides the best possible location of any sort. OEMs can use the best available location result depending on the use case, and if necessary, filter out locations that are above a certain error estimation value.

Though it's difficult to compare different systems, it's a necessary step to choose a technology.
      Consider the following guidelines and recommendations to compare systems as equally
        possible:
- Select a bounding box for the test area with enough device traffic and assumed access
          point/cell coverage to remove signal scan and data processing variability.
    - This bounding box can be in metropolitan areas or suburban environments where
              network-based locations are most valuable to a system but also have the potential for
              accurate GNSS reference locations.
- Ensure that the request or test sample size of a given region is large enough to
          evaluate the systems appropriately.
    - Though it depends on the available data, time, or bandwidth of the tester, it's
              recommended to take several thousand samples for each region evaluated.
    - Sample counts should accompany test results to understand the scale at which the
              solutions were tested in each region.
- Ensure to evaluate both accuracy and yield in a given region.
- To analyze accuracy, calculate the distance to ground truth (error) in each individual
          location response.
    - Use these values to calculate and plot a CDF in increments of 10% across the entire
              set for each test region or in aggregate.
    - For example, an accuracy value of X meters at the 50th percentile indicates 50% of
              locations in the set are either equal to or less than error X.
    - Include the 67% as this represents 1-sigma or 1 standard deviation from the
              statistical mean.
- Standardize an error value for each failed location result.
    - It's important to not skew accuracy data in higher percentiles for locations that
              can only be returned for a given system.
    - Set a large distance for this error value, for example, 100,000 m or greater.

Last Published: Apr 01, 2026

[Previous Topic
Sample TPS XML API request](https://docs.qualcomm.com/bundle/publicresource/80-42216-1/topics/21_Sample_TPS_XML_API_for_testing.md) [Next Topic
References](https://docs.qualcomm.com/bundle/publicresource/80-42216-1/topics/references.md)